Tools, Specialists and Faith
Quote from Michael Nygard – The Future of Software Development”:
"Speaking of languages, many of the problems we face today cannot be solved inside a single language or application. The behavior of a web site today cannot be adequately explained or reasoned about just by examining the application code. Instead, a site picks up attributes of behavior from a multitude of sources: application code, web server configuration, edge caching servers, data grid servers, offline or asynchronous processing, machine learning elements, active network devices (such as application firewalls), and data stores. "Programming" as we would describe it today--coding application behavior in a request handler--defines a diminishing portion of the behavior. We lack tools or languages to express and reason about these distributed, extended, fragmented systems. Consequently, it is difficult to predict the functionality, performance, capacity, scalability, and availability of these systems."
I couldn’t agree more with this…
The amount of expertise and know-how that can be needed to implement, understand and maintain what can appear to be “a simple website” is usually underestimated.
It really is amazing what can be lying under the hood of a “web page”, and the amount of knowledge needed to fully control and understand the consequences and the outcome that will be triggered by the execution of your code:
Distributed transactions, dynamic deep domains, load balanced clustered black box environments, 3rd party constraining libraries, stateless http legacy protocols, evil deadlock-friendly isolation levels , resource bottlenecks, legacy client browsers, classloader hell…
When a problem arises, usually in the form of an unexpected behavior, the hunt starts for any information that can help understand exactly what is going on.
As Michael states, “coding application behavior in a request handler–defines a diminishing portion of the behavior”.
Sometimes it is quite astonishing the number of things that must go right just so a simple http request can be processed in the way we want:
First of all, a cross-browser friendly piece of javascript needs to be correctly interpreted by any of many equal-but-totally-different browsers. If this already complex first step is ok, then, somehow, the request must be almost miraculously routed and guided through an almost infinite network of broken nodes and dead ends to a hopefully responsive socket. Supposing the request has actually been able to trigger the spawning of a thread on a valid member of a clustered environment, the clock starts ticking on all sorts of request, session and transaction timeouts that have to be beaten in order to correctly access, use and close all sorts of different resources, and produce a usable and expected response, that must of course be somehow guided back to the original startpoint.
Multiply this sequences of events by the hundreds, add transactionality, performance and security, and it is not difficult to imagine the madness that occurs in a shared memory multithreaded environment such as a JVM….
It is also not surprising that the number of elements that have to be perfectly synced and configured to work together in order for the global system to work, can be frightening…
From what I have seen so far, only specialization plus information gathering, through a vast array of tools, can help.
All sorts of sources must be gathered and delivered to the right experts for analysis: transaction logs and deadlock dumps for DBAs, thread dumps to java engineers, system logs for operations…
It is not always possible to implement simple solutions: sometimes complex system implementations are inevitable.
And when this is the case, interesting challenges always appear before being able to offer a “simple web site” that “just works”.
No related posts.
Tags: I.T Evolution, Michael Nygard