Andy's post on a mailing list today reminded me of something I meant to blog awhile ago, its a nicecogent engineering summary by Henry Petroski of the Minneapolis 35W bridge that fell (I drove across the day before it happened, and thought this is actually a really good shortcut, I should come this way more often) (emphasis added):
Engineers often look to examples of success and failure to guide their designs. Paradoxically, it is the failures that are the more reliable teachers. As the case of the Minneapolis bridge so clearly shows, a structure that stands successfully for decades is not necessarily a sound design. However, when a bridge fails, it provides invaluable lessons in what not to do.
...
No matter how carefully bridge designers anticipate failure on the drawing board (or computer screen), their structures will only be as reliable as how carefully built, maintained and inspected they are. Just because a bridge has given decades of successful service under adverse conditions of increasingly heavy traffic and neglect does not mean that it will continue to do so. It is the function of regular and careful inspections to catch what designers might not have anticipated.
In the wake of Minneapolis, there will no doubt be renewed vigilance. More careful inspections and more conservative interpretations of their findings may cause some immediate inconveniences, but they will also likely prevent some imminent failures.
In bridge design, as in all structural engineering, success can breed hubris and catastrophe, while failure nurtures humility and caution. Unfortunately, it does seem to take a collapse to re-sensitize inspectors and operators to the real dangers that lurk among rusting steel and cracking concrete. Let us hope that the lessons learned in Minneapolis are not forgotten once more.
This has broad and deep implications for information security, to touch on two quick ones a specific example and a more general one. First the specific one, I have blogged a number of times about the MQ Series security issue. I see it as one of the most important (and least discussed) enterprise security issues. In a nutshell, MQ Series is often the bridge to the mainframe that runs the business, when it was initially deployed, it was early days of the Web or even pre Web at some companies, so the internal user only logic held sway and millions and millions of queues were created and attached to company mainframes with authorization but not authentication. This was a questionable idea, but pre-Web you wouldn't get too much of an argument. Problem was, the new Web apps need the data on the mainframe, so what did they do? Connected them via queues, "the way we've always done it."
But MQ Series was designed for a benign environment not a hostile one. Because the mainframe plays a central role in many companies' culture they continued to connect the way they always had, and the inspectors (auditors, pen testers) didn't really notice because they focus mainly on the front door, but lo and behold, we knew there was a cloud in front of the web app, but there was also a cloud inside, and it didn't use authentication and it was directly connected to the mainframe, the crown jewels. We now have a second cloud problem - the cloud inside
This is a great illustration of what Petroski described - assuming that because "its the way we 've always done it" means that it should work even in the face of massive technical and business changes. Secondly, the inspection and assessment does not take into account the new use cases and deployment environments. So the failure comes in both new development based on ritual rather than engineering, and in assessment that's rooted in pre-Internet thinking. I really, really hope this story is not repeated in the new cloud apps being built out now
MQ 7.0 now has an HTTP interface. The documentation mentions how this can be used from AJAX applications. That takes a couple of hops out of your diagram :)
Posted by: Dave Tauzell | April 02, 2009 at 09:56 AM
hits head against desk
Posted by: Gunnar | April 03, 2009 at 09:21 AM