« Google and Baidu | Main | Fear the Boom and the Bust - Keynes v Hayek rap »


Adrian Lane

OMG ... so they had no forensic data to review because it was lost in the blast. You forget about the possibility of complete data loss at the time of the incident if you leave data at the local collector for any length of time. This is a good example of why you want to get the data off the device ASAP, and consider redundancy in tiered data collection models. Log poisoning, explosion, whatever.

Clive Robinson

One thing I noticed years ago about logging and process validation is that developers push error checking to the left and business logic to the right.

That is they try to check everything at the front door whilst doing business out the backdoor with presumed OK input...

The results if you are lucky is two (or more) logs that just don't hang together in a usefull way for diagnoses or measurment.

Worse the logged event usually only has mangled data from previous processing and lacks the original input so little information is available to developers and maintanence staff to trap and walk through an event.

A simple solution is "one step back logging", that also has the original input kept along with the last ok change to the input by the previous stage in the process chain of the business logic to that which is actually currently processing the input (ie stage n-1).

Thus if the current processing stage (stage n) either barfs or does not notify the previous stage (n-1) via exception handaling that it has processed the input properly the previous stage can log both the original input and the input that caused the current stage to barf. Thus the fault (if determanistic) can be followed through the system.

But more importantly it also it alows you to "roll back" on a fault exception, which is way more important these days than it used to be.

If for instance you have a distibuted system (front middle back end etc) if something occurs like the coms link goes down between two parts of the system then this can be switched to an alternative system etc...

This method of dealing with faults migrates from single systems to fault tolerant distributed systems relativly painlessly as the designers and programers are effectivly forced from the outset to do it in a manner that will scale.

As old as the idea is (goes back to when objects where decidedly odd ;) it still appears new to most designers / programers who never seem to consider that they might on exception roll the process back and go down a diferent path.

Clive Robinson

As for BP it's a long while since I was rig hopping.

Old habits die hard one of which is the assumption of "minimal coms" this is despite the likes of the Alexander Keelan, Piper Alfa and a few other disasters. Jackets are viewd as "vessels" not "plant" even though the "captin" is known as the Offshore Instalation Manager (OIM).

I must admit I have a certain degree of sympathy for their position they are just one of a number of very deep sea drillers and have to be competative. and they are all broadly the same thus it could just have easily been a US or other countries oil company in the same position.

Unfortunatly the oil industry is a little like NASA and the space shuttle, they regard each non accident day as proof they are doing it right, not that the dice have not landed on a one.

What is odd is that it appears (from what little has realy been said) that the blow out preventer failed. This is generaly a standard bit of kit and usualy is designed to work in a fail safe manner.

The fact that it has failed due to currently unknown reasons is quite worrying, because of it's standard design that is used not just for very deep sea drilling but all offshore drilling.

What has not helped is the political rhetoric of "Drill baby Drill" that pervaded the US Presidential campaign. It is an indicator of the endemic view point of "get it up and out cheep" so the SUV's can keep rolling on a buck a gallon.

God alone knows what will happen when the antarctic treaty ends and we start drilling in realy hostile waters down there.

The comments to this entry are closed.