[LINK] People Make Poor Monitors for Computers

[Ivan Trundle]

On 12/04/2012, at 7:50 AM, Kim Holburn wrote:

>> So how can they perform adequately when they are called upon to reenter the control loop? The evidence is that this task has become so alien and the system so complex that, on a significant number of occasions, they perform badly.

With regard to aeroplane systems (looking at total control from take-off to landing): there are two main factors at play.

The first is that, increasingly, automated or 'intelligent' systems (I use that term advisedly, since the intelligence is only programming) are removing more opportunities for human control initiatives. This is becoming more apparent in these systems, and it's not a trivial matter to determine the path forwards.

The second is that the systems are inherently very complex, and very difficult to comprehend even in theory, let alone when in a crisis. Many systems make choices on factors either outside of human observation, or understanding (especially in fly-by-wire planes, but also in automated systems which ensure that planes do not bump into each other). The systems are capable of making decisions based on factors which humans are unable to comprehend at a time of crisis, nor can humans be placed in a position to approve such actions.

The reliance on computer systems alone is not yet considered safe enough in the aviation world. 

Crew/pilots/controllers all have to be part of the loop of information, and systems have to have redundancies to ensure that the likelihood of catastrophe is minimised. Even the humans require redundancies (which is why we still have co-pilots).

It might be of interest to note in the recent A380 engine explosion, all of the computer power available gave readings which were based on incomplete data, and the computational calculations derived from the available data deemed that a successful landing on the chosen runway would be impossible. It would be hard to predict what an automated 'intelligent' system would have chosen to do under these circumstances.

iT