[LINK] Killer robots and military ethics

[Roger Clarke]

At 15:43 +1000 18/7/13, Craig Sanders wrote:
>  ... robot weapons do what they're programmed to do ...

A refinement (not a correction, because I'm sure that Craig is well 
aware of this!):

What a programmer intends a device to do is not necessarily what it does.

Put another way, what a device is programmed to do means 'what the 
code does in any given constellation of circumstances'.

This may be identical to, similar to, a bit different from, or very 
different from:
(a)  what the designer (if there was one) envisaged;  or
(b)  what the designer (if there was one) would have envisaged, if they
      had ever contemplated that particular constellation of circumstances

It's necessary to say 'if there was one', because a great deal of 
code development, of deep-nested libraries and even of upper-layer 
applications, is done by coders on a let's-try-this basis, with 
iterations of changes until it (seems to) work.  Of course, no such 
approach is ever adopted in any 'military-grade' code ...

It used to be moderately difficult to phrase these things back when 
there were only machine-languages, assembly-languages and procedural 
languages.

But then we got clever and invented declarative, descriptive and 
facilitative languages (and lots of arguments about what to call 
them).
See http://www.rogerclarke.com/SOS/SwareGenns.html#ASGLA

Basically, if the programming involves rules, or better still just a 
bunch of examples that facilitate machine-learning, then:
(a)  there is arguably no designer, and/or
(b)  there is very probably no conception by a designer of a great
      many of the contingencies that devices will encounter during
      the lifetime of the program

So yes, it's rather scary.

And Asimov, Arthur C. Clarke (no relation) and others investigated it.

Which is why I wrote that dry, derivative, but potentially valuable 
piece all those years ago: 
http://www.rogerclarke.com/SOS/Asimov.html

It's accumulated a couple of hundred citations since 1993, despite 
being a rather obscure topic.


-- 
Roger Clarke                                 http://www.rogerclarke.com/

Xamax Consultancy Pty Ltd      78 Sidaway St, Chapman ACT 2611 AUSTRALIA
                    Tel: +61 2 6288 1472, and 6288 6916
mailto:Roger.Clarke at xamax.com.au                http://www.xamax.com.au/

Visiting Professor in the Faculty of Law               University of NSW
Visiting Professor in Computer Science    Australian National University