5

Are there any real-world examples of unintentional "bad" AI behaviour? I'm not looking for hypothetical arguments of malicious AI (AI in a box, paperclip maximizer), but for actual instances in history where some AI directly did something bad due to its direct action that was not intended behavior.

Interpretations of the meaning of "AI", "bad", and "unintentional" are left open due to obvious reasons. Be free with your interpretation, but use some common sense please.

Ex: Microsoft Tay became a racist not too long after being hooked up to the internet, thanks to internet trolls "teaching" her bad things.

I can't think of any other instances. So the following examples are just hypothetical scenarios to demonstrate what I mean.

Ex: A self-driving car drove off-track after being presented with an adversarial example, crashing into people.

Ex: Surgery bot goes haywire and accidentally kills someone.

Ex: Weaponized drone targets civilians against its design.

k.c. sayz 'k.c sayz'
  • 2,121
  • 13
  • 27

2 Answers2

5

There is the case of the tesla accident where the car was in autopilot and crashed into a truck because it appears the vehicle mistook a lightly coloured truck for the sky, killing the driver: https://www.newscientist.com/article/2095740-tesla-driver-dies-in-first-fatal-autonomous-car-crash-in-us/

Having said that, it appears the car had been trying to tell the driver to pay attention: http://uk.reuters.com/article/us-tesla-crash-idUKKBN19A2XC

redhqs
  • 291
  • 1
  • 5
4

The infamous Flash Crash of 2010 may qualify.

It didn't involve Artificial General Intelligence (which is still a hypothetical) or even "strong narrow AI" (such as AlphaGo) but does involve algorithmic decision-making, which is a form of basic Artificial Intelligence.

Algorithmic trading already represents a significant percentage of all market activity, and I suspect that percentage will only increase.

From Business Insider: Algos could trigger the next stock market crash

DukeZhou
  • 6,209
  • 5
  • 27
  • 54