Fuck AI

A podcast I listened to recently spoke about failure modes of AI. They used an example of a toll bridge in Denmark where it was impassable recently because it only took card payments, and the payment processing system was down. It would be sensible in this scenario for the failure mode to be for the toll barrier to be open and for them to just let cars through if technical problems means it's impossible for people to pay the toll. Unfortunately, this wasn't the case, and no-one had the ability to manually make the barrier go up. Apparently they ended up having to dismantle the barrier while the payment system was down.

This is very silly, and highlights one of the big dangers of how AI systems are currently being used (even though this particular problem doesn't have AI involved, I don't think, just regular tech problems). The point is that tech can be awesome at empowering us, but we need to think about "okay, but what happens when things go wrong?", and we need to be asking that question in a manner that puts humans at the centre.

That was a far more trivial scenario than the situation described in the article. If AI tools help improve detection rates, then that's awesome. But we need to actually address what happens if those technologies cease to be available (whether because the tools rely on proprietary models, or power outages, or countless other ways that this could go wrong)