Imagine a future where algorithms or even supercharged artificial intelligence (AI) make decisions in multiple aspects of your life: your job, your education, your welfare, and even your health.
In this future, the police use algorithms and AI systems to predict where crime will be committed and automated alerts trigger police to despatch vehicles. Police vehicles are fitted with automated number plate and face recognition cameras that identify people on watch lists. Police can ‘stop and scan’ people to verify their identity, using on-the-spot fingerprint scanners to check against crime and immigration databases. Following arrests, an algorithm can assess information held about suspects and decide whether they should be kept in custody.
Meanwhile, the intelligence agencies use automated programs to sift through billions of communications intercepted from entire populations, home and abroad, every day. Their programs automatically read, listen to, and watch private conversations and web browsing, advising who to subject to more intense surveillance.
That future is here and now.
As the trend for automation takes pace, our lives and indeed our freedoms risk being increasingly governed by machines. New technologies provide great promise in a range of sectors, from science to transport to health – but when automation is used to make decisions about citizens’ basic rights, the risks are extremely grave.
European law provides us with the vital right not to be subjected to automated decisions. However, our Government is abandoning this vital right in the Data Protection Bill currently going through parliament – opening the door to employers, authorities and even the police using machines to make life-altering decisions about us.
Is it right to rely on machines to decide who is eligible for a job; who is entitled to welfare; or even, who is innocent or guilty?
Unsurprisingly, automated computer programs don’t tend to deliver humane solutions.
An automated benefits system in the US resulted in a million benefits applications being denied over a three year period – a 54% increase from the three years before. It often blamed its own mistakes on claimants’ “failure to co-operate”. One such claimant was a woman suffering ovarian cancer. Without welfare, she lost the ability to pay for her medication, her transport to medical appointments, and …