AI threats, action and regulation

“The development of full artificial intelligence could spell the end of the human race.”
Stephen Hawking, 2014

AI is the most divisive issue since the propriety of pineapple on pizza. Is it a genuinely transformative force for good, or a precursor to robot slavery? Will it make us all redundant, ultimately end life on earth, or improve our lives?

While industry critics and wannabe tech-philosophers ponder such questions, there can be no doubt something big is happening. AI is enabling medical breakthroughs, accelerating scientific research, helping to combat climate change, detecting vulnerabilities and cyber threats, automating routine and repetitive tasks… the list goes on.

Conversely, AI can require the large-scale collection of personal data to train and run. There are risks of job displacement, misinformation, deepfakes and hallucinations. There’s concern so much power rests with such a small number of ‘Big AI’ platforms, whose rogue AI agents appear to be able to escape their digital confines. Thousands more AI firms are emerging globally all vying to challenge big tech. AI is powering ever sophisticated cyber-attacks, with developers breaking ranks to warn of impending doom.

Meanwhile governments, policymakers, regulators and organisations are chasing their tails trying to put guardrails in place. More than a few of us, I imagine, are simply trying to keep up with what’s really going on.

As we sit on our sofas eating popcorn scrolling Instagram, a turbocharged contest between cyber attackers and cyber defenders is unfolding. The risks of AI going wrong becoming an ever-pressing priority for those in power. It really is the stuff of a Ridley Scott script.

So, with the news agenda packed with AI stories, here’s my roundup of those which caught my eye and I thought would be interesting for data protection folk.

Cyber risk intensifies

Heard of the PHP code flaw? It wasn’t a cyber-attack, but a vulnerability affecting the language underpinning around 70% of world’s websites. A flaw which has led to governments and companies shutting down web platforms as they rush to implement a critical patch. A flaw which was only identified by using frontier AI models which bad actors are no doubt using AI to try and exploit.

This came just days after Sir Ken McCallum, Director General of MI5 warned of “AI’s potential to change the stakes on existing or even new bioweapons” and told us terrorists have already tried to use AI to improve reconnaissance and plan attacks.

The National Cyber Security Centre (NCSC) has repeatedly warned AI is making cyber-attacks more effective and more frequent. AI has meant the time between a flaw being disclosed and attackers exploiting it has shrunk to days, making prompt patching critical.

That’s even before I mention rogue AI agents. In June, an OpenAI agent breached Australia’s Medicare statistics reporting service. In July, AI agents escaped their testing environment and breached the systems of the Hugging Face platform. And there are reports of other Big AI platform’s agents going rogue and escaping out of the guardrails built to contain them.

As much as AI can be used to attack, it’s equally proving vital in combatting attacks. It can analyse huge volumes of network activity, logins and systems logs to spot unusual behaviour in real-time, catching attacks other tools miss. It can summarise incidents, take automatic actions and cut response times from hours to minutes. It can be used as an intelligence tool, scouring dark web chatter to predict emerging trends and the tactics attackers might use.

UK Government and AI

When he took up the post of Prime Minister in July, Andy Burnham signalled the importance he places on AI by appointing Kanishka Narayan as the UK’s first Cabinet-level Minister for Artificial Intelligence. Narayan recently said the Government will be looking to legislate wherever necessary to mitigate the risk of AI threats.

The Chancellor John Healey has announced an AI security package, including £165m for a National AI Biosecurity Initiative, an extra £120m for the AI Security Institute, and £130m for AI-driven cyber defence and vulnerability detection, alongside further funding for sovereign AI and local AI partnerships.

Plans are afoot to strengthen the Cyber Security and Resilience Bill, with a statutory obligation to report AI incidents, including near misses. The Bill, which is reaching Report Stage in the House of Lords, is expected to be enacted in early 2027.

Meanwhile the Department for Digital, Culture, Media and Sport (DCMS), the NCSC and the AI Security Institute are said to be conducting rigorous assessments of advance AI models and looking to develop ways to contain agentic AI incidents. Read more from the Government here.

There’s a balance for the UK government, or any government, between trying to control AI, to protect against the risks, while still enabling innovation and reaping economic benefits. On the latter, AI could play a central role in Andy Burnham’s efforts to build closer ties with our EU neighbours.

Against the backdrop of concern about UK dependence on US technology, there’s a new fear British firms could be shut out of the European tech market by new rules. The proposed EU Cloud and AI Development Act, expected to take effect in early 2028, would require European governments to prioritise European companies when procuring AI technology. On this, the PM may be hoping he can persuade the EU to give British firms a form of trusted status.

What’s not yet clear is whether the Burnham government will focus solely on backing British firms or whether it supports UK AI regulation along the lines of the EU’s AI Act. A clearer picture may emerge after the UK-EU summit in Brussels on 20 November.

But as the quote famously attributed to Harold Macmillan goes: “Events, dear boy, events.” The agenda may well have shifted by then.

AI and UK healthcare

The UK may not have an overarching AI regulation (yet), but in early October the Government accepted every recommendation made by the National Commission into the Regulation of AI in Healthcare.

The Commission’s main conclusion is AI-enabled medical devices must be monitored throughout their working life, not just at the outset. The Medicines and Healthcare products Regulatory Agency (MHRA) is expected to issued draft guidance by the end of the year on how to manage AI devices and tools, and will consult next year on how these are qualified and classified.

The Government says it’s committed to strengthening patient engagement, making it easier for patients to complain or seek compensation if things go wrong, and improving transparency about how AI is used in their care.

Suppliers and NHS trusts will be expected to fully assess the lawfulness and risks of using AI tools before and during deployment, along with raising the bar on making sure patients are fully informed of AI use.

This link to the right to be informed brings me on to the right to object.

44,000 people have exercised their right to object in one of the largest co-ordinated uses of this right under UK GDPR. These tens of thousands are calling on NHS England to stop Palanatir’s Federated Data Platform (FDP) from storing, using or sharing their health records.

Palantir is a US data analytics and AI company. The FDP it built for NHS England is described as a data integration and analytics platform, and the contract officially bars Palantir from training its AI on NHS data. However, many are sceptical, a view fuelled by comments such as those of Liberal Democrat MP Martin Wrigley, who described the FDP as a “data-hungry AI solution”.

The Financial Times recently reported more than 20 NHS Trusts have stopped using FDP. The seven-year multimillion pound contract was signed with Palantir in 2023. A key decision is looming for the Government, as it could trigger a break clause in February 2027.

Information Commission and AI

Meanwhile, the ICO has hailed a breakthrough in its work with foundation model developers operating in the UK. It says ten developers have made, or have committed to make, key data protection changes. These include enhanced transparency information, more robust safeguard assessments and stronger mechanisms for people to exercise their privacy rights.

The ten developers are Amazon, Anthropic, Apple, Cohere, DeepSeek, Google, Meta, Microsoft, OpenAI and Stability AI, and the ICO says it will continue to monitor developments closely and engage with industry. Read more from the ICO here.

The ICO has also published a set of expectations for the development of generative AI-models covering areas such as transparency, lawful basis, data subject rights, models, personal data and the regulator’s position on how special category data can be used lawfully. It also launched an Agentic AI call for evidence, which closes on 20 November 2026.

All of this will shape the forthcoming statutory AI Code. In a similar way to the ICO’s Children’s Code and the Data Sharing Code, this will set a benchmark for what UK good practice looks like. The AI Code is expected to be finalised during 2027.

Action under EU AI Act

Across the channel, in its first test of enforcing the AI Act’s rules, the European Commission’s AI Office has sent out formal requests for information to Open AI and Anthropic. These focus on the incidents of rogue AI agents escaping their testing environments and gaining access to outside organisations. The AI Office took this action after judging the companies’ initial voluntary disclosures to be insufficient. We’ll have to wait and see how much muscle the regulator has in practice, not to mention any pushback from the AI companies themselves.

EDPB statement on smart glasses

AI is even on people’s faces. It seems almost trivial to talk about smart glasses, when we face AI cyber threats and the potential extinction of life on earth. Nonetheless, as they’ve been so hotly debated, it seemed appropriate to mention we now have an official statement from the European Data Protection Board.

The Board says purely personal uses of the glasses, for example filming the family or real-time translation, will fall within the household exemption. However, the Board says wearers will become controllers under GDPR if they share footage on social media platforms, or if the glasses are used for commercial, official or political purposes. It will be interesting to see how this plays out. Smart glasses, smart policies.

There are simply too many AI stories to cover them all, so this has been my pick. A little snapshot of good AI detecting critical vulnerabilities, bad AI exploiting them, naughty AI agents staying up past bedtime, and the not-so-personal use of ‘pervert’ glasses. Perhaps only one thing is certain, there will be plenty more events, dear boy, before the year is out.