Is YOUR data training THEIR AI?

In our personal and working lives we’re under a barrage of notifications inviting us to use new AI functionality. Sometimes it’s not even a feature we actively turn on, it’s automatically on by default and we need to take steps disable it or opt-out.

The problem is we often don’t know how this AI actually works, what it might do, what data it uses, or what data is used to train it.

Recently LinkedIn announced it has started sharing user generated content for LLM training. While you may be happy with this, for those that aren’t, you need to actively go into your data privacy settings and switch it off.

More broadly, many organisations are being encouraged to take advantage of shiny new AI capabilities offered by their existing software providers. HR, Finance, IT and CRM software can seemingly do so much more if the latest AI tool is enabled.

It’s very tempting to give it a try. And I suspect many data protection teams are struggling to keep up with parts of the business which have drifted into using AI tools without much deliberation.

We need to be aware using AI for seemingly innocuous purposes can have unexpected consequences. We’ve written about the risks to consider when using AI to transcribe or record meetings.

Did you know there’s a feature on MS Teams which can automatically detect when an employee is connected to the company wi-fi and update their location to ‘in the office’? This may seem like a simple and useful feature to switch on. But in essence this is a form of workplace tracking, and may raise some considerations, not least is it proportionate and lawful?

Even with our existing suppliers, we’d be wise to conduct some due diligence. AI functionality isn’t always a straight-forward extension of an existing service.

We should assess the benefits and risks, be clear about our objectives and whether we are a controller or joint controller. Make sure our activities are lawful, fair and transparent. Be sure our data is still being processed by the same party and in the same country. Understand if our data will be used to use to train the software providers’ models, and where data is anonymised or aggregated, be confident this is effective enough in preventing risk.

It may feel daunting, but we should try and have some level of understanding about how a third-party supplied AI system works. Ultimately, we’re responsible for complying with data protection law for any personal data we allow to be used in or by an AI system.

Recently I was reviewing the AI usage of a client’s existing software provider. They were ambiguous about the use of the client’s personal data to train their own models. It became clear processing was no longer taking place in Ireland, but in the United States and India. I’ve seen other AI software where it’s transpired it’s not been developed by the software provider themselves, but they are using AI provided by a third party (who the data is shared with). Which made me wonder; is the AI provider using that data for their own purposes, such as AI training?

Ideally we should be asking AI providers, whether they be new or existing suppliers, to work with us to conduct a Data Protection Impact Assessment. If they’re reluctant to help, or not able to answer key questions, this might raise concerns.

I’m not saying all AI tools are inherently a bad thing. There are many benefits to be gained! Just do some digging, and keep your eyes open. How to govern your organisation’s use of AI