ANALYSIS: Will Companies Keep AI Models Fed With Employee Data?

Two topics have dominated business headlines throughout 2023: workers’ rights and generative AI. In 2024, they will converge. As companies clamber to capitalize on AI and become increasingly desperate for high-quality data to train AI models, many will look to their own employees’ data—in the form of both work product and personal information—as a source to consider, due to the dearth of regulation regarding its use and the oceans of it available to most companies.

Today, as legislators debate how much privacy an employee should reasonably expect at work, employers amass huge stores of data on everything an employee does—from their web browsing patterns to footsteps taken to even their mood.

This lack of regulation on the immediate horizon breeds a problem with few solutions: there is little preventing employers from risking their employees’ privacy by feeding their personal information to new, unregulated AI technology for the sake of profit and innovation, and giving employees little agency in this process.

Companies Need Proprietary Data to Train AI

Creating useful, robust data sets that are capable of training sophisticated AI models requires massive amounts of high-quality data. Recently, the legality of the method of web scraping used to create many of the data sets behind popular AI products like OpenAI‘s ChatGPT and Dall-E has been challenged, provoking companies interested in building proprietary AI models to look elsewhere for training data.

Leveraging data that a company already owns would give that company an easy solution to this problem. And the bulk of the data that is best suited for training AI will be related to its own employees.

A company’s employee privacy policy—if it has one at all—will usually list what data is collected by the company, how it is collected and used, and who it will be disclosed to.

In these policies, the types of personal information a company collects can be very specific, while the disclosed purpose for its collection can be broad and vague. For example, Nike’s employee privacy policy lists “airplane seat preference” as one of the hundreds of data points they collect from employees. Contrast this, for example, with the catch-all purpose tacked on to the end of software company GitHub’s policy: “Other purposes permitted by applicable privacy and data protection legislation.”

The Limited Scope of Employee Data Protection

Some specific categories of employee data are protected by either federal or state laws that restrict how the data can be handled. For example, some health information collected in the course of employment is protected by HIPAA and can only be used for purposes allowed by law.

But when it comes to non-sensitive, non-health-related data, existing applicable privacy laws focus more on transparency rather than establishing limitations. Generally speaking, if an employee is given notice before their data is collected, employers are granted an incredible amount of freedom.

Two state-specific laws do provide limited protection for employees: the California Consumer Protection Act, which was recently amended by the California Rights Privacy Act (CCPA/CPRA) to include employees in its scope, and the Illinois Biometric Information Privacy Act (BIPA).

Both laws limit how personal information can be handled, which in turn limits how employers can use employees’ personal information to train AI models. But neither is a perfect remedy. As state laws, they cover only a small fraction of the population, and neither was specifically written to address this issue. Meaning that, if an employer chose to, the law is unlikely to prevent it from using most employee data to train AI models.

Still, BIPA is being used increasingly to push back on company collection of employee biometric data and workplace AI surveillance. Two BIPA lawsuits against Amazon specifically allege the company used employees’ facial geometry to train the company’s Rekognition AI. And the California Attorney General in July announced an investigative sweep to determine if California employers were in compliance with the law.

It is also important to consider the Federal Trade Commission, which employers would have to face if a future court were to determine that using employee data this way is illegal.

Algorithmic disgorgement is a legal remedy used by the FTC to prevent companies from profiting from deceptive data practices. It would require any artificial intelligence-powered algorithms built using illegally obtained data to be completely destroyed.

On-Point Legislative Action Is Unlikely in 2024

The US has no comprehensive federal personal data protection law and most states explicitly exempt employees from the scope of their consumer data privacy laws. It is unlikely legislators en masse will suddenly redirect their attention to pass laws specifically protecting employees from having their personal information exploited to train AI models in the near future.

Relevant state and federal legislative proposals aimed at protecting employees from the ill effects of AI largely focus on preventing discrimination related to the use of the technology in hiring practices or limit its role in the growing shift toward extreme workplace surveillance. These proposals provide only incremental progress toward granting employees agency over their personal information in an employer’s possession.

Implementation of these laws would limit the sources from which an employer could collect data from their employees. But it wouldn’t address how data already in employers’ possession, or other lawfully collected personal information, could be used.

Collective Action Is the Most Visible Deterrent

In the current legal landscape, the only clear check on employer AI use appears to be in the area of labor relations. For example, the National Labor Relations Board issued a memo last year warning employers not to use AI to engage in unlawful surveillance of union activities.

And with unionization efforts gaining momentum, a growing number of labor relations issues are touching on the threat of emerging technologies.

The AI-related demands negotiated in the Hollywood writers’ and actors’ strikes foreshadow widespread conflict in 2024, as more industries identify employee data as an abundant source of exploitable intellectual capital and rush to develop largely unregulated proprietary AI models.

These strikes reflect many workers’ own AI-related anxieties: How can employees protect their likeness, work product, and intellectual property from being used to train the technology that could eventually replace them?

In nonunion workplaces, employers that choose to use employee data to train AI models may end up pushing workforces to turn to collective action for protection. Short of this, employers risk alienating workers, harming morale, and contributing to higher turnover.

AI Anxiety Persists

Uncertainty around the status of workers’ rights have come to the forefront of the public consciousness at the same time as most people’s first exposure to generative AI. Both will combine and contribute to a miasma that will persist into 2024.

With public concern about AI’s workplace implications mounting, employers might find themselves wanting to step more carefully into this emerging area.

Access additional analyses from our Bloomberg Law 2024 series here, covering trends in Litigation, Transactions & Contracts, Artificial Intelligence, Regulatory & Compliance, and the Practice of Law.

Bloomberg Law subscribers can find related content in our In Focus: Artificial Intelligence (AI) page, Practical Guidance: AI in the Workplace, and the Workplace Privacy Toolkit.

If you’re reading this on the Bloomberg Terminal, please run BLAW OUT <GO> in order to access the hyperlinked content, or click here to view the web version of this article.