AI Companies Are Charting a Path to Lock Users in for Life

Artificial intelligence tools dominating the market of helping people navigate everyday life is making it easier for AI labs to turn to an old tactic to achieve anti-competitive, anti-consumer ends: lock-in.

Lock-in involves creating artificial barriers to make it difficult for consumers to switch products. Some readers might recall when buying a phone locked them into a specific cell service provider. Others with expensive tastes in cars understand the hassle of restrictions on where they can go for regular auto service.

Troubling signs suggest that AI labs are building walled gardens to lock users into using only a particular company’s products. Absent an insistence on interoperability and data portability, users may find themselves reliant on companies that don’t share their values, offer inferior products, or otherwise are no longer right for their business. Given the dramatic expectations for how thoroughly integrated AI products will become in our lives in the near future, this isn’t a trivial or academic concern.

Companies that lock in consumers retain them as customers for reasons other than offering superior services or goods. Many Facebook users, for instance, would likely prefer a platform with a different approach to content moderation and a stronger track record of adhering to its privacy policy.

The absence of pro-competitive, pro-consumer business practices isn’t confined to the social media context. iPhone users have long expressed a desire to use certain third-party apps on their phones. To Apple Inc.’s credit, the technical standards required for an app to operate on iPhones are widely available. Nevertheless, open standards are distinct from open access.

Beyond hindering consumer choice and autonomy, lock-in stifles innovation. If Apple decided to open its marketplace to more app developers, the logical outcome would be an increase in apps and an incentive to develop a range of offerings to appeal to consumers. Instead, app developers with great ideas find themselves outside of the market, banging on Apple’s door to let them in. Apple, of course, would say that its cultivation of the market is in the best interest of users.

Keeping information inside a company’s walls might be warranted in some cases. If, for example, transfer of data from one platform to another would result in the user’s data being exposed to untrustworthy third parties, there may be a good-faith case for maintaining that barrier. In general, barriers to interoperability and data portability should be subject to a high level of scrutiny and maintained only when doing so clearly furthers the interest of consumers.

However, AI labs are collecting and retaining as much personal information as possible while trying to prevent users from moving their data profiles elsewhere.

Microsoft Corp. wants to create models that form “a lasting, meaningful, trusted relationship” with users. Key to that relationship is memory—the ability of a model to gather, aggregate, and act on sensitive, personal information about a user.

OpenAI Inc. is working on the capacity of its models to carry information it gleans from you during one session to the next. The company is beginning to “roll out long-term memory in ChatGPT—a function that maintains a memory of who you are, how you work, and what you like to chat about.”

Soon your query to ChatGPT about how to treat your young daughter’s cold will come into play when you later ask for fun places to visit in Nashville. Rather than telling you to go to bars downtown, it may direct you to the Adventure Science Center or the Nashville Zoo.

It’s easy to imagine far less innocuous uses of all these retained memories.

Over time, the model and, by extension, OpenAI, Microsoft, or another AI lab can generate a lifelong profile of your career aspirations (a resume you uploaded), your love life (your request for first-date restaurant recommendations), and your vices (your search for casino game tips). The more these companies learn about you, the more they will cater to your specific needs and the more inertia you will feel to stay with that company’s products without regard to quality, privacy, or any other characteristic.

This lock-in will be felt by the many millions or billions who have come to rely on that one company. It will also be felt by the entrepreneurs who see their novel models flounder as too few users are willing to incur immense switching costs, even for demonstrably better products.

Lawmakers eager to tackle AI during legislative sessions this year should circle interoperability and data portability requirements as a key priority. Given the difficulty in moving any legislation through Congress, state legislators have an opportunity to lay the groundwork for interoperability and portability mandates. They need not specify the technical means for how these processes should occur, but they should insist on the goal of free and frictionless movement from one model to another.

Such legislation can’t wait. As companies evolve and models gather more information, they will only have a stronger case against empowering consumers to take their information and business elsewhere. They will complain that it’s too expensive, technically challenging, and invasive into their proprietary processes. Before we all surrender our memories to a single company, we need to make sure we’re not locked in.

This article does not necessarily reflect the opinion of Bloomberg Industry Group, Inc., the publisher of Bloomberg Law and Bloomberg Tax, or its owners.

Author Information

Jonathan Skrmetti is the Tennessee attorney general.

Kevin Frazier is an affiliated scholar of emerging technology and the law at St. Thomas University and an adjunct professor at Delaware Law School.

Write for Us: Author Guidelines