AI Personalization and the Illusion of Consumer Choice

The most successful piece of digital marketing infrastructure ever created may be a brightly coloured button labelled "Accept All". It asks no difficult questions. It does not explain what the system will infer about you six months from now. It certainly does not mention that the product may use your hesitation, location, purchases, abandoned searches and late-night scrolling to decide which version of reality converts best. It simply offers a small exchange: one click for the disappearance of an irritating box. Legally, that click may be recorded as consent. Commercially, it opens the door to personalisation. Psychologically, it often means something rather less dramatic: please let me read the page. This is the fiction at the centre of AI personalisation. Companies behave as if an interface captured a meaningful choice. The user behaved as if they were clearing an obstacle. Both sides got what they wanted. Only one of them understood the transaction.Consent is not the same as resignationUnder the UK GDPR, valid consent must be freely given, specific, informed and unambiguous. It must involve a clear affirmative action, and withdrawing it should not be more difficult than giving it. That is a demanding standard. It is not satisfied merely because a database contains a timestamp beside the word "accepted". The Information Commissioner's Office distinguishes between three types of information commonly used for personalised advertising:Provided data, which people intentionally submit, such as their age or interests.Observed data, collected from their behaviour across products, websites or devices.Inferred data, which the system derives, such as likely interests, habits, financial position or future behaviour.The third category is where the fiction becomes most obvious. A user may understand that a retailer remembers the shoes they viewed. They are less likely to understand that the system has combined browsing time, device type, location and purchase history to place them in a behavioural category that influences prices, offers or urgency messages. The user agreed to the inputs they could see. The commercial value lies in the inferences they could not. This does not automatically make the processing unlawful. It does make "the customer consented" an incomplete answer.Not every personalisation system requires consentThere is a habit in AI commentary of treating consent as the single legal basis for any use of personal information. That is convenient, memorable and wrong. UK data protection law provides several lawful bases for processing. Depending on the context, an organisation may rely on consent, contractual necessity, legitimate interests or another basis. Special category data brings additional restrictions. Electronic tracking technologies may separately engage the Privacy and Electronic Communications Regulations. The correct question is therefore not:Did the customer consent to AI?It is:What personal information is being processed, for which purpose, using which lawful basis, under what additional rules, and with what effect on the individual?That question is considerably less attractive on a cookie banner. The legal position around automated decisions has also changed. The Data (Use and Access) Act 2025 replaced the previous UK GDPR Article 22 framework with new provisions governing automated processing and significant decisions. By June 2026, the data protection provisions of the Act were fully in force.The revised regime allows organisations to rely on a wider range of lawful bases, potentially including legitimate interests, for some significant decisions made without meaningful human involvement. Stronger restrictions remain where special category data is involved, and safeguards are still required. These include informing people about significant automated decisions, allowing them to challenge those decisions, make representations and seek human intervention.This is a more permissive framework. It is not the liberation of every personalisation algorithm from adult supervision.Many advertising recommendations will not, by themselves, constitute decisions with legal or similarly significant effects. Others may become significant when personalisation affects access to credit, employment, insurance, housing or essential services."AI personalisation" is not a legal category. Purpose and consequence matter.The EU has already tested the excusesConsent is not the only basis a company can choose, and choosing something else is not automatically a shortcut. For years, Meta did not rely on consent at all for behavioural advertising on Facebook and Instagram. It relied on "contractual necessity" — the argument that showing personalised ads was baked into the service users agreed to when they accepted the terms.In January 2023, Ireland's Data Protection Commission fined Meta a combined €390 million after the European Data Protection Board found that this reasoning did not hold up. The EDPB's position was that the contractual obligation to deliver personalised advertising, if it existed at all, ran between Meta and its advertisers — not between Meta and its users. A user's contract for a social media service does not, without more, require behavioural profiling as a term of using it.This is an EU GDPR decision rather than a UK GDPR one, and the two regimes have diverged since Brexit. But the underlying lesson travels across the Channel without difficulty: naming a different lawful basis is not the same as satisfying it. "It's in the contract" invites exactly as much scrutiny as "the user consented", and regulators are willing to look behind either label.The EU has also gone a step further than testing individual justifications. Article 38 of the Digital Services Act requires very large online platforms to offer at least one version of each recommender system that is not based on profiling at all. TikTok announced in 2023 that EU users would be able to switch their For You feed to a version built from popular content rather than personal data. That is a structural remedy rather than a consent mechanism. It sidesteps the question of whether an individual's agreement was meaningful by simply making meaningful agreement unnecessary for at least one version of the product.The UK has not adopted an equivalent requirement. The Data (Use and Access) Act 2025 framework, discussed above, still operates through lawful basis, transparency and individual rights rather than mandating a profiling-free alternative. For now, a company operating in both markets may find itself explaining a lawful basis to a UK regulator while building an entirely separate, non-personalised product tier for EU users.The right to object did not vanishIn 2025, human rights campaigner Tanya O'Carroll settled a long-running case against Meta after seeking to stop Facebook from using her personal information for targeted advertising. The settlement prevented a full trial and therefore did not produce a judicial ruling establishing a general precedent. Meta nevertheless agreed to stop processing O'Carroll's personal data for direct marketing.The ICO had intervened in the proceedings and publicly stated that online targeted advertising should be treated as direct marketing. Its position was straightforward: people have the right to object to the use of their personal information for direct marketing, and organisations must provide a clear route for doing so. The ICO statement matters because personalisation programmes often obsess over the moment consent is obtained and neglect the rights that operate afterwards. A system can have an immaculate consent record and a deliberately miserable opt-out journey. One click turned the personalisation on. Turning it off requires six screens, an account password last used in 2019 and the emotional stamina of someone renewing a British Rail season ticket. Consent is not valid forever simply because withdrawal is tedious."Consent or pay" is still a choice designed by the sellerSome platforms have responded to challenges around personalised advertising with a new proposition: agree to data-driven advertising or pay for an alternative service. The ICO does not consider every "consent or pay" model inherently unlawful. Its existing guidance on these models, which is under review following the Data (Use and Access) Act, identifies four factors relevant to whether consent is freely given:power imbalance;the appropriateness of the fee;equivalence between the services offered;Privacy by design.The framework exposes the weakness in treating consent as a binary event. Imagine a social platform that has accumulated a user's photographs, contacts, professional network and years of conversation. The platform then offers two choices: accept personalised advertising or start paying. Technically, there is an alternative. Economically and socially, leaving may be costly. The existence of a second button does not prove that the choice between them was free. The design of the alternatives matters as much as their presence.AI personalisation changes after the user has agreedTraditional consent language imagines a relatively stable activity. A company states what it wants to do, the individual agrees, and the processing begins. AI systems are less polite. The personalisation model changes. New data sources appear. Objectives are adjusted. What began as product recommendations expands into churn prediction, price optimisation or emotional targeting. An inference created for one campaign becomes an input for another. The original consent remains preserved in the database, frozen in time like a tiny legal fossil. The processing around it evolves.This is similar to the organisational problem I described in The Hidden Cost of AI: individual actions become faster while the systems responsible for coordinating and governing them become less coherent. Marketing can launch new segments in hours. Data teams can add new signals in days. Privacy notices, consent wording and internal accountability move at the traditional speed of a committee looking for a suitable Tuesday. Eventually, the consent describes the system the company used to operate, not the one it operates now.Personalisation is a chain of decisionsA banner presents personalisation as one activity. In practice, it is a chain:Data is collected.Behaviour is observed.Attributes are inferred.A profile is created or updated.The person is placed into a segment.Content, timing, price or treatment is selected.The person's response becomes new training or optimisation data.A user may be comfortable with one step and strongly object to another. They might welcome recommendations based on their purchase history, but reject cross-device tracking. They may accept product suggestions while opposing an inference about their health, income or vulnerability. They may agree to personalisation within one service but not expect the resulting profile to travel across a corporate group. A single "Accept All" button collapses these distinctions because operationally, that is easier. Ease for the controller should not be confused with clarity for the individual.The consent record is not the governance systemMany organisations can prove exactly when a user clicked a button. Far fewer can answer:Which model used the resulting information?What inferences were produced?Which campaigns or decisions relied on them?Whether the purpose changed after collection.How withdrawal propagates through downstream systems.Whether a deleted profile persists in an audience export, feature store or model-training dataset.This is where consent becomes an infrastructure problem. Two figures make this concrete enough for a product team to actually own. The first is opt-out friction: how many clicks, screens and re-authentications separate a user from turning personalisation off. If your own analytics show a 90% abandonment rate somewhere inside that flow, that is not evidence people don't want the option — it is evidence of a dark pattern, and regulators increasingly read it that way. The second is inference drift: the gap between what a user actually told the system and what the system has since inferred about them. A user who typed "interested in fitness" and is now being served content correlated with disordered eating or financial distress has crossed from provided data into an inference that deserves its own review, not quiet reuse.In The Real Risk in AI Teams Is Missing Review Loops, I argued that human vigilance is not a control. The same is true of privacy paperwork. A policy cannot enforce purpose limitation, remove an audience segment or stop a model using a forbidden attribute. Consent needs an operational path through the system. When an individual withdraws, the instruction should reach every relevant component. When a new personalisation purpose is proposed, it should trigger a review of the lawful basis and existing transparency. When an inference may reveal special category information, the system should escalate before the campaign launches, not after somebody notices an unsettlingly specific advert. Otherwise, the business has evidence of a click and no reliable control over what the click unleashed.A better test for meaningful choiceCompanies auditing AI personalisation should stop beginning with the banner. Start with the actual system and work backwards. For each personalisation purpose, ask:What changes for the person? Is the system rearranging products, setting a price, determining eligibility, selecting an emotional appeal or influencing access to an opportunity?What information drives the decision? Separate provided, observed and inferred data. The fact that an input is an inference does not remove it from data protection law.What is the lawful basis? Do not write "consent" because it feels safest. Do not write "legitimate interests" because it feels easiest. Document why the selected basis fits the purpose and the impact.Can the person reasonably understand the processing? Listing thirteen categories of "partners" in a collapsible menu is technically informative in the way that handing someone a telephone directory is technically giving them a number.What happens when the person says no? Test refusal and withdrawal as real user journeys. Check what stops, what remains and what reaches downstream systems.When does the decision become significant? Routine recommendations and decisions affecting rights or essential opportunities should not share the same governance threshold.Who owns the explanation? If marketing, legal, product and data teams each believe another department understands the personalisation system, the user certainly does not.The fiction is becoming expensiveAI personalisation is commercially powerful because it turns uncertainty into probability. It predicts what people may want, what they may fear, what they may buy and which message is most likely to move them. Consent is supposed to give individuals agency within that system. Too often it does something narrower: it gives the organisation a record that the agency was performed. Those are not the same thing.The legal question is no longer whether a button was clicked. It is whether the choice was real, the purpose was understandable, the processing remained within its boundaries, and the individual retained a practical route out. If the personalisation system cannot survive a customer saying no, then the company has not designed meaningful consent. It has been designed with a compliance theatre with a very effective call-to-action.This article provides general analysis and does not constitute legal advice.