The Division of Learning

Executive Summary

In 2015, seven years before ChatGPT and two years before the transformer architecture that made it possible, Shoshana Zuboff published a paper in the Journal of Information Technology called "Big other: surveillance capitalism and the prospects of an information civilization." She was not writing about artificial intelligence. She was writing about Google, and specifically about two articles by Google's chief economist Hal Varian describing what computer-mediated transactions make possible.

The paper is worth reading now for a reason its author could not have intended. Zuboff was describing the logic of an apparatus that did not yet exist in its finished form. The AI assistant is that apparatus. It sits closer to thought than any previous instrument, it is used voluntarily and continuously, and it is the most complete answer yet given to the question she said would define the century.

That question is what she calls the division of learning. As she puts it: "Who learns from global data flows, what, and how? Who decides? What happens when authority fails?" She argues that the division of learning will shape the coming era the way the division of labor shaped the last two, and that the answers are being settled not by deliberation but by a logic of accumulation operating largely out of view.

This paper does three things. It sets out what Zuboff actually argued, as carefully as we can, because the argument is regularly flattened into a slogan about privacy and it is not about privacy in the ordinary sense. It then applies her three questions to the AI assistant, which is where her framework earns its keep, and it takes seriously her most useful and least quoted reframe: that privacy is not secrecy but decision rights. Finally it describes what a different architecture answers, and, at equal length, what that architecture cannot claim. Zuboff's critique is structural. No product is a remedy for a structural problem, and any company telling you otherwise is doing the thing she warned about.

1. What Zuboff Actually Argued

1.1 The Distinction That Starts Everything

Zuboff's foundational idea is much older than this paper and does most of the work in it. In research beginning in the early 1980s she observed that information technology has a duality that mechanization never had. It can automate, substituting machines for human effort in the way that machines have always done. But it simultaneously informates: it produces a record of its own operation, rendering the activity it touches "visible, knowable, and shareable in a new way." Her line for it, written in 1985, was that the informated workplace "may no longer be a 'place' at all, but rather an arena through which information circulates."

Every automated action leaves a text. That text creates new opportunities for learning, and therefore new contests over who does the learning. This is why she insists that big data is not a technology, an effect, or an inevitability. It is "the foundational component in a deeply intentional and highly consequential new logic of accumulation."

1.2 The Four Uses

Her analysis is organized around four uses Varian identifies as following from computer-mediated transactions: data extraction and analysis; new contractual forms due to better monitoring; personalization and customization; and continuous experiments. Read them as a description of any modern AI product and they are uncomfortably exact. Extraction of interaction data. Terms enforced by the fact that the product observes its own use. Personalization as the headline feature. Continuous experimentation, which in current practice is called evaluation and A/B testing and happens constantly.

Zuboff's move is to take Varian's celebration seriously and follow it to its institutional conclusion, which is where the two of them part company entirely.

1.3 Big Other, Not Big Brother

Her central coinage is deliberately not Orwellian. Big Brother is centralized, visible, and coercive, and Bentham's panopticon at least had the courtesy to be a building you could walk out of. What she names instead is "a ubiquitous networked institutional regime that records, modifies, and commodifies everyday experience from toasters to bodies, communication to thought, all with a view to establishing new pathways to monetization and profit."

The consequence she draws is the sentence that has aged most sharply: "Unlike the centralized power of mass society, there is no escape from Big Other. There is no place to be where the Other is not."

That is the claim to hold in mind through everything that follows, including our own argument, because a product cannot falsify it and should not pretend to.

2. The Division of Learning in 2026

Zuboff frames the stakes as civilizational rather than commercial. The division of learning, she writes, is "always shaped by contests over these questions: Who participates and how? Who decides who participates? What happens when authority fails?" Her answers in the paper are bleak and specific. Who participates? Those with the material, knowledge, and financial resources to access Big Other. Who decides? The answer is decided by new markets in the commodification of behavior.

Now ask the three questions of the thing on your screen.

Who learns? Every session teaches the provider something: what you asked, how you phrased it, what you accepted, what you rejected, what you rewrote. Aggregate interaction data is among the most valuable training and product signal in existence. Whether any individual conversation is used for training is a policy question that varies by vendor and tier. That it constitutes learning by one party is not in question.

What do you learn? In the durable sense that Zuboff means, close to nothing. The session ends. Unless you have built something yourself, no structure accumulates on your side. Your half of the exchange evaporates while the other half is retained, indexed, and compounded.

What happens when authority fails? Models get deprecated on schedules you do not set. Terms change. Prices change. Accounts get suspended by automated systems with no counterparty to appeal to. Vendors get acquired. Zuboff's third question is often read as abstract political theory. In practice it is an operational question with a testable answer: if the vendor failed tonight, what would you still have tomorrow?

This is a sharper version of the division of learning than Zuboff had in 2015, because search queries are fragments and a working relationship with an assistant is not. The assistant sees the draft before the draft is good. It sees the question you would not ask a colleague. It sees the reasoning, not just the conclusion. In 1985 she wrote that the informated environment made the workplace legible. The AI assistant informates thinking.

3. Formal Indifference and the One-Way Mirror

Two of her terms do the most explanatory work when carried forward.

The first is formal indifference. Google, she writes, "is 'formally indifferent' to what its users say or do, as long as they say it and do it in ways that Google can capture and convert into data." What matters is quantity, not quality; the content of your life is not interesting, its capturability is. This is not malice, and reading it as malice is the standard mistake. It is indifference, which is structurally more durable than malice because nobody has to intend anything.

The second is extraction. She is precise about the word: "Extraction connotes a 'taking from' rather than either a 'giving to,' or a reciprocity of 'give and take.'" Extraction is a one-way process, not a relationship. And the processes "typically occur in the absence of dialogue or consent."

Put those together and you get the defining asymmetry, which she states plainly: the firm "knows far more about its populations than they know about themselves. Indeed, there are no means by which populations can cross this divide." Not a gap that effort closes. A structural absence of feedback loops.

Six rows, and every one of them runs one direction. That is the one-way mirror, and it is what makes the arrangement extractive in Zuboff's exact sense rather than merely commercial. Paying for a service is a reciprocity. Paying for a service while the service accumulates an asset from you that you cannot see, correct, move, or destroy is a different arrangement, and it deserves a different name.

4. Privacy Is Decision Rights, Not Secrecy

The most useful passage in the paper is also the least quoted, and it dissolves the tired argument about having nothing to hide.

Zuboff notes that covert data capture is usually described as an erosion of privacy, and argues the framing is wrong: "The work of surveillance, it appears, is not to erode privacy rights but rather to redistribute them. Instead of many people having some privacy rights, these rights have been concentrated within the surveillance regime." The regime acquires extensive privacy rights for itself, in the form of secrecy about its own operations, while depriving populations of choice about their own lives.

Then the reframe. Privacy and secrecy are not opposites but "moments in a sequence. Secrecy is an effect of privacy, which is its cause. Exercising one's right to privacy produces choice, and one can choose to keep something secret or to share it. Privacy rights thus confer decision rights." She quotes Justice Douglas in 1967: "Privacy involves the choice of the individual to disclose or to reveal what he believes, what he thinks, what he possesses."

This changes what to measure. The question is not how much is hidden. The question is who holds the decision. For an AI system there are exactly four decisions worth counting.

1. Entry. What gets into the system at all, decided before it enters rather than regretted after.
2. Retention. What persists after the session, and in what form.
3. Recall. What surfaces in a given answer, and whether you can see why.
4. Destruction. What can be made genuinely gone, everywhere, and verified.

Score honestly. A settings toggle is not a decision right; it is a request submitted to a party that also writes the rules and audits itself. A right you cannot verify is a promise, and Zuboff's whole argument is about what happens to promises in an arrangement with no reciprocities and no feedback loops.

Our rubric, not a measurement: how many of the four decision rights (entry, retention, recall, destruction) are held by the user rather than requested from a vendor. Reasonable people will score the middle rows differently. The top and bottom rows are not close.

5. The Un-Contract

One more idea repays the transfer, and it is the one that explains why reading the terms of service does not help.

Varian is enthusiastic about contracts enforced through monitoring: the lender who can instruct the vehicle not to start, the insurer who verifies driving behavior directly. Zuboff's response is that this is not a new contractual form at all. It is the un-contract. Contracts exist because the future is uncertain and people are fallible; a promise is how humans establish "islands of predictability" in exactly that uncertainty. Strip out the uncertainty through total observation and you have not perfected the contract, you have eliminated the conditions that made trust necessary and therefore possible. Her citation is Arendt: the force of mutual promise is the only alternative to "a mastery which relies on domination of one's self and rule over others."

The AI-era un-contract does not need vehicle immobilizers. It reads: we may change these terms at any time. The clause is not hidden, it is not unusual, and it is not enforceable against the party that wrote it. Everything downstream inherits it. The model you built a workflow around can be deprecated. The retention policy can be revised. The plan you priced your business against can be repriced.

Against that, a stated policy is the weakest possible instrument, because it is a promise from the only party who can revise it. The strong instrument is architecture, which is not a promise at all. A file on a disk you own is not a commitment about the future; it is a fact about the present, and it stays a fact whether anyone honors anything. This is the entire practical content of local-first design, and it is why we treat it as a structural position rather than a feature.

6. Anticipatory Conformity, Updated

Zuboff's most unsettling section concerns what she calls anticipatory conformity. Conformity under a panopticon is a performance you can drop when you leave the building. In a world of Big Other, "without avenues of escape, the agency implied in the work of anticipation is gradually submerged into a new kind of automaticity," which she describes as "a lived experience of pure stimulus-response." The point is not that you obey. It is that the small internal act of anticipating and deciding gets skipped, and nobody notices a skipped act.

Varian's own ambition sharpens it. He wants Google to reach the point where, instead of your asking questions, it will "know what you want and tell you before you ask the question." That is now a shipped product category rather than a prediction.

The AI version of anticipatory conformity is quiet and does not feel like compliance. You stop pursuing the question the model handles badly. You accept the third draft because iterating is tedious, and the third draft becomes your position. You phrase the problem the way that gets good completions, and the phrasing shapes the problem. None of these are failures of will. They are the ordinary economics of friction, which is precisely why they compound.

The defense is not abstention, which is neither realistic nor desirable. The defense is provenance: a record of which conclusions were yours, which were suggested, what evidence each rested on, and what has since been corrected. This is why we have written more about claim verification and source attribution than about almost anything else we build. A system that remembers where a belief came from is the only kind that can hand the anticipating back to you. A system that produces confident, unsourced, unattributable output is the automaticity machine, no matter how good the output is.

7. What a Different Architecture Answers

Zuboff's three questions are usable as a specification. Here is how we have tried to answer them, stated as claims that can be checked rather than as values.

Who learns? Kent's knowledge graph is a local database on your own machine, encrypted at rest. Every skill execution, dropped file, connector query, and correction feeds it, and it resolves them into people, organizations, commitments, and claims with sources attached. The learning happens on your side of the mirror. That is a file path, not a policy.

Who decides? All four decision rights sit with the user by construction. Entry: connectors read at the source and private mode makes zero outbound requests. Retention: the graph is yours, inspectable, and exportable as a structure rather than a transcript pile. Recall: answers carry source labels, so you can see what was used and why. Destruction: forgetting is a durable tombstone applied across every data surface rather than a hidden row, and it survives restarts, syncs, and reindexes. Erasure is the right most systems quietly omit, and it is the one that converts custody into ownership.

What happens when authority fails? Six providers read the same graph, so switching model is a dropdown rather than a migration, and the frontier lab is a supplier rather than a sovereign. In private mode, inference runs locally and no authority is required at all. If our servers vanished tonight, your graph would be exactly where it was this morning. That is the operational form of Zuboff's third question, and it is answerable.

Our companion papers carry the pieces: The Only Thing You Can Own on why memory is the sole ownable layer, Thinking in Private on local inference, The Leak Was Never Storage on the honest security surface, and The Signature You Didn't Sign on provenance and consent.

8. What This Argument Cannot Claim

Zuboff's is an argument about institutions, markets, and law. Ours is an argument about software. Those are different orders of claim, and collapsing them would be the exact move her paper is written against, so here is the boundary drawn explicitly.

A purchase is not a politics. Surveillance capitalism, on her account, is a logic of accumulation sustained by asymmetries of knowledge and power and by the absence of legitimate authority to constrain it. That is remedied by regulation, litigation, collective bargaining, and public understanding. It is not remedied by anyone's product, including ours. Choosing better tools is a reasonable individual response to a structural condition. It is not a solution to one, and treating it as one is a way of not doing the harder thing.

Connecting is not exiting. Kent connects to Gmail, Drive, Calendar, and Notion because that is where your work already lives. Using Kent does not remove you from Google's data flows. It changes who holds the interpretation layer, which is a real and limited change. Your mail is still their mail.

Routed inference is still inference somewhere. When you send a query to a cloud provider through Kent, that provider processes that query under its own terms. What changes is that the accumulated context stays local and the provider is substitutable. Only private mode makes the external count zero, and private mode costs capability, which is a trade we describe rather than hide.

Owned surveillance is still surveillance. A graph that remembers your clients holds data about people who never agreed to your graph. Ownership relocates a responsibility; it does not dissolve one. Anyone building a personal knowledge system on other people's information inherits obligations that predate any of this technology.

Individual exit is not collective remedy. Zuboff's warning about the "psychic numbing" that "inures people to the realities of being tracked, parsed, mined, and modified" applies to a comfortable local-first user as much as to anyone. Solving it privately and losing interest in solving it publicly is a failure mode worth naming, and it is the one we would most likely fall into.

Conclusion

The paper ends with a question rather than a prediction. Zuboff asks whether surveillance capitalism will become the hegemonic logic of accumulation in our time, or whether it will prove "an evolutionary dead-end" that gives way to other information-based market forms. She does not claim to know. She claims only that the answer will shape the character of information civilization the way industrial capitalism's logic shaped the last two centuries, and that it is being settled now, at speed, mostly out of sight.

Eleven years later the settlement is further along, and its terms are being written in a place nobody thinks to look: in defaults. Not in policy documents that nobody reads and that can be revised anyway, but in where the memory lives, in whether erasure is real, in whether an answer can tell you where it came from. Those are architecture decisions made by product teams on ordinary Tuesdays, and they are answering her three questions on behalf of everyone who does not know the questions were asked.

We would restate her hardest sentence rather than dispute it. There is no place to be where the Other is not, and that remains true. What is also true is smaller and worth having: there is one place where the learning can accumulate on your side of the mirror, and it is a disk you already own.

The machines are going to learn from you. That was settled some time ago and no amount of principle reverses it. The open question, the one that is still open, is whether you are also a party that learns.

References

1. Zuboff, S. (2015). "Big other: surveillance capitalism and the prospects of an information civilization." *Journal of Information Technology*, 30, 75-89. doi:10.1057/jit.2015.5

1. Zuboff, S. (1988). *In the Age of the Smart Machine: The Future of Work and Power*. New York: Basic Books.

1. Varian, H. (2014). "Beyond Big Data." *Business Economics*, 49(1), 27-31; and Varian, H. (2010). "Computer Mediated Transactions." *American Economic Review*, 100(2).

1. Arendt, H. (1998). *The Human Condition*, 2nd edition. Chicago: University of Chicago Press.

1. Polanyi, K. (1944). *The Great Transformation: The Political and Economic Origins of Our Time*. Boston: Beacon Press.

1. Madden, M. (2014). *Public Perceptions of Privacy and Security in the Post-Snowden Era*. Pew Research Center.

1. Hoofnagle, C., King, J., Li, S., & Turow, J. (2010). *How Different Are Young Adults From Older Adults When It Comes to Information Privacy Attitudes and Policies?*

1. *Warden v. Hayden*, 387 US 294, 323 (1967), Douglas, J., dissenting.

1. Kent. (2026). Internal Architecture Notes: Knowledge Graph Locality, Tombstone Erasure, and Provider Routing.

Kent Research, September 2026. Quotations are from Zuboff (2015) as published in the Journal of Information Technology; the extension of her framework to AI assistants is ours, not hers. Kent runs 13 built-in skills and unlimited custom skills against six AI providers, cloud or fully local, and stores its knowledge graph as an encrypted local database on your own machine. Connectors cover Gmail, Google Drive, Google Calendar, Notion, PostgreSQL, MySQL, SQLite, MongoDB, REST APIs, and MCP servers. The decision-rights chart is our own rubric and is labeled as such. mykent.app