AI unbundled information from accountability

I was speaking with a senior leader at a banking software company over lunch recently, and he shared something that has stayed with me. A father brought his son to a hospital already convinced the boy had leukemia. He had put the blood work and the symptoms into a large language model, run correlations across the results, returned to it over several sessions, and settled on a conclusion. What he wanted from the doctor was confirmation.

Stories like this are usually told as evidence that AI is coming for doctors. I read it differently. The father walked in holding one part of what a doctor provides and lacking every other part. The distance between those two says a good deal about where value in professional work is heading.

Information without judgment or authority

He had information. What he lacked was a way to judge it. Childhood leukemia is rare, so a careful assessment starts from a low prior. The real question is how much more likely his son's particular results are under leukemia than under far more common explanations, and how much weight the model's answer deserves as evidence at all. He cannot answer that last part. Running correlations across a set of results tells him what moves with what. It does not tell him how often the model is wrong on cases like his son's, and a language model writes just as fluently when it is wrong as when it is right.

His method may have compounded the problem. He asked repeatedly, and models are sensitive to how a question is framed. Repeated questioning of this kind does not supply independent evidence. When each rewording carries more of the asker's suspicion, the answers converge on that suspicion rather than on the underlying likelihood, and the apparent agreement across sessions is largely an artifact of the prompts. The process produces confidence without calibration.

Suppose he is right anyway. He still cannot order the bone marrow biopsy that would confirm it, or start treatment. If he is wrong, there is no one whose job it is to find the error, answer for it, and put it right.

That sequence is worth pulling apart. Information is what the model gave him. Judgment is deciding what the information means for this child. Authority is the right to act on that judgment: to order the test, to prescribe. Execution is the machinery that turns a decision into something real, meaning the lab, the pharmacy, the clearing house. Accountability sits on top of all of them. It is the obligation to answer for the decision, together with the institutional capacity to detect mistakes and remedy them. It depends on the authority and execution beneath it, since no one can meaningfully answer for a decision they had no power to make or carry out. The father had the first link and, at best, part of the second.

Why professions bundled advice with assurance

For most of the history of professional work, you could not buy the first link without the rest. Expert information about your health, your legal position or your money came from people who were also licensed and equipped to act on it, so the price covered the whole chain. Professionals sell other things too, including access, coordination and relationships. But the bundling of knowing with answering for is the part that matters here.

The economics behind it is familiar. In his paper on the used car market, George Akerlof showed that when buyers cannot distinguish good quality from bad, a market can unravel toward the bad, because sellers of quality cannot get paid for something buyers cannot see. He noted that institutions such as guarantees, brand names and licensing partly exist to counteract this, and he named doctors and lawyers among the licensed. A patient cannot evaluate a diagnosis. What the patient can rely on is that the person giving it is qualified, bound by obligations, and working inside an institution that must answer if it goes wrong. Advice and assurance were paid for together because the advice was worth little without the assurance.

The collapse in marginal cost

For a large class of problems that can be described in public terms, the marginal cost of producing plausible, expert-sounding analysis has collapsed. Information in the broader sense has not become free, since proprietary data, verification and context all still cost money. But for many ordinary questions, the first link of the chain now costs about as much as a subscription.

Consider what this does to the lemons problem. The consumer now faces an abundant supply of advice whose quality they cannot assess, and none of it comes with a guarantee. Cheap information leaves the buyer with more to check and no better way of checking it. The scarce part of the bundle is the part that was always hardest to see: judgment backed by authority, execution and an obligation to answer for the result.

Asymmetry reversal at the point of contact

Here is where operating models start to strain, because the two sides adopt AI under different rules.

A consumer carries almost none of the institutional accountability for how they use a model. There is no validation, audit or review, so they adopt at the speed of curiosity. An institution that places AI inside a consequential process takes on responsibility for how the system is deployed and what it is permitted to decide, so it adopts at the speed of its governance.

The result can be a reversal of the old asymmetry. For generations, the professional across the desk knew more and had better tools than the customer. Now the patient may have spent a week working through the case with a frontier model, while the clinician has the electronic record and a short appointment. It is easy to picture the same scene in a bank: a customer arrives with a model-generated analysis of their renewal options and speaks to an advisor whose internal assistant is still in review. At the front door, the information gap can run in the opposite direction.

What the institution still holds is everything after the first link. Its advantage has moved from knowing more to being able to decide, act and answer for the outcome.

Capability versus workflow reliability

That shift explains why the father's use of AI and a hospital's use of AI are different systems, even when the underlying model is similar.

For the father, the relevant question is capability: can the model produce a sensible answer? For the hospital, that is only the starting point. It needs to know how the model performs on its own patients, whose mix of ages, conditions and data may differ from the cases the model was tested on. It needs to know whether the model's stated confidence is calibrated against observed outcomes, so that a reported probability corresponds to the frequency with which such cases actually resolve that way, and whether that correspondence holds in the subgroups where errors are most costly. Ultimately, the hospital answers for the workflow, not the model: the model, the clinician reviewing its output, and the process that catches mistakes before they reach a patient. An accountable institution is judged on the reliability of that whole workflow. A consumer may care about it a great deal when the stakes are high, but is never structurally required to answer for it.

The distinction extends well beyond medicine. When I use AI to help draft a post like this one, an error is cheap, reversible and mine alone to fix. That is a perfectly legitimate use, and governing it like a clinical tool would be a waste. The stakes of the decision should set the level of control, not the mere presence of a model.

Regulation as allocated responsibility

Regulation does not create the value of professional judgment. It allocates responsibility explicitly, specifying who must answer for which decisions and what they must be able to demonstrate when asked. A hospital cannot quietly decline to answer for a diagnosis, and a bank cannot quietly decline to answer for a payment it released. That is why the unbundling is easiest to see in regulated industries. The links in the chain that AI has not cheapened are the ones regulators have already written down.

Read this way, regulation is not simply a speed limit on innovation. By requiring that identified parties answer for consequential decisions, it also protects the pricing of the work that carries that obligation, since the buyer cannot obtain the decision anywhere else. That protection is narrower than it looks, because it covers the decision and the responsibility for it rather than the surrounding explanation, which customers can now obtain elsewhere. The tempting move for regulated institutions is to keep competing as the best explainer of products and conditions their customers can research themselves. The more durable position is to build around the links only they hold.

Private context and mandatory accountability

In this picture, value concentrates where two things hold at once: the context needed for a good decision is not public, and someone is obligated to answer for the decision. A hospital's patient history and a bank's view of a customer's accounts and transactions are examples. A general model does not have that context, and even if it did, it would have no authority to act on it.

That gives a better argument for domain-specific AI than the usual one. The argument does not depend on specialized models scoring higher, and on many tasks they will not. It rests on the fact that a system built inside an institution's data, controls and review processes can produce outputs the institution is able to stand behind, which is what the institution is actually paying for. The same reasoning applies to people. Professionals whose value was mostly retrieving and explaining public knowledge face real competition. Professionals whose value lies in judgment tied to authority, private context and responsibility face much less.

The limits of automating accountability

The strongest objection is that accountability is not a permanent human preserve. Institutions already let automated systems make consequential decisions. High-volume, low-value card fraud screening, for instance, runs largely without human involvement, while complex or ambiguous cases are escalated for review, and banks own the outcomes either way. If that works, why should accountability stay scarce instead of becoming the next thing AI absorbs?

Part of the objection is correct. Where decisions are frequent, outcomes are observable soon afterward, and individual errors are small, accountability behaves like an insurable risk. An institution can measure the error rate, price it and absorb it statistically, without a person standing behind each decision. In those domains, the cost of accountability will keep falling, and the professional bundle will shrink with it.

Look at what the objection concedes, though. Handing the decision to a model did not dissolve the accountability. It stayed with the bank, which could carry it because it had the data to measure the system and the balance sheet to absorb its failures. Accountability becomes expensive again where the statistics run out: rare, high-stakes and hard-to-reverse decisions, where there are too few cases to know the error rate and a single mistake is severe. A childhood leukemia diagnosis is one of those, and so is a large irreversible payment. The claim that survives is narrower and, I think, more useful. The cost of standing behind a decision rises with its stakes and falls with how measurable its outcomes are, and the most defensible professional work sits at the expensive end.

Operating implications

Because accountability at the high-stakes end resists automation, institutions should organize their operating models around holding it rather than around information work that no longer distinguishes them.

For a hospital, this means treating a patient's AI work as evidence to be weighed rather than noise to be dismissed. Intake can capture what the family asked and what the model concluded, so the clinician can test that hypothesis directly. Any change should be judged on what matters to patients, including diagnostic accuracy, missed diagnoses and outcomes, and not on speed alone.

For a bank, it means competing less on explanation and more on assurance and execution: making decisions, moving money, resolving disputes and standing behind the results. It also means closing the tool gap at its own front door. Frontline staff should have governed AI at least as useful as what customers bring in, with controls scaled to the stakes of each task.

Whatever the biopsy showed, the father needed something the model could not supply: a person with the authority to test his conclusion, the means to act on it, and the obligation to answer for the result. That combination, and the institutions able to provide it, is what professional work now has to be organized around.