The model is not the molecule

There is a sentence that shows up in almost every deck for an AI drug-discovery company: our model predicts X with state-of-the-art accuracy. It is usually true. It is also usually beside the point.

The model is not the molecule. The molecule is not the mechanism. The mechanism is not the patient. Each step in that chain is a translation, and each translation loses information that no amount of held-out accuracy recovers.

Three places the chain breaks

Target validity. A model can rank targets by how well they fit a learned representation of disease biology. That says nothing about whether modulating the target will change the disease in a human, in the right tissue, at a tolerable dose. Target validity is a causal question, and most of our data is correlational. A model trained on expression atlases learns association beautifully and intervention poorly.

Assay transfer. The same compound can look promising in one cell line and inert in another. A model evaluated on a random split of a single assay is mostly measuring its ability to memorize the assay’s idiosyncrasies. When teams move from retrospective benchmarks to prospective experiments, performance drops — not because the model got worse, but because the distribution changed underneath it.

Delivery and exposure. Even a perfect binder can fail because it does not reach the tissue, is cleared too quickly, or hits an off-target that the model never saw. These are not modelling failures in the narrow sense. They are the rest of the problem, and they dominate the outcome.

None of this is an argument against machine learning in drug discovery. It is an argument against using the wrong scoreboard.

A benchmark is a compressed claim about what matters. When the compression is wrong, the benchmark does not just fail to measure progress — it redirects it.

What better evaluation looks like

The useful question is not “how accurate is the model?” but “what decision does this model change, and what happens when it is wrong?”

That reframing has consequences:

• Evaluate at the decision boundary. If the model prioritizes ten targets for a wet-lab campaign, evaluate precision at ten, not AUC across the whole ranking. The top of the list is where the money and the animals go.
• Report failure modes, not just averages. A model that is excellent on well-characterized targets and useless on novel ones is not uniformly good. Slice the evaluation and say so.
• Use prospective tests with pre-registered endpoints. Retrospective benchmarks are necessary but cheap. The credible signal is a prediction made before the experiment and scored after it.
• Model the measurement process. Assay noise, batch effects, and selection bias are part of the data-generating process. A model that ignores them will look better in cross-validation than it can ever be in the lab.

The industry incentive problem

This is where the science meets the business model. Benchmark improvements are legible to investors and easy to put in a press release. Prospective validation is slow, expensive, and often produces ambiguous results. The incentive is to optimize the legible thing.

The companies that will matter are the ones that build the unglamorous infrastructure: standardized assays, prospective evaluation loops, and a culture that treats a negative result as information rather than a failure of storytelling. That is a harder pitch than a new foundation model. It is also much harder to copy.

A modest reframe

When you read the next paper claiming a new state of the art for molecular property prediction, ask three questions:

1. What decision is this prediction supposed to inform?
2. Was the evaluation built around that decision, or around the data that happened to be available?
3. What would change in the lab if the model were wrong?

If the paper cannot answer those questions, it may still be good machine learning. It is not yet evidence that we are closer to a medicine.

The model is not the molecule. Keeping that sentence in view is not pessimism. It is the only way to build something that survives contact with a wet lab.