What MCC 0.56 means, and what it doesn't

Drylab labels one of its results checked: predicting which yeast genes are essential, meaning the cell cannot grow without them. The score is a Matthews correlation coefficient of 0.56, measured on 1,481 genes the model never saw during training.^[1]

Why MCC, and not accuracy

Most genes are not essential. A model that answered 'not essential' every time would be right most of the time, and useless. MCC scores yes-or-no predictions from −1 to 1, where 0 is no better than chance, and it stays honest when one answer is much rarer than the other.

0.56 is clearly better than chance and clearly short of perfect. Before calling it checked, Drylab also requires the model to pass gated checks on the same held-out genes, including beating three simple baselines: predicting the average, predicting no change, and copying the nearest known perturbation.^[2]

Held out means held out

The 1,481 genes were set aside before training and never used to train or tune the model. That separation is the difference between checked and theory. A score on data the model has already seen is a measure of memory, not of prediction.

What it does not cover

• →Checked applies to this test: yeast gene essentiality, at this tier.
• →Drylab's drug screens are labeled theory, on every result, until they are checked against real data.
• →A good score on essentiality says nothing, on its own, about growth rates, double deletions or other organisms.

1. Notes
2. [1]Drylab validation report: yeast gene essentiality, MCC 0.56 on 1,481 held-out genes. Source ↗
3. [2]Drylab docs, How we check: gated checks against baselines on held-out data. Source ↗