Your ML Model Doesn’t Compete With the Market. It Competes With Other Models

Imagine you build a trading model that predicts the next market move correctly 60% of the time. That sounds impressive. Now imagine that every serious trading firm has the same model. The accuracy is still 60%. The model has not become worse. But the edge may be gone. This is one of the strangest things about machine learning in finance. In most ML domains, a useful model stays useful even if somebody else independently builds the same thing. In markets, the fact that other participants know the same pattern can destroy its economic value. That means a financial model should not be evaluated only by asking whether it predicts the market. A more important question is: what does this model know that competing models have not already priced in? That turns financial ML into a relative game. This is where game theory offers a useful lens. Markets are strategic environments: the payoff from one strategy depends on what other participants are doing. In a simplified zero-sum game, an opportunity that everyone can exploit cannot remain an opportunity for everyone. As more models discover the same signal, their actions change the payoff itself. This is closely related to the idea of a Nash equilibrium: a state in which no participant can improve their outcome by changing strategy alone. A model can remain statistically accurate and still lose its edge once enough competing models have learned the same pattern. The Market Does Not Pay for Accuracy. It Pays for Scarcity Suppose a model discovers that a certain combination of signals predicts positive returns. At first, only one team knows about this relationship and trades on it. Then another team finds it. Then ten more do the same. The statistical relationship does not necessarily disappear immediately. The feature can remain predictive, but much more capital is now trying to exploit the same pattern, so the expected profit available to each participant becomes smaller. This is why predictive power and alpha are not the same thing. A pattern can exist in the data without offering much economic value. What matters is not only whether information predicts future prices, but whether that information is still scarce. I find it useful to think about alpha this way: alpha is not just useful information. It is useful information that has not yet become common knowledge. That last condition changes everything. Common Knowledge Is the Enemy of Alpha Take a simple trading signal. Perhaps a certain market event tends to be followed by a move in one direction. If only a few participants recognize it, they may be able to profit from it. Once enough participants recognize the same signal, they start acting before the expected move fully develops, and eventually their own reaction changes the opportunity. What was once a five-minute effect may become a one-minute effect, then a ten-second effect, and eventually something that remains statistically visible but becomes difficult to monetize. The interesting part is that nothing necessarily “broke.” The relationship simply became known. This is different from ordinary model decay. The model may not have failed because the data became noisier or because the market randomly shifted into a new regime. It may have stopped making money because the knowledge spread. Public knowledge in quantitative trading has a strange property: the more useful it becomes, the faster it can destroy its own usefulness. There is another effect of having more competitors: liquidity and strategy capacity are limited too. Suppose we know with very high confidence that a price will move up by 0.5%. Any available liquidity below that level looks attractive: if we can buy at those prices and exit after the expected move, the trade should be profitable. But what if there is only $10,000 available? That $10,000 is the entire opportunity. If another strategy detects the same signal and reaches the market first, we may get nothing while the competitor gets the full fill. The signal is still correct. The opportunity is still there. We simply lost the race to capture it. Execution speed can make the situation even worse. If we send a market order without a price limit, being late can mean paying prices that are already above the level justified by our signal. In the extreme case, we can end up buying after the expected 0.5% move has already happened and turn a theoretically profitable signal into an actual loss. At that point, however, we are no longer talking only about competition for alpha. We are talking about latency and execution. Now imagine there is $10 million of liquidity instead of $10,000. There may be enough capacity for both strategies to participate. But even then, the first strategy to react will generally get better execution, while the second gets what remains. This is the important part: adding competitors does not create more opportunity. If several models react to the same signal at the same time, they are competing for a finite pool of liquidity. The more participants there are, the smaller the expected payoff can become for each of them—and sometimes the competition can make the trade unattractive for everyone. The market is not elastic. A signal can be real, predictive, and economically meaningful, while still having very limited capacity. The Second-Order Prediction Problem Most ML models are trained to answer a first-order question: what happens next? Markets often require a second-order question: what does everyone else think will happen next? Sometimes the problem goes one level further: what does everyone else think everyone else will do? This is where financial ML starts to look less like ordinary forecasting and more like a strategic game. Suppose you believe an asset is undervalued. That belief alone is not enough. You also need to know whether everyone else already believes the same thing. If they do, the price may already reflect that view. The value is not in the prediction itself; it is in the difference between your prediction and the market’s aggregate expectation. So the important quantity is not simply: P(price goes up) but something closer to: P(price goes up | what everyone else already knows) That is a much harder target. A Better Model Can Still Have No Edge Imagine two research teams. Team A builds a technically sophisticated model with excellent predictive metrics. Team B builds a simpler model, but it uses information that few other participants have incorporated. Which one has the stronger strategy? Potentially Team B. This sounds obvious once you say it out loud, but it runs against how ML research is usually organized. Researchers naturally optimize against benchmarks: higher accuracy, lower loss, better calibration. In trading, those improvements matter only if they improve something competitors have not already captured. A model that is 5% better on a public benchmark may be less valuable than a mediocre model trained on a genuinely differentiated information set. That is why the best trading model is not necessarily the most accurate one. It may simply be the one with the most unique residual information. There is another reason differentiated information matters: robustness and survivability. A model can have excellent average returns and still be a terrible strategy if one market crisis wipes out everything it made during the rest of the year. You can make money for eleven months and give it all back in the twelfth. From a trading perspective, that is not a minor statistical inconvenience. It is a failure of the strategy. I’ve touched on this problem in my previous articles. Financial models operate in an environment where the underlying conditions keep changing, so a model that looks impressive over a limited period may simply be well adapted to the regime it happened to observe. The real test comes when that regime changes. This is where less widely known information can provide another advantage. If fewer competing models rely on the same data or economic relationship, there may be less crowding around the same trade. When market conditions change, you are less likely to have dozens of strategies failing for exactly the same reason. This does not make a model immune to crises. But it can improve its chances of surviving them. In an evolutionary environment, being well adapted to the current environment is not enough. You also need enough diversity to survive when the environment changes. In trading, survival is part of performance. A strategy that earns less but survives a regime change can be more valuable than one that earns more until the environment changes. What Happens When Everyone Finds the Same Edge Competition does not necessarily eliminate a signal instantly. More often, it compresses the return available from it. First there is an inefficiency. Then someone discovers it. Capital enters. Other participants notice the profit. Similar strategies appear. The trade becomes crowded. Expected return falls. Eventually the opportunity reaches a new equilibrium where the remaining profit is too small to attract much more capital. That is exactly what you would expect from a competitive market. The interesting implication for ML is that the objective itself is moving because competitors are learning. A successful model creates evidence that an opportunity exists. That evidence attracts competition. Competition reduces the opportunity. Success therefore contains the mechanism of its own decay. Sometimes the process looks almost like a self-fulfilling prophecy. Suppose a weak signal suggests that a price is likely to move up. If enough participants believe the signal and trade on it, their collective buying can push the price up. The signal becomes stronger precisely because people believe it. This raises a surprisingly difficult question: what exactly is the signal—the underlying pattern in the data, or the market’s belief in that pattern? For financial ML, the answer may be both. A model does not always observe an independent feature of the market. Sometimes it observes a pattern that exists partly because other models are observing it too. The Red Queen Problem There is an evolutionary idea called the Red Queen effect: you have to keep running just to stay in the same place. Quantitative trading often feels exactly like that. Suppose your team improves its models by 10%. That sounds great. But if your competitors improve by 15% at the same time, your relative position has actually become worse. Your technology got better, but your edge got smaller. This is easy to miss because we tend to measure progress in absolute terms. The model is more accurate. The infrastructure is faster. The research pipeline is better. But none of that tells you whether you are gaining ground. Financial ML does not compete against a fixed benchmark. The benchmark learns too. A model that was state of the art three years ago can become mediocre without changing at all. The market did not necessarily become more random. Other participants simply learned to do the same things better. Think about upgrading a smartphone. You may not need a new phone because the old one has stopped working. But after several operating-system updates, the same hardware may start to feel slower. To maintain the same experience, you eventually need an upgrade—not because the old hardware got worse, but because the environment around it became more demanding. The same thing happens in quantitative trading. Maintaining the same economic performance can require continuous improvements in research, data, models, and infrastructure simply because the competitive baseline keeps moving. There is also an old joke that captures the idea nicely: when you meet a bear in the forest, you do not need to run faster than the bear. You just need to run faster than the slowest person in the group. That is why saying “our model is improving” is not enough. The real question is: are we improving faster than the competition? Crowding Is Not Just “Too Many People in the Same Trade” Crowding is usually discussed as a portfolio problem: too much capital is positioned in the same direction. From an ML perspective, crowding starts earlier. It begins when many models depend on the same assumptions, the same datasets, the same features, the same market relationships, or the same behavioral patterns. Two strategies can look completely different at the code level and still compete for exactly the same underlying alpha. This creates correlated failure. If the underlying edge disappears, dozens of supposedly different systems can deteriorate at the same time. That is why model diversification should not be understood only as having many models. Ten models exploiting the same economic effect are still one bet. A simple example is the relationship between price deltas and changes in trading volume. At first glance, a model using price changes and another using volume changes may seem to rely on different signals. In practice, they can be strongly related: prices generally do not move without trades, while unusually large trading volumes often accompany or contribute to price movements. The two models may therefore be capturing different expressions of the same underlying market mechanism. For quant teams, this distinction matters. Model diversity is not architecture diversity. It is edge diversity. The Hardest Question in Quant Research Before spending months improving a model, I think there is a more important question to answer: why can this opportunity exist in a competitive market? There should be a reason. Maybe the information is difficult to collect. Maybe processing it requires unusual infrastructure. Maybe the opportunity is too small for large firms to care about. Maybe it comes from participants who are forced to trade for reasons unrelated to profit maximization. Maybe exploiting it requires holding a risk that others cannot or do not want to hold. Or maybe the relationship itself is simply very difficult to discover. These are economic explanations for alpha. Without one, a beautiful model should make you suspicious. If an opportunity is obvious, scalable, easy to trade, based on public data, and highly profitable, why has competition not removed it already? That question is often more useful than another round of hyperparameter tuning. The Goal Is Not Better Prediction. It Is Differentiated Prediction This is where I think many discussions about AI in trading go wrong. They focus on building increasingly powerful predictors: larger models, more features, better architectures, more compute. All of that can help, but a more powerful predictor does not automatically create more alpha. If every competitor gets access to the same architecture, the improvement simply becomes part of the baseline. The real objective is not just better prediction. It is differentiated prediction. You need to know something earlier, infer something differently, combine information in a way others do not, or operate where competing capital is weaker. That is the part of the problem that does not show up neatly on an ML leaderboard, and it is probably the part that matters most. This is where the idea of adversarial networks becomes useful. In an adversarial setup, one model learns in response to another model that is actively trying to challenge it. Financial markets have a similar dynamic: when one model discovers a profitable pattern, other models can learn to exploit the same pattern, changing the environment in which the original model operates. The model is therefore not predicting a passive market. It is predicting a market that is reacting to predictions. This creates a feedback loop: a model finds an edge, competitors detect it, they trade on it, and the edge changes. The original model may still be making accurate predictions, but the economic value of those predictions can decline as the market adapts. That is one reason why financial ML cannot be evaluated purely through prediction metrics. A model that wins on a historical benchmark may still lose in production if the behavior it learned has already become common knowledge. Financial ML Is a Relative Game This is the main difference I would keep in mind: a financial model does not have intrinsic value. Its value depends on what the rest of the market can do. A 55% accurate model can be extremely valuable if everyone else is around 50%. A 70% accurate model can be worthless if everyone has exactly the same prediction. A feature can remain statistically significant while alpha disappears. A model can remain unchanged while its competitive value collapses. And a successful strategy can weaken precisely because others discovered why it was successful. That is why I do not think the main task of financial ML is “predicting the market.” The market is not a passive target. It is a population of other prediction systems. The real task is to find information that remains useful after accounting for what those systems already know. That creates a very different definition of a good model. The market does not pay you for knowing the answer. It pays you while the answer is still rare.