Predictivity Basics
What is Predictivity?
We look at our pricing model's price and compare it's mid against the unfiltered pool of aggregated LP market data.
If the pricing model mid is lower than the pool mid its betting the rate is going down, and if it is higher than the mid it is betting the rate is going up.
If your rate is predictive, you can move your rate out of the way of harmful moves and attract good business when doing the business is favourable. This is one of the strongest proxies to future P&L.
We check these bets on every tick and a rolling assessment of the pricing models predictiveness is formed.
Each of those bets is summed with an average yield at 30, 60 seconds and 5 minutes being produced
Why is it important?
1) Problems with data sparsity of Trades + PnL + Risk
Client trades, Risk and PnL can be hard to predict. These events occur relatively infrequently. Single events can dominate an entire week's PnL in some cases.
Optimising the per tick predictivity is a quicker feedback cycle on future performance. It is a powerful and accurate indicator of future PnL. It's a proxy to PnL optimisation
This allows us to improve with reduced time to action
2) Avoiding overfitting
Historically, people have given too high weights to overfitting individual trading or pricing scenarios.
Remediative actions are sometimes performed without any business level or simulated backtesting. There is a hunch that the problem is now "solved"
Even when backtesting is performed, it can result in overfitting the data to the case observed and produce a recency bias on system configuration.
Scientific Method
The danger with single-scenario overfitting is that by doing that, we may have improved things from one of the ways with which we assess the pricing or trading but have made things worse in a number of others.
Also, when no backtesting has occurred, we haven't always proved that a change will stop it happening in the future.
We run backtests to establish the cause of system behaviour and verify we have a correct understanding of the causes. Then, not only rerunning the backtest to establish the change was good and fixed the problem, but also rerunning the backtest for a longer period to establish pre and post change consequences.