Statistical Significance
How many data points do you need to action something?
https://www.investopedia.com/terms/s/statistical-significance.asp
Yield Profiles - Group by Day/Week can be used to help explore/validate this.
Automation rehabilitation => When do they no longer satisfy a criteria
Guilty until proven innocent, or innocent until proven guilty?
A discussion of how many runs needed to get a good understanding of algorithmic order execution selection.
"Whenever you feel tempted to judge an algo after five runs - it happens to us all - please think of this study. Recall that this hypothetical algo with objectively strong results over the long run - it buys 50mio GBPUSD for 0.5 pips - is likely to deliver an average result after five runs of between minus 2.5 pips and plus 3 pips. There is simply too much noise or ‘luck’ in which outcome you’ll get over a handful of runs"