The Dangers of Backtesting - Best Practices to Avoid Disappointment

for the investors reading this article, you can skip ahead to "what is overfitting?" by this point you may have run a few backtests , on a simple script or using one of our educational scripts you may have also achieved a very high profit/loss % and are wanting to run the strategy with real assets before this happens, i want to caution you and help set expectations what is overfitting? if an analysis corresponds too closely with a limited set of data we call this overfitting markets inherently have noise, that is to say small and frequent idiosyncrasies in the price data when modelling a strategy, we want to avoid optimizing for a specific period because there is a chance this won't predict the future it'd be like tuning a car specifically for one racetrack, while expecting it to perform well anywhere how do i avoid overfitting? 1\ test for random periods running a backtest during bull and bear markets is a good first step however, the reality is you are probably not starting a strategy at the beginning of a new market cycle, but in the middle of one https //www investopedia com/terms/s/sidewaysmarket asp can be especially frustrating as a strategy tries to decide bear and bull trends how would you react if your strategy lost money for the first 3 months? 2\ be granular in your testing due to market volatility, it's possible that one good month (or one good trade) is enough to offset a large loss profitable trades and mdd offer a window to overall performance, but does not tell a detailed story to help, we've made our own power bi template that organizes trade data for deeper analysis 3\ run a paper trade (forward test) this one's more of a reminder i want to end in saying that these methods are not comprehensive each strategy is different, making it hard to offer blanket advice happy trading!