The overfitting trap
Overfitting means your strategy is optimized for historical data but not for the market going forward. It is the most common way traders deceive themselves.
Signs of overfitting:
- Your strategy has more than 5 parameters
- It performs perfectly on 1 year of data but fails on another
- The profit factor is above 3.0 on the training period
How to backtest honestly
1. Split your data. Use the first 70% to develop your rules. Use the remaining 30% as an out-of-sample test. Never touch the test set until you've finalized your rules.
2. Keep rules simple. The fewer parameters, the harder it is to overfit. A strategy that works with simple rules on 3 years of data is more robust than one that requires 10 parameters to work on 1 year.
3. Test on multiple instruments. If your strategy only works on EUR/USD and fails on GBP/USD, it's probably overfit to EUR/USD's specific price action.
4. Look at drawdown, not just returns. A strategy that doubles your account but has a 60% drawdown is not tradeable for most people.
Using Edgeely's Backtesting Journal
Edgeely lets you define your rules once and run them across multiple instruments and time periods. The side-by-side comparison view makes it easy to identify which parameters are robust and which are overfit.
Start with simple rules. Add complexity only when the data demands it.