Simulation: Beyond Backtesting

Graybox Trading

Aug 29

One problem with traditional backtesting is that it relies on the presupposition that there are repeating predictive patterns in the market. In fact, most trading methodologies rely on this assumption. And  yet, we know the disclaimer that past performance is not indicative of future results.

And, yet backtesting largely assumes that the future will be similar to the past. Yet, we can imagine the possibility for non repeating but predictable profit opportunities. Even without getting into those possibilities, we can imagine that if we can model the dynamics of the market accurately that we can predict new outcomes that cannot be extrapolated from the past.

The way this is accomplished is simulation. Simulation offers the powerful promise of allowing us to make use of historical market data under varying conditions of future similarity. Simulation, massive simulation, is also poised to impact every aspect of our lives.

Imagine for a moment that you are a world class MMA fighter or boxer and you’re competing against a similar well ranked fighter. What should your strategy be? In the past, you might have studied your opponent and intuited a strategy. Perhaps, if you were more sophisticated you might have even used crude statistics such as counting to figure up the risk of and probability of a given move working. But, today, it is surely possible to feed in your moves into a computer with precise timing and force calculations. Next, it is possible to infer the same regarding your opponent by using previous fight videos. In addition, by using the fighters height, weight, and other statistics it is possible to model how well they could perform even moves that were not recorded. Once all the data is put into the computer then you can run thousands or hundreds of thousands of fight simulations. The best performing simulations will yield the best strategies. The strategies that are discovered may be non intuitive and completely innovative. These can be used, with human cognition and consideration, as the basis for your game plan.

Now, imagine how this would work for the trader. It is not just running thousands of simulations on past data. But, you must infer how future traders will react to changing market conditions. This is the difficult part because you need to know how the combination of variables will impact their behavior.

Even if that level of simulation is beyond the average developers capability or can only provide rough approximations due to the difficulty in modeling, it is still possible to start thinking more along the lines of simulation to explore creative opportunity and risk management.

Some ideas for how you might do this:

  • Use random and partially randomized entries and exits to try to find more universal or robust settings for your strategies.
  • Create synthetic market data where you change the amount of volatility, trend, and mean reversion to see how it might impact your strategies.
  • Create models of how traders might act in certain scenarios and look for situations that might offer predictive advantage.
  • Use monte carlo analysis with randomized entries to come up with pessimistic capital requirements
  • Try to find optimal strategies for given market conditions
  • Build self-learning strategies with limited capacity for memory and try to find the optimal rules for trading

About the Author

The author is passionate about markets. He has developed top ranked futures strategies. His core focus is (1) applying machine learning and developing systematic strategies, and (2) solving the toughest problems of discretionary trading by applying quantitative tools, machine learning, and performance discipline. You can contact the author at

  • […] –by Curtis White from blog, Beyondbacktesting […]

  • […] –by Curtis White from blog, Beyondbacktesting […]

  • >