Using histograms for trading system evaluation

System Trading

Sep 29

Recently, I had a discussion with a trader who expressed a belief that it was not possible to test for an edge when using asymmetrical risk to reward. The idea he expressed was that the skew would make it impossible to know whether or not one had an edge.

And, it is more generally the question of whether or not one has an edge over the market returns, and the method I will share will apply in either case. The more general question might apply if one has a long only strategy tested over a bullish period.

There are probably a few ways to determine that. Statistical tests like the t-test have been developed to compare the distributions of different data sets and determine if they are similar. The method I wanted to share, however, is a simpler visual method.

In our hypothetical case, we have a 4:1 risk to reward system (risk 4 units to make 1 but it could be the other way), and we want to compare our returns to the normal returns.

One simple way to see the difference is to create a frequency histogram of a Monte Carlo simulation of randomized entries and the same risk to reward constraints. The results will need to be re-scaled or normalized (i.e percent) for comparison.


Simulation results generated in Python. The left result assumes exit is strict at profit target or stop loss. The right results shows a range.

There may be another benefit. The Monte Carlo results provide insight into what the lower returns of the system might look like should the edge lose efficacy provided and provided that the general market’s distribution remain similar.

Note, the above graphs are merely simulations generated in Python, as such there may be certain details I missed. It would be useful to show the mode and median, for example. I plan to use this sort of analysis and other more creative applications when analyzing my trading systems in the future.


About the Author

The author is passionate about markets. He has developed top ranked futures strategies. His core focus is (1) applying machine learning and developing systematic strategies, and (2) solving the toughest problems of discretionary trading by applying quantitative tools, machine learning, and performance discipline. You can contact the author at