*Why You Really Should Optimize And Even If You Don’t…*

Today’s topic is on optimization. Some system developers don’t believe in optimization. However, I’m going to show you why you should really learn how to optimize, and second I’m going to demonstrate that even if you don’t believe in optimization then “not optimizing” is not the solution. “Not optimizing” is not the opposite of optimization!

First, we’re going to assume you want supernormal returns. Let’s just think about what it means for both the strategy and about the markets. If a strategy can deliver supernormal returns without optimization then it suggest that there is robustness to the strategy. That’s a good thing! But, what does it say about the market? It suggests the market is not very efficient. We know that’s not the typical case.

The “opposite of optimization” is not “not optimizing”! Merely looking at a single performance report won’t tell you about how robust a strategy is nor what the typical return is likely to be. What you are really doing when you “don’t optimize” is making an arbitrary decision or a random decision. You are picking some random value. It doesn’t tell you conclusively about what the typical result looks like. Now, if you were to randomly pick values for your variables and do that over and over again then you’d start to get some real insights into the strategy. And that’s the idea behind monte-carlo methods. An optimization algorithmic is just a more intelligent take on the same theme. Instead of randomly selecting values, the optimizer simply steps through the reasonable values one by one (or uses genetic or other advanced methods), and goes one step further by highlighting the very best input values for you. You see the true opposite of optimization is diversification. Diversification, in this case, means to trade multiple combinations of parameters. Instead of trading a strategy with optimal parameters with more size, you could trade a basket or “class” of similar strategies all running with slightly different parameters. In a future post, I’ll share with you an advanced technique for achieving both a higher degree of optimization and diversification and without requiring a larger account.

But, here’s a simple way to optimize sensibly. Run your optimization and look over all returns. Do they look random or do you see any clear relationships? Look at the values above and below the best returns. Are they still reasonable? This is called sensitivity analysis testing and is important. Next, take an average of all returns say 25% above and below your optimal and average them to get a more realistic outlook on your system. You can also take an average of the top 25% of all returns and the bottom 25% of all considered returns. You can do this for drawdowns and other system measures too. You can compute also the min/max for these values which will provide more information than a single performance report. By going through these processes, you’ll start to build up a more complete perspective of the performance metrics of your system.

Examples follow:

Sensitivity analysis reveals that the returns are random. Don’t trade this system.

Input: Return

12: -3,000

11: 2,000

10: 25,000

9: -8,000

8: -15,000Sensitivity analysis looks good. Even though the best return was $29,000, the average of the best returns was only $20,000. Going forward, we might hope to get $29,000 over the same length of time but we know that $20,000 is a more realistic optimistic return.

Input: Return

12: 18,000

11: 23,000

10: 29,000

9: 17,000

8: 15,000What’s a pessimistic return for the strategy below over the same length of time? The tricky part is determining what a “reasonable” value is for a parameter. But, let’s assume all the values shown are reasonable. We take the average of the bottom 3 returns which yields appx $400. One benefit to analyzing your systems this way is it gives you a more complete and realistic view.

Best Case Return: $24,000 (MAX)

Optimistic/Target Return: $19,000 (Average of top 3rd)

Average: $10,000 (Average)

Pessimistic: $400 (Average of bottom 3rd)

Worst Case: -$1500 (MIN)

All values are rounded to avoid giving a false sense of precision.Input: Return

10: 24,000

9: 17,000

8: 15,000

7: 12,000

6: 10,000

5: 11,000

4: 2,500

3: -1,500

2: -700

1: 3,500

Curtis is passionate about markets. He has developed top ranked futures strategies. His core focus is (1) applying machine learning and developing systematic strategies, and (2) solving the toughest problems of discretionary trading by applying quantitative tools, machine learning, and performance discipline. You can contact him at curtis@beyondbacktesting.com.

**Session expired**

Please log in again. The login page will open in a new tab. After logging in you can close it and return to this page.

[…] –from blog Beyond Backtesting […]