Converting a quantitative system into a graybox

Let’s imagine you have a profitable system already and you want to see if you can improve the results by using discretion or maybe you have a quasi-defined strategy that you think could work with discretion. Below, I list steps you might take to go about the conversion to increase your probability of success:

1. Decrease the selectivity of the system. Think about it, in order to improve the system you need to allow for the possibility of the potential for more wins or losses. It is also important to increase the frequency because simply most systematic strategies trade too infrequently for a discretionary trader to gain any experience with.
2. Divide your historical data into 3 sets: training, test, and validation. Now using market replay or bar-by-bar playback you test your ability to trade/improve results on the training set. You allow yourself  to cheat on the training set because you can replay it over and over. Finally, you validate on the test set. The test set becomes the new training set. And you can do a final validation on the validation set. What is relevant is that you are training, specializing, to trade under specific conditions suggesting you might be able to gain relative advantage.

If you beat the system significantly then you have evidence you can improve the results. You might, also, be able to identify new rules or improvements which you can add to the system.

If you have a few systems, you could follow the same process with each system and then you’d have a playbook. Obviously, this is the opposite way that most traders go about things. There is a valid question that even if you can manage to improve the results whether or not you can improve them significantly enough such that it makes it worth the time and increased risk. But, this approach is certainly more refined then simply trying to “wing it”and make changes on the fly.