Confidence Measures Part 1

System Trading

Mar 11

Accurate confidence is one of the most important skills a strong discretionary trader (or predictor) must develop and possess. A strong discretionary trader should be able to weight how good of a trade or prediction they have and use that weight constructively. If a trader can develop awareness of how good of a trade they possess then they can use confidence based betting. I liken it to “chaos within limits”. It doesn’t mean betting the farm. But accurate confidence allows a trader to bet more on better trades and seek to minimize risk on the poorer trades. Trading systems, also, could benefit from a confidence measure.

While most systems should have a variable that correlates to confidence, it is too often implicit and baked into the system. The system is either confident enough to take a trade or not. Even though whether to take a trade often winds up as a binary outcome, there are still several important reasons one might want a system to be able to output a confidence score:

  • For confidence based betting– bet more on the better trades
  • For determining which signal and market to trade when trading a portfolio
  • For the purposes of building graybox systems that combine both trader discretion and quantitative systems, possibly using Bayesian statistics. The discretionary trader’s market bias and confidence could be used to bias a trading system which would have its own bias and confidence. And, if one wanted to get really meta, a fully objective overseer system could have its own independent confidence and bias could be used to determine the ultimate portfolio weighting or trade decision!

We intuitively understand that the “confidence of a trade” working out equates with the “goodness” of the trade. But, there are actually a few different candidate measures that are reasonable ways to define confidence:

  • Probability of a trade making money, i.e. a corollary to winning percentage
  • Profit factor or maximization expected profit for a given trade.
  • Reward to risk. A few variations could be exp reward to exp risk taken and exp reward to maximum risk.
  • Model or regime “fit”
  • A composite metric combining multiple measures
  • Some other less common measures: expected time in market, minimization of pain, maximization of luck

You might wonder or ask the question, “But if a trader or system can somewhat predict the quality of their trades then why they would take anything less than the best trades?” There are a few answers to deal with that question. For the discretionary trader, the importance is the feedback and interaction with the market. Some might argue that a rational agent will take any trade that yields a profit because that will maximize the net returns. However, in the real world there is uncertainty and risk. As such, we may rationally establish a higher bar than simply making a profit. On the other hand, establishing a very high bar would result in waiting for exceptional opportunities which tend to only manifest during exceptional times. The problem with only trading during exceptional times is those are the times when the risk is greatest! A system that makes fewer trades needs to place bigger bets, and bigger bets means bigger swings.

In the next part of this topic, I will follow up with some concrete examples and attempt to convert a traditional binary thresholded system into a system that outputs a bias with confidence.

About the Author

The author is passionate about markets. He has developed top ranked futures strategies. His core focus is (1) applying machine learning and developing systematic strategies, and (2) solving the toughest problems of discretionary trading by applying quantitative tools, machine learning, and performance discipline. You can contact the author at