The Basics of Evaluating Backtests: What You Need to Know
Two Quants and a Financial PlannerJune 17, 202401:01:16

The Basics of Evaluating Backtests: What You Need to Know

In this episode of the podcast, we dive into the world of backtesting investment strategies. We discuss the importance of approaching backtests with a healthy dose of skepticism, as it's easy to create a backtest that looks impressive but may not hold up in the real world. We explore various questions to ask when evaluating a backtest, such as whether it accounts for human emotions, has a sound economic reasoning behind it, covers a sufficiently long time period, and includes periods of struggle. We also touch on the potential pitfalls of data mining and the challenges of creating a truly "pure" bet without unintended exposures. Throughout the conversation, we emphasize the need to understand the limitations of backtesting and to use it as a tool for learning rather than a definitive guide to future performance.

00:00 - Introduction
00:32 - Introducing the topic of backtesting
01:25 - The importance of understanding backtests for investors
02:13 - Why backtests often show unrealistic performance
04:25 - The need to consider human emotions in backtests
06:40 - The importance of understanding the details behind a backtest
09:29 - The challenges of evaluating backtests based on private data
12:16 - Question 1: Does the backtest account for human emotions?
20:33 - Question 2: Does the strategy have an economic reasoning behind it?
26:44 - Debating the importance of economic reasoning in light of recent research
29:39 - Question 3: Does the test cover a very long period of time?
34:24 - The limitations of using rolling 30-year periods in backtests
36:30 - Question 4: Does the test show periods where the investment strategy struggles?
42:10 - The importance of identifying struggle periods and alternative strategies
46:20 - Question 5: Does the backtest assume knowledge that wasn't available until after the fact?
50:32 - The impact of personal experiences on investment decisions
53:04 - The concept of unintended bets in backtests
56:46 - The importance of skepticism and learning from backtests

SEE LATEST EPISODES
⁠⁠⁠https://excessreturnspod.com

FIND OUT MORE ABOUT SUNPOINTE INVESTMENTS
⁠⁠⁠https://sunpointeinvestments.com/⁠⁠⁠

FIND OUT MORE ABOUT VALIDEA CAPITAL
⁠⁠⁠https://www.valideacapital.com⁠⁠⁠

FOLLOW MATT
Twitter: ⁠⁠⁠https://twitter.com/cultishcreative⁠⁠⁠
LinkedIn: ⁠⁠⁠https://www.linkedin.com/in/matt-zeigler-a58a0a60/

FOLLOW JACK
Twitter: ⁠⁠⁠https://twitter.com/practicalquant⁠⁠⁠
LinkedIn: ⁠⁠⁠https://www.linkedin.com/in/jack-forehand-8015094⁠⁠⁠
Related Videos
Practical Lessons from Jim Paulsen
Two Quants and a Financial PlannerFebruary 10, 202501:05:58

Practical Lessons from Jim Paulsen

In this episode, Matt and Jack break down key insights from our interview with market strategist Jim Paulsen. Paulsen offers unique perspectives on how markets have evolved over his 40-year career, fr...

Practical Lessons from Jason Buck | The Benefits of an All-Weather Investment Strategy
Two Quants and a Financial PlannerFebruary 03, 202500:48:58

Practical Lessons from Jason Buck | The Benefits of an All-Weather Investment Strategy

In this episode, Jack and Matt dive deep into their conversation with Jason Buck about building truly robust investment portfolios that can withstand any market environment. Moving beyond the typical ...

Practical Lessons from Ed Yardeni
Two Quants and a Financial PlannerJanuary 27, 202500:52:23

Practical Lessons from Ed Yardeni

In this deep-dive conversation, Jack Forehand and Matt Zeigler break down key insights from their interview with Dr. Ed Yardeni, one of Wall Street's most respected market strategists. The discussion ...