Search
• John Hawver

# Base Rate Optimism

Updated: Aug 10, 2019

"You don't win by predicting the future; You win by getting the odds right." - Will Bonner

It’s been a nice summer. I took some time off from my fledgling blog during my self-imposed technology blackout, which I do each year during our family vacation. And then took some more time off after that to reflect.

I struggle with analysis paralysis. I’m sure I’m not alone; there should be a support group out there, Quants Anonymous. When you build models, write code, and work hard to get to the “right” answer, sometimes it seems you never quite get there. So stepping away in the summer is a good thing for my mindset and subsequently my portfolio.

This summer I also decided I should be a bit more disciplined with my reading habits and put together a reading list; most summers I just grab whatever is in the popular fiction section. My self-curated list centered around decision theory this summer. In particular, I enjoyed the books “Superforecasting” and “Range.”

Investing, in any form, is a forecasting problem. One of the concepts in Superforecasting is finding a base rate when approaching a problem. So, I thought, looking at a base rate is probably a good first step to break out of analysis paralysis; you know, do even more analysis to stop doing analysis.

Anyway, I pulled data (code below as always) going back to 1897. I wanted to know, what is the probability, if I invested in the market on any given day, that my forward one-year returns would be positive? Essentially, what is the base rate for the most simple market prediction, buy and hold?

Here’s the table:

DJIA, 1897 to 1916: 56%

DJIA, 1914 to 1968: 65%

Wilshire 5000, 1980 to present: 79%

The odds are pretty good. The modern era also seems markedly better than the first two sets of data. I suspect this is due to a few factors: 1) we are using a broader index of stocks post-1978, 2) Monetary policy post-1971 targeting inflation without the strictures of the gold standard, and 3) Increasingly stringent capital market regulations.

Regardless, the data is clear, it pays to be patiently optimistic.

Of course, I took this analysis a few more steps. Next, I asked, what is the forward positive return probability when the curve is inverted? That falls to 43% and throws a little shade on our optimism. But, if you can handle a little volatility in your portfolio and ask what is the probability, when the curve is inverted, that my forward one-year returns are greater than -20%? That is 87%. So, even when the curve is inverted, only 13% of the time do things get really ugly. Chart below. For two more bonus analysis rounds, I did some Bayes analysis and looked at returns/volatility during each presidential term; you can examine both in the code below.

If you’re not a number nerd like me, here’s the 411: you’d have to be very good at prediction, with a very disciplined approach, to have better odds than simply being a passive long-term investor. It pays to be a patient optimist.

John

# Setup

library('quantmod')

library('PerformanceAnalytics')

# helper function

# Get Data

djia1 <- getSymbols('M1109AUSM293NNBR', src = 'FRED', auto.assign = F); names(djia1) <- 'DJIA1'

djia2 <- getSymbols('M1109BUSM293NNBR', src = 'FRED', auto.assign = F); names(djia2) <- 'DJIA2'

W5K <- getSymbols('WILL5000INDFC', src = 'FRED', auto.assign = F); names(W5K) <- 'W5K'

W5K <- W5K['1980::'] # Data before then is monthly not daily

crv <- getSymbols('T10Y3M', src = 'FRED', auto.assign = F)

head(djia1); tail(djia1) # 1897 to 1916

head(W5K); tail(W5K) # 1978 to Present

# Add fwd prices and 1y fwd returns

djia1\$DJIA1_fwd <- xts(fLead(coredata(djia1), 12), order.by = index(djia1))

djia1\$DJIA1_fwd_rtns <- (djia1\$DJIA1_fwd / djia1\$DJIA1) - 1

djia1 <- na.omit(djia1)

djia2\$DJIA2_fwd <- xts(fLead(coredata(djia2), 12), order.by = index(djia2))

djia2\$DJIA2_fwd_rtns <- (djia2\$DJIA2_fwd / djia2\$DJIA2) - 1

djia2 <- na.omit(djia2)

W5K\$W5K_fwd <- xts(fLead(coredata(W5K), 252), order.by = index(W5K))

W5K\$W5K_fwd_rtns <- (W5K\$W5K_fwd / W5K\$W5K) - 1

W5K <- na.omit(W5K)

# Percentage of time forward 1y returns are positive; .57 / .65 / .79 / .04

sum(djia1\$DJIA1_fwd_rtns > 0) / length(djia1\$DJIA1_fwd_rtns)

sum(djia2\$DJIA2_fwd_rtns > 0) / length(djia2\$DJIA2_fwd_rtns)

sum(W5K\$W5K_fwd_rtns > 0) / length(W5K\$W5K_fwd_rtns)

sum(W5K\$W5K_fwd_rtns >= -.2) / length(W5K\$W5K_fwd_rtns)

# forward returns when curve inverted; when curve inverted, 43% of returns positive

crv <- na.omit(crv)

crv_data <- na.omit(merge(W5K\$W5K_fwd_rtns, crv))

crv_rtns <- crv_data[crv_data\$T10Y3M <= 0]

sum(crv_rtns\$W5K_fwd_rtns > 0) / length(crv_rtns\$W5K_fwd_rtns)

sum(crv_rtns\$W5K_fwd_rtns > -.2) / length(crv_rtns\$W5K_fwd_rtns)

# plots

par(mfrow=c(2,1))

hist(W5K\$W5K_fwd_rtns, 50, col = 'lightgreen', main = 'Histogram of 1Y Forward Returns', xlab = '1Y Forward Returns'); grid(); abline(v = median(W5K\$W5K_fwd_rtns), col = 'blue')

hist(crv_rtns\$W5K_fwd_rtns, 50, col = 'lightgreen', main = 'Histogram of 1Y Forward Returns WHEN curve inverted', xlab = '1Y Forward Returns'); grid(); abline(v = median(crv_rtns\$W5K_fwd_rtns), col = 'blue')

### Bonus Analysis ###

# Bayes analysis; https://en.wikipedia.org/wiki/Bayes%27_theorem

rtn_threshold <- 0

crv_inv <- crv_data[crv_data\$T10Y3M < 0]

crv_not_inv <- crv_data[crv_data\$T10Y3M >= 0]

# a is probability that 1y fwd returns are over the threshold

a <- sum(crv_data\$W5K_fwd_rtns > rtn_threshold) / length(crv_data\$W5K_fwd_rtns)

# b is probability that the curve is inverted

b <- sum(crv_data\$T10Y3M < 0) / length(crv_data\$T10Y3M)

a_given_b <- sum(crv_inv\$W5K_fwd_rtns > rtn_threshold) / length(crv_inv\$W5K_fwd_rtns)

# we are interested in the probability that fwd returns are below the threshold given the curve is inverted

not_a_given_b <- sum(crv_inv\$W5K_fwd_rtns <= rtn_threshold) / length(crv_inv\$W5K_fwd_rtns)

a_given_not_b <- sum(crv_not_inv\$W5K_fwd_rtns > rtn_threshold) / length(crv_not_inv\$W5K_fwd_rtns)

not_a_given_not_b <- sum(crv_not_inv\$W5K_fwd_rtns <= rtn_threshold) / length(crv_not_inv\$W5K_fwd_rtns)

P_B_A <- b * a_given_b

P_B_not_A <- b * not_a_given_b # this is the conditional of interest

P_B_not_A

P_not_B_A <- (1-b) * a_given_not_b

P_not_B_not_A <- (1-b) * not_a_given_not_b

P_B_A + P_B_not_A + P_not_B_A + P_not_B_not_A

# Presidents

Reagan <- '1981::1988'

Bush1 <- '1989::1992'

Clinton <- '1993::2000'

Bush2 <- '2001::2008'

Obama <- '2009::2016'

Trump <- '2017::'

presidents <- c('Reagan', 'Bush1', 'Clinton', 'Bush2', 'Obama', 'Trump')

term_length <- c(8, 4, 8, 8, 8, 2.5)

# Calcs

W5K\$daily_rtns <- CalculateReturns(W5K\$W5K)

pres_vols <- c(sd(na.omit(W5K\$daily_rtns[Reagan])) * sqrt(252),

sd(na.omit(W5K\$daily_rtns[Bush1])) * sqrt(252),

sd(na.omit(W5K\$daily_rtns[Clinton])) * sqrt(252),

sd(na.omit(W5K\$daily_rtns[Bush2])) * sqrt(252),

sd(na.omit(W5K\$daily_rtns[Obama])) * sqrt(252),

sd(na.omit(W5K\$daily_rtns[Trump])) * sqrt(252))

pres_annrtns <- c(Return.annualized(W5K\$daily_rtns[Reagan]),

Return.annualized(W5K\$daily_rtns[Bush1]),

Return.annualized(W5K\$daily_rtns[Clinton]),

Return.annualized(W5K\$daily_rtns[Bush2]),

Return.annualized(W5K\$daily_rtns[Obama]),

Return.annualized(W5K\$daily_rtns[Trump]))

presDF <- data.frame(Presidents = presidents, TermLength = term_length, AnnualVol = pres_vols, AnnualReturns = pres_annrtns)

# plot

ggplot(presDF, aes(x = AnnualVol, y = AnnualReturns)) +

ggtitle('Annualized Returns and Volatility by President since 1980') +

geom_point(aes(color = Presidents, size = TermLength)) +

geom_smooth(method = 'lm')

# regression

summary(lm(presDF\$AnnualReturns ~ presDF\$AnnualVol)) See All