Something to know if you use Monte Carlo Analysis

[jfoxcpacfp]

Speaking of relying on historical returns in another thread referring to 1% (no, that's not you evil, filthy rich doctors)...

Just out by Michael Kitces - Does Monte Carlo Analysis Actually Overstate Tail Risk in Retirement Projections?

Read the Executive Summary if you don't want to struggle through the weeds. Very interesting research.

[Zaphod]

Thanks for the link, this is a great article. I was not aware about that aspect of MC, and still unaware of most of how it works.

I actually tried to convey the gist of this article in another thread, stating how ridiculous these assumptions were and that if you expected 40 million dollars upon death on some normal and totally doable savings rate and draw down you were crazy. Yes, I knew the numbers worked, but if it was true you'd see it in real life and we wouldnt all be busting our humps to get a small fraction of that.

Very interesting how there is no mean reverting nature to the Monte Carlo program, maybe that was a simplification or limitation of the math at the time. I have always preferred rolling real returns as it just gives you a context for the MC, anything much above or below that is able to be cut out as unrealistic.

Great article.

Seems very odd that it wouldnt be a super obvious caveat that the program doesnt make any allowance for regimes or patterns and it would be known and communicated as an obvious shortcoming to interpreting the results too literally. Crazy.

[NJDoc]

Interesting , thx for sharing that link.

 

[Donnie]

The article is somewhat confusing as the author uses "Monte Carlo analysis" and "normal distribution" somewhat interchangeably. If there is a problem with the predictions, it would not be the Monte Carlo analysis that is flawed, it would be the assumption of normally distributed returns.

That said, the article has some issues. The author seems to have a tenuous grasp of statistics. He says that because some of the 10k Monte Carlo simulations are way worse or way better than those that have been observed or because some scenarios haven't been observed at the predicted frequency, that must mean the Monte Carlo model isn't working. That's just silly. He has a limited sample size, so "eyeballing it" will lead to misleading conclusions. He would need to actually perform statistical tests to determine whether the observations fall outside the expectations of normally distributed returns. The approach of the article is similar to the gambler's fallacy - just because we flip a coin 10 times and they are all heads, doesn't mean that coin flips aren't random.

[Zaphod]

Donnie wrote:

The article is somewhat confusing as the author uses “Monte Carlo analysis” and “normal distribution” somewhat interchangeably. If there is a problem with the predictions, it would not be the Monte Carlo analysis that is flawed, it would be the assumption of normally distributed returns.

That said, the article has some issues. The author seems to have a tenuous grasp of statistics. He says that because some of the 10k Monte Carlo simulations are way worse or way better than those that have been observed or because some scenarios haven’t been observed at the predicted frequency, that must mean the Monte Carlo model isn’t working. That’s just silly. He has a limited sample size, so “eyeballing it” will lead to misleading conclusions. He would need to actually perform statistical tests to determine whether the observations fall outside the expectations of normally distributed returns. The approach of the article is similar to the gambler’s fallacy – just because we flip a coin 10 times and they are all heads, doesn’t mean that coin flips aren’t random.

Click to expand...

What now? That doesnt make a lot of sense. It stated that the Monte Carlo isnt very accurate, which is true if you can cut off the top and bottom 5% of results and not even eliminate a single historical market result. I believe this post was simply a distillation of a paper that I am confident had lots of statistical analysis. Sample size conclusions makes no sense, we only have the data we have and its more than enough occurrences, and if you run several MCs you get more than enough comparisons.

Bottom line is it isnt working if it isnt helping people model their future and make the right decisions as much as possible. The errors had to do with not acknowledging the regime change mean reverting nature of the market. There were instances of -10% for years followed by a -50%, that would not and has not happened in real life ever. It was an over estimated likelihood on both the high/low due to that factor, which seems a bit ridiculous really. If in 10,000 scenarios a MC gives 10% of results that have never occurred over 200 years, including 2 generational bubbles/recession/bears, than it certainly needs to be assessed.

A MC is simply a model. If the model fails to predict/project accurately to reality, it is not reality that is wrong, and the model needs to be recalibrated to be better.

[Donnie]

Do you believe market returns are random? If yes, then there is a probability distribution that describes that randomness. The probability distribution does not have to be normal. If the correct probability distiribution is used in the Monte Carlo model, then the model will predict reality by defininiton.

Yes, sample size is important, just as it is in the gambler's fallacy example I mentioned. As I said, the author would have to perform a statistical analysis around the returns we have observed and find the probability of that occurring under normally distributed returns. If the probability is too small, than that would bolster the author's claim that a normal distribution of returns is not appropriate.

Japan has a had a series of pretty negative returns, so I wouldn't rule out the normal distribution as being appropriate for equity returns just because we haven't seen such returns in the US. That said, I think people agree that the normal distribution is not perfect, but I don't think it is as far off as the author suggests.

[Zaphod]

Donnie wrote:

Do you believe market returns are random? If yes, then there is a probability distribution that describes that randomness. The probability distribution does not have to be normal. If the correct probability distiribution is used in the Monte Carlo model, then the model will predict reality by defininiton.

Yes, sample size is important, just as it is in the gambler’s fallacy example I mentioned. As I said, the author would have to perform a statistical analysis around the returns we have observed and find the probability of that occurring under normally distributed returns. If the probability is too small, than that would bolster the author’s claim that a normal distribution of returns is not appropriate.

Japan has a had a series of pretty negative returns, so I wouldn’t rule out the normal distribution as being appropriate for equity returns just because we haven’t seen such returns in the US. That said, I think people agree that the normal distribution is not perfect, but I don’t think it is as far off as the author suggests.

Click to expand...

I agree the US market is an outlier in all regards, but there isnt a great reason to see that change much in the intermediate future.

Otherwise, that makes the same mistake that MC makes, that returns are independent of each other on a year to year basis, which isnt true. A 50% down year is not just as likely to be followed by another 50% down year as a 50% up year as the MC treats it. Same is true on the upside of course. It way over estimates your likelihood of a massive estate as well.

Theres basically no valuations/earnings filter, which while still useful is a good lens to interpret and maybe throw off the top and bottom 3% of results as very very unlikely. These are probabilities, and you neednt a 100% probability and if you think you have it you've fooled yourself anyway. It need only be good enough in the most likely of scenarios including a couple bad ones. The bad/good ones shouldnt be so outlandish and unlikely to the point they change how you save/invest/plan. Its likely to end up poorly.

[Donnie]

Since 1990, the Nikkei had two three-year stretches as follows:

1990-1992: -39%, -4%, -26%
1999-2001: -27%, -24%, -19%

Just because somethig didn't happen doesn't mean that the probabilities were wrong.

[Zaphod]

Donnie wrote:

Since 1990, the Nikkei had two three-year stretches as follows:

1990-1992: -39%, -4%, -26%
1999-2001: -27%, -24%, -19%

Just because somethig didn’t happen doesn’t mean that the probabilities were wrong.

Click to expand...

Except we are not Japan nor their market. You realize their demographics are horrendous and they had a stock market and real estate bubble that makes the 2008 great recession look like a day in the park. At one point in time real estate in Tokyo alone was worth more than all the land in the US. The Nikkei rose 4x higher multiple than the SP is currently at. It was an impressive bubble on all fronts. We could get there maybe, but it would take the market more than quadrupling in size from here. If you took the US tech bubble and the real estate bubble (twice as big though) and had them rise and crash at the same time, that would be a good approximation of what happened in Japan. Combine that with their demos and structure/culture and its pretty tough to get out of.

A three year stretch isnt a big deal, but 10 to 15 like you see in those MC simulations are wholly different. We do have a great market and one day it may give way to not being such an outlier, that is a real risk, but it has to be weighed against the structure and demographics of the country of course. Luckily we arent stuck with only buying home as well, was harder in the past.

[Donnie]

You said -50% is way more likely to be followed by +50%. While not 50% swings, I provided examples where there were several of years of negative returns in a row. You go on to say Japan equities are different, and bubbles are easy to spot there. I guess efficient market theory breaks down over there because they aren't the US. Sounds like you have the makings of a winning investment strategy!

[Zaphod]

Donnie wrote:

You said -50% is way more likely to be followed by +50%. While not 50% swings, I provided examples where there were several of years of negative returns in a row. You go on to say Japan equities are different, and bubbles are easy to spot there. I guess efficient market theory breaks down over there because they aren’t the US. Sounds like you have the makings of a winning investment strategy!

Click to expand...

I just picked 50s for example that show how MC and real life are different. Each year is an independent event in MC, which isnt true in the market up until this point. In a MC analysis, if you have several negative returning years, you are no more or less likely to have another negative one, which is not true in the market. Recessions end, expansions end and they follow each other cyclically. The longer you go on with either streak (positive longer than negative of course) the more likely it shifts, this is the big difference.

I dont get what is so shocking about several years in a row of negative returns. I already conceded that the US market is different. In fact if you compare us to several other developed markets you'd find we are the outlier in a lot of categories and things we hold to be truths are idiosyncratic in nature. This is our market and the forces that have made that so in the past may be less but they are not gone entirely. When that happens, ie, loss of the greatest productive capacity in the world, wealth, etc...then you should definitely adjust your expectations.

I dont see what you're trying to say with the Japan bit. All I did was try to describe the sheer massive size of what happened there to give a context to their subsequent issues, I mean if you have not read about it you should it was nuts. You think if the real estate issues were more than twice the size here and the stock market was more overvalued than even the dot com era that we wouldnt be still struggling in a real way? We would as well, luckily our bubbles have been a little less coincident. I never mentioned anything about being able to spot bubbles. I didnt say Japanese equities are different, just that specific bubble, and Japan as a country is very different and hence their GDP and its growth trajectory are different than us. Thats not some left field esoteric assumption, I mean everyone knows that. More than 33% of their population is over 60 and their median age is second oldest in the world (US is 63rd).

[Donnie]

What you are suggesting is that you can recognize patterns in the stock market.  If what you say is correct, then it is exploitable - buy the dips and sell the peaks.  Good luck doing that with an index!

From 1950 to 1988 the Nikkei had a 15% total annualized return.  I'm sure in 1988 everyone was saying the same thing about Japan that you are saying about the U.S., i.e. it's different than other markets.  All bubbles are easy to diagnose once they occur.  For what it's worth, the U.S. population growth rates have been declining for decades and birth rates continue to fall.  The U.S. could easily become Japan in a couple decades if the population begins to shrink as a result of declining birth rates and suppression of immigration.

[jfoxcpacfp]

Donnie wrote:

The U.S. could easily become Japan in a couple decades if the population begins to shrink as a result of declining birth rates and suppression of immigration.

Click to expand...

I disagree. There's no more rapidly aging population in any major economy on the planet than Japan's. Japan never even had a postwar baby boom – the country was simply too devastated – with the result that its age mix is significantly older than ours. This is true even in ways that are not immediately obvious, as – for instance – the growth of population under age 25 in Japan is actually negative, even as ours is positive. In other words: not just more and older old people, but fewer young people.

 

[Donnie]

The US population growth rate is declining.  The US population growth rate averaged :

1.24% during the 1990s.

0.95% during the 2000s.

0.76% during the 2010s.

The general fertility rate fell in 2015 to tie the lowest level on record and has never been lower than the rate recorded in 2014 and in 2013.  Population growth has historically been a big driver of GDP growth.  If the US population contracts, that will have negative impact on GDP growth.

 

 

 

[jfoxcpacfp]

That is a long line to draw to " The U.S. could easily become Japan in a couple decades..."

Some problems with Japan that we don't share