When is a good time to invest in the stock market?

The answer’s easy – when future returns are going to be fantastic. The only problem is, how do you know when future returns are going to be fantastic? I mean, was it possible to know that in 1985 we were going to have an amazing, almost seven-fold run up over the next 15 years? Or that, in 1999, we were going to go precisely nowhere for more than a decade?

Let’s start by looking at what the market is.

As I’m sure you already know, the FTSE 100’s price level represents the market value of the largest 100 companies on the London Stock Exchange.

So let’s break that down into two bits. One bit is the 100 companies; the other bit is the market value.

The 100 companies are things like Vodafone, HSBC and BT, some of which are vast, multi-billion pound operations which span the globe and employ tens, if not hundreds of thousands of people. Most of them have existed for decades, and some of them have existed for centuries. So keep that in your mind – 100 vast companies, multi-billion pound profits, operations all over the globe, and decades of operating history.

The other side of the market is the price. This is the thing that we see whizzing up and down on computer screens, or failing that, we see it being announced on the news every evening. If it goes up or down a lot in a short period, everybody starts getting excited and we might see pictures of traders shouting into a telephone.

The next question is – what is the relationship between these two things, these 100 companies and this FTSE 100 number we see bobbing up and down?

The answer is, not very much actually. There is not a particularly hard link between what the 100 companies are doing out there in the world, how much profit they’re making, how much they’re growing and how much they’re returning to shareholders as dividends, and that market price.

That isn’t to say there’s no relation at all. That’s why we haven’t seen the FTSE 100 fall to 1,000 in this bear market, and it didn’t go to 100,000 in the last bull market. So the FTSE 100 isn’t on a drunken, or random, walk with no restrictions. It’s more like a drunk, staggering down the road towards a pub a mile or so in the distance. The journey may zig and zag, but every time he starts to veer off into the weeds he makes an extra effort to head back towards the middle of the road (or the pavement if he wants to avoid being run over).

Let’s try to get back on topic with some charts from the stock market, in this case the S&P500 from the US, because there’s much more data available for it. The data I’m using here is available from the very excellent Robert Shiller.

Here is the real (inflation adjusted) price of the S&P over the past 100 or so years.

As you can see, the market zigs and zags around like a drunk. However, in the longer-term the price is headed up and to the right because it’s loosely tied to the earnings of the underlying 500 companies, which tend to grow with inflation, plus a bit more.

Here’s another chart, this time of the real earnings of those 500 companies over the same time period.

Well, that’s pretty ziggy and zaggy too. In some years these companies make a lot of money, but sometimes, like the early 1930’s, they hardly make any money at all. But again, you can see that in general the earnings march upwards, in real terms, over time. Given how volatile those earnings are, I’m not surprised that the price moves all over the place if the earnings do too.

So let’s see how well the market does at keeping in the middle of the road, in other words, at keeping in step with the current earnings of those 500 companies.

I think by and large I’d have to say that’s not a bad job; most of the time the market (in red) keeps more or less in pace with earnings (in blue).

There are some notable exceptions though. In about 1915 there’s a massive spike in earnings that the market didn’t seem to notice. Then again in the 1950’s earnings have jumped up and the market takes a while to respond. Then in the 1970’s and 80’s the market seems to be lagging behind earnings, only to overshoot in the late 90’s, more or less until today (or at least 2010 which this data goes up to).

So now we know at least part of the reason why the market moves around all over the place – because it’s trying to keep track with the current earnings. Just look at the 1920’s bubble which ended in the 1929 crash. In terms of keeping up with earnings, the market did a good job. Earnings are shooting up and at the same time so is the market. Then earnings fell off a cliff into the Great Depression and so did the market; so full marks to Mr Market on that one.

This drunken walk, both of earnings and the market, is basically what the efficient market crowd mean when they say that the market follows a random walk. Earnings zig-zag around, and they’re not predictable. The market zigs and zags around somewhere near some multiple of the earnings (in the chart above the market plot is set at 12.5 times the earnings), and that’s also not predictable.

That’s why the academics mostly say that we should just buy the index and settle for the market rate of return.

 Special Offer: Invest like Buffett, Slater and Greenblatt. Click here for details »

But I want more than the market rate of return.

Perhaps there is some way of cancelling out some of the noise so that we can get a clearer picture of what’s going on.

One way to do this is to look at earnings over a longer period than just the last year. There is no magic amount, but a popular smoothing technique is to average out the real earnings over the previous 10 years, and compare the earnings to that.

So what does this real 10 year average earnings look like?

Wow, that’s about the most boring chart ever. Although, it does show that even over 10 year periods, the earnings of a national index can go down, in real terms, for a long period of time. Just look at the 1920 to 1940 period. But it’s a whole lot smoother than just the yearly real earnings, so perhaps this might give our drunken market something a little easier to follow?

Surely if the market was focused on this 10 year earnings period it wouldn’t zig and zag around nearly as much as it usually does? Let’s put the two together.

Well I’m no rocket scientist but that sure does look better to me. The green line (real 10 year average earnings) provides a much clearer signal, or middle of the road, for the market to follow. Once again, the market seems to do a pretty reasonable job of tracking earnings over a century. But, again there are some notable exceptions.

This time the first exception is around 1920. The market falls off a cliff and is miles away from its normal multiple to the 10 year average earnings. This is a clear signal that the drunk (the market) is well off track and should get on back to the middle of the road. And that’s exactly what it does.

Between 1920 and 1926-ish, the market has an incredible run up, with huge returns from that low point which was clearly visible to anybody who had been looking at the 10 year average ratio at the time.

The next exception is the 1929 peak. This time the market is way above the earnings line, and again it’s a clear signal that something needs to change… and it does. This time the market crashes and all those who were in the market at those high valuations get burned very badly.

The interesting thing here is that by looking at the one year earnings there was absolutely no warning signal at all. The price to one year earnings chart shows the market doing its job, tracking earnings. It’s only by looking at the 10 year earnings (or other similar measures like Tobin’s Q) that it becomes clear that something is out of whack.

This pattern happens again in 1932, when again the market is way off line. Then it’s clear that the market may be overvalued in the late 60’s, undervalued in the late 70’s and early 80’s, and very overvalued in the late 90’s and early 2000’s.

Some of this is visible with the one year data, but no to the same extent and not as clearly.

And so finally I’m going to get to the point.

The point is that when the market is ‘undervalued’, i.e. when the price is well below its average multiple of earnings, the market is more likely to go up and therefore future returns are likely to be higher. Just look at 1920, 1932, 1941, 1949 and 1983. The times close to these years saw future returns much higher than the market has returned on average.

The standard way to measure an asset classes is by its returns and the variability of those returns. However, very few people seem to take note of how those returns vary over time and how it might be possible to spot, beforehand, when future returns are more likely to be higher.

With hindsight, it’s clear that returns from 1932 were spectacular, while returns from 1929 were terrible. In real terms it took some 30 years to break even from 1929, while the returns in the same period from 1932 were about 500%. That’s a pretty big difference and by looking at 10 year earnings averages at the time, you could have come to the same conclusion, which means you may have been able to sidestep the 80%+ losses in the downmarket by getting out of equities when they were obviously expensive. Even better would have been to get back in in 1932 when they were obviously cheap. But it was only really obvious if you were looking with the right tools, and PE isn’t the right tool.

So one possible answer to the question “When is a good time to invest in the stock market?”, is that the lower the current price is to the average of the last 10 years earnings (real or nominal), the more likely it is that future returns are going to be higher.

Going by history, if the price is only five to seven times that earnings average, then future returns are very likely to be spectacular. If the price is over thirty times that average, then returns are likely to be very bad indeed. Between those extremes is a continuum from good to bad, with average results coming from average valuations.

So, if it’s possible to say that at some times the market is likely to beat its own average returns, like in 1932, is there perhaps some way that we can use that information to produce a market beating strategy?