PsyFi Search

Saturday 13 November 2010

Twits, Butter and the Super Bowl Effect

Eclipse of the Twits

As we more or less know, the sheer randomness of the world makes predicting stockmarket movements not so much a fool’s game as Russian Roulette. If you believe you can outwit the markets on a day to day basis it’s only a matter of time before the hammer falls on a firing cap.

Despite this people keep trying, because it’s a basic human urge to try to make sense out of the nonsensical. The ancient Chinese believed a solar eclipse was caused by a celestial dragon munching up the Sun. At the time that counted as advanced thinking: indeed it still does in some parts of the world, but generally it’s a lot harder these days to believe that people could think stockmarket movements can be predicted, say, by something as random as the trivial ramblings of posters on Twitter. Can’t be right, can it?

Moody Twits Drive the Markets

In Twitter Mood Predicts The Stock Market the researchers find that Twitter mood is 87.6% successful in predicting the daily movements of the Dow Jones Industrial. The researchers suggest that this is an outcome of behavioral finance: the idea being that “financial decisions are driven by emotion and mood”.

Which is true, to some extent, but it rather beggars belief that almost 90% of daily stockmarket movements are attributable to this, even more so that the world’s stockmarket movers intersperse their frantic trading to bother explaining how they’re feeling. Far be it from this less than august journal to argue in favour of the Efficient Markets Hypothesis, but most stock market movements aren’t that irrational most of the time.

The study employs lots of whizzy technology including a “Self-Organizing Fuzzy Neural Network” which sounds like a handy gizmo to have around the office, but is basically a program that has learned to make predictions based on past data – in this case either the last three days of market movements or the same data with various bits of mood related data cleverly extracted from Twitter.

Bangladeshi Butter

To be honest it’s a stunning result but its very accuracy raises questions about whether it’s actually measuring some real correlation with stockmarkets or whether it’s simply a repeat of the case of the Bangladeshi Butter. Because, oddly enough, Bangladeshi butter production figures have also been shown to be a darn fine predictor of stockmarket movements.

In one of the classic papers that all first year economic graduates should be made to learn by heart David Leinweber showed exactly how easy it is to find some way of predicting market movements – in hindsight. However, as Stupid Data Miner Tricks also shows, these models have about as much future predictive ability as a government finance minister in election year.

The Super Bowl Effect

To give another example, one of the more remarkable forecasters of future market movements was the sportswriter Leonard Koppett who successfully predicted the direction of the S&P 500 for many, many years. His original Sporting News article from 1978 shows an eleven year streak, and ended up being correct for 18 out of 19 years.

Impressive, you might think, but your regard for him as a stock market pundit will probably be slightly reduced when you discover that his predictive algorithm was based on whether the winner of the Superbowl came from the AFC or the NFC. The general point, of course, is that you don’t need to invoke complex explanations of behavioral economics when a rather more straightforward explanation is the randomness of the world.

(In)credible Correlations

Which is also what Leinweber set out to show. He too took the S&P 500 and went looking for something – anything – correlated with it. He found:
“Butter production in Bangladesh. Yes, there it is. A simple dairy product that explains 75% of the variation of the S&P500 over 10 years”
By adding in US butter production, US cheese production and the sheep population in those two countries he was able to improve his accuracy to 99%. The point being, of course, that no one in their right mind would believe that this particular set of variables is in any way whatsoever causing the movements of a US stockmarket. However, as he goes on to point out, it’s rather too easy to take exactly the same approach and make it sound like it makes sense:
“If someone showed up in your office with a model relating stock prices to interest rates, GDP, trade, housing starts and the like, it might have statistics that looked as good as this nonsense, and it might make as much sense, even though it sounded much more plausible”.
Patrick Burns has performed an out of sample analysis of the aforementioned Super Bowl Effect, which is interesting in its own right, but ends with a neat summary of how psychological expectations condition our belief, or otherwise, in the predictive capability of economic models:
"People are perfectly willing to believe that economic variables may have predictability for stock markets, but sporting variables are given close to zero credibility. It would take overwhelming evidence to get people to invest based on the outcome of a ball game. To put it in statistical jargon, investors act like Bayesians - results from data are tempered by prior beliefs".
Problem Models

In fact it's simple to construct a model out of historical data. Take two arbitrary data points and connect them via a straight line, make the y axis time and the x axis stockmarket returns. It’s trivially easy to now predict next year’s return, and highly improbable that the prediction will be be correct. However, just adding more data points doesn’t make the model’s predictive capability any better. Leinweber starts out with ten data points:
“We have 10 points on the S&P annual series from 1983 to 1992 … and it hits every annual close exactly. We’ve got 100% in-sample accuracy with only one variable … What closing value the S&P did this method predict for the end of 1993? Minus 63,311. Fortunately for the global economy , it actually turned out to be positive, +445. We seem to have a problem with our model’s out of sample performance”.
The basic problem is something we’ve looked at before: “the market has only one past”. Odd though this may be it actually means we only have one sample to base our models on. What we actually need is to be able to take the data from an infinite number of universes. However, we can’t do that yet so we have to be suitably suspicious of anyone waving around a plausible and unexploded new correlation.

Death by Data Mining

There is so much data out there and so much computing power capable of crunching it, looking for correlations, that there are bound to be very good fits between some data sets. That doesn’t tell us anything about causality but, of course, the more plausibly related the data sets are the more likely we are to be convinced.

Yet if behavioural economics tells us anything, it’s that it’s highly unlikely that complex systems including human beings can be reduced to simple correlations. In the case of our Twitter research, it’s a nice conceit that market movements can be reduced to the random babblings of twitchy tweeters through some kind of wisdom of the crowds behavioural effect. It even sounds sort of plausible. However, as Leinweber points out, in nice friendly large letters:

If it seems to be too good to be true it probably is.

Damn right. Read the paper.



Related articles: Exploiting the Anomalies, Investing in the Rear View Mirror, Correlation is not Causality (and is often Spurious)

6 comments:

  1. it rather beggars belief that almost 90% of daily stockmarket movements are attributable to this

    How so?

    The U.S. stock market has been increasing in value by 6.5 percent real per year for as far back as we have records. The rational take is that this is what it will continue to do from this point forward.

    It follows that any move up of greater than 6.5 percent annually is rooted in emotion and that any move up over the course of a year of less than 6.5 percent is rooted in emotion.

    I don't have at all a hard time going along with the proposition that the vast majority of price moves are rooted in emotion. I certainly agree that there is rationality mixed in as well. But that's like saying that a drunk is rational in the way he goes about the process of buying the drink that is killing him. When we buy overpriced stocks, we are engaging in irrational behavior regardless of whether we add some rational elements (like reading annual reports) into the mix.

    As for the claim that "If it seems too good to be true, it probably is," does this apply to claims that stock prices cannot be effectively predicted? Many people very, very, very much want to believe that stock returns are not predictable (because acknowledging the reality would mean leaving emotional Get Rich Quick approaches behind) and the marketing slogans asserting that "you can't beat the market" or that "timing doesn't work" are nicely tailored to satisfy this widely felt emotional need.

    The Buy-and-Holders like to suggest that it is market timers who believe in something too good to be true. I submit that it is just the opposite -- it is the idea that stocks are the only thing that can be bought or sold for which the price you pay makes no difference that is too good to be true. The price we pay for stocks has always affected the long-term return we obtain from them and always will. Or at least that is my sincere belief based on the work I have done in this area.

    Rob

    ReplyDelete
  2. Thanks for the nice comments on Stupid Data Miner Tricks

    For the latest and even more stupid update, see
    http://nerdsonwallstreet.typepad.com/my_weblog/2009/08/index.html

    Dave L

    ReplyDelete
  3. Hi Dave

    You're welcome. Clickable link here: Back into the Data Mine

    ReplyDelete
  4. I tried to invest in the S&P 500 based on the performance of Bangladeshi Butter, but had to stop when when I couldn't answer my broker's margarine call.

    ReplyDelete
  5. Rob,

    Not sure twitter is the best guide, but your basic thinking is correct.

    I would disagree on one aspect of your point though. The 6.5% real number is for total return, the actual trend growth line of the index is only @1.7% real since 1870. The rest of the return is dividends.

    Of course that means the second part, dividends, can give you some insight as to whether the index (and growth of dividends) is likely to actually achieve its trend growth, or move lower over reasonable time frames.

    Still, since I am aware such valuation based arguments irritate people I would restate your point as:

    "It follows that any move up of greater than 1.7 percent real annually is rooted in emotion and that any move up over the course of a year of less than 1.7 percent real is rooted in emotion."

    I would prefer something different. That any gain when the pricing is such that the combination of the dividend yield and a 1.7% real gain in the index (which should over time approximate growth in real dividends) is less than @6% is irrational, and any gain less than the combination of the dividend yield and a 1.7% real gain in the index when the combination is above @6% is irrational.

    An additional complication, is that that trend growth in real dividends is only about 1.1%. So .6% per year has been the index getting more expensive! So probably we should move that number down to 1.1% real. Almost all of that above trend growth was from the '90's. Looked at that way, it would be unsurprising that since then the market has been working off that growth that exceeded trend growth in real dividends. Thus one would expect that that process will continue until trend growth in the index is close to trend growth in real dividends, or irrationally, below.

    That combines your basic point with the valuation component. Interestingly, over longer time frames the market behaves in just that fashion.

    Lance Paddock

    ReplyDelete
  6. since I am aware such valuation based arguments irritate people

    Thanks for your feedback, Lance.

    The implicit point made in your words quoted above is a big deal, in my assessment.

    People don't get irritated by arguments that they do not perceive as having power, right? So the irritation we see tells us that people get the point. At the same time people continue to invest as if they do not get the point. They get it and don't get it at the same time.

    Humans!

    Rob

    ReplyDelete