PsyFi Search

Thursday 16 July 2009

What's Your Financial IQ?

Take the Cognitive Reflection Test

Answer the following questions and then read on.

(1) A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? _____ cents

(2) If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets? _____ minutes

(3) In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake? _____ days

Intuitively Obvious – and Wrong

Each of the questions on the Cognitive Reflection Test has an immediate, intuitive answer – which happens to be completely wrong. If you answered, or even considered, any of 10 cents, 100 minutes or 24 days you’re in good company (yours truly included). Out of 3428 people tested in the original research a third got all three answers wrong while only 17% got all correct. Even many of those who got the answers right had to think at least twice.

These particular questions are the mental equivalent of optical illusions. They appear to be easy, but aren't. Tests of similar complexity which are more obviously difficult yield more correct answers so it's not simply a question of basic maths skills. Something else is going on here. Something, it turns out, that may completely screw up psychology's ideas about the way people think about finance.

Dumb or Smart?

When the respondants were split into groups dependent on how well they’d done on the CRT and (painlessly) probed for underlying differences what was found, broadly, is that those who did well are generally smart and those who did badly are generally dumb. Which is the psychologist's equivalent of discovering that gravity makes things fall by dropping dumbbells on your feet.

However, when the "smart" and "dumb" groups were investigated further something genuinely interesting showed up. The "dumb" group turned out to behave exactly like behavioural finance predicts but the "smart" group didn't. Which is a bit of a problem for the economists who've spent thirty years developing models for making money which rely on everyone acting dumb. It's roughly the equivalent of discovering that gravity doesn't work the same way everywhere; which can be quite disconcerting if your dumbells suddenly float off into the distance.

The Findings of the CRT

There were two main findings from the CRT. The first is that the "smarter" people were better at discounting time. Or, in lay language, they're prepared to wait longer for a larger reward rather than taking a small, certain amount immediately – as long as the odds favour them. However, this only applies in the short-term – once timelines stretch out this effect disappears, which is also sensible, since the longer you have to wait for your delayed reward the more likely it is to never occur.

It’s the second finding that’s really interesting, though. Prospect Theory, the cornerstone of behavioural finance describing people’s risk taking behaviour, predicts that people are risk adverse when protecting a gain and risk takers when chasing a loss. This was spectacularly true of the low CRT scoring group, but not true at all of the high scoring group – suggesting that cognitive ability, aka I.Q., is critical in evaluation of decision making theories.

Or, to put it bluntly, behavioural finance is wrong.

What Do I.Q. Tests Measure?

Shane Frederick, the researcher, compared respondents' CRT scores with results from other I.Q. tests and discovered a good but not perfect correlation. The CRT is measuring something similar, but not precisely the same as these other tests. But what do I.Q. tests measure anyway?

The founder of I.Q. testing, Alfred Binet, developed the concept to track improvements in learning, not absolute levels of intelligence. Scientists to a man (and they’re nearly always male) have proceeded to ignore him and spent the century or so since drawing bell curves to “prove” their preconceived hypotheses.

It was the U.S. Army in World War 1 that picked up the idea as a general measure for recruits and it’s been downhill ever since. Stephen Jay Gould in The Mismeasure of Man lays out the whole sorry story. It’s almost impossible to quickly convey exactly how manipulated the idea of I.Q. tests has been, but the concept keeps on rearing its ugly head like a particularly demented, stake-proof, garlic-loving vampire. By way of example, in the U.K. the academic Cyril Burt’s separated twin studies proved that genes are more important than upbringing in intelligence – i.e. that intelligence is innate, not learned. His findings influenced half a century of British education so it was just a tiny little bit of a shame that it turned out he’d faked his results.

So we can say, fairly safely, that whatever the CRT is measuring it’s not exactly intelligence. Frederick argues that it’s showing something he calls “cognitive reflection”—the ability or disposition to resist reporting the response that first comes to mind. If correct, why should this matter so much?

Does the CRT Invalidate Prospect Theory?

Prospect Theory tells us something about how people make decisions under conditions of uncertainty. The new research is telling us that some, thoughtful, people will make different decisions from those predicted by the theory and that we can identify these people, ahead of time, by using the CRT.

Follow up work from Frederick has indeed shown significant differences in risk taking behaviour amongst low and high CRT groups. Which would you prefer – the certainty of $500 now or a 15% chance of $1 million?

High CRT scorers overwhelmingly went for the gamble – 82% of them chose it while only a quarter of low CRT scorers gambled. This leads us straight into the quagmire of moral relativism: who decides what a “better” choice is and who is to say that the higher CRT scorers are making “better” decisions than the lower CRT scorers?

Dumb Money

Well, I will for one. Choosing five hundred bucks over a better than 1 in 7 chance of a life changing amount of money would be a totally bloody stupid decision no matter how poor, ignorant and desperate you are.

Still, this is all theoretical and we just don’t know how this would work in real situations. We’ve seen a similar proposal before from John A. List, whose staged but naturalistic experiments on trading behaviour suggested that experienced traders could overcome the loss aversion traits shown in the original research. Since then, as we’ve also seen, a completely naturalistic study using professional golfers has provided support for the original findings of Prospect Theory, throwing doubt on whether List’s experiment is really replicating real-world conditions or is somehow generating something even more artificial.

Separating genetic capabilities and learned ones is more like unbaking a cake than unwinding tangled threads. However, if the CRT tells us anything about investing it's that we need to think hard and think properly and to take our time about it. Such skills can be learned and there are techniques investors can use to help develop them: just as long as they recognise they need them in the first place, of course.

Oh Yeah, The Results

The Cognitive Reflection Test (CRT) was developed by Professor Shane Frederick as described in Cognitive Reflections and Decision Making.

The correct answers are:
  1. 5 cents (not 10 cents)
  2. 5 minutes (not 100 minutes)
  3. 47 days (not 24 days).

Related Articles: Loss Aversion Affects Tiger Woods, Too, The Death of Homo economicus, Pascal's Wager - For Richer, For Poorer

23 comments:

  1. Call me dumb but I got both answers right for questions 2 and 3 but for the life of me, I still can't understand why the answer to question 1 should be 5 cents instead of 10 cents.

    ReplyDelete
  2. x+100+x == 110
    solve for x

    ReplyDelete
  3. hey first time seeing your blog, interesting stuff. Came here via AbnormalReturns. Have you done much reading on the topic of behavioral finance by chance? Only reason I ask is because we track hedge fund portfolios on our blog & hedge fund Blue Ridge Capital had a recommended reading list on that topic and was curious as to your take on some of their recs: http://www.marketfolly.com/2009/05/behavioral-finance-recommended-reading.html

    Pleased to make the acquaintance and look forward to reading more.

    Jay

    ReplyDelete
  4. Got all three right. And I would pick the chance for a million vs. $500. The payoff is just too good for those odds.

    ReplyDelete
  5. I wouldn't say that Behavioral Finance is wrong as a result of this. There are minorities for which it is not true, and many of them, like Buffett, have high IQs.

    PS -- those were easy questions, but then, I'm a mathematician and used to deception in question design.

    ReplyDelete
  6. RE Q2: 100 minutes could be correct. There is nothing in the question that states the machines and their production rates are identical. Heck, they could all be making 5 different widgets, each with different production rates.

    Machine 1: 1 bat / min
    Machine 2: 1 ball / 200 min
    Machine 3: 1 car / day...etc.

    In the testing, how many people asked, "Not enough information?"

    ReplyDelete
  7. Excuse my stupidity, but still don't get number 1. Why 5 cents? Or why is it x+100+x, not just x+100?

    ReplyDelete
  8. x+y=$1.10 and x-$1.00=y, solve this and you'll see $0.05

    ReplyDelete
  9. ball costs $0.05, bat costs $1.00 more, or $1.05

    I had to read it several times before I realized what it mean't - or maybe its just too late on a Friday at work.

    ReplyDelete
  10. Hi market folly

    That looks like a decent enough reading list. I’m very sceptical, however, that simply knowing about behavioural biases is enough to stop people being afflicted by them – there’s plenty of evidence that even experts can’t stop themselves from suffering from these problems.

    Caldini and McKay are essential reading. I’d suggest anyone interested in the subject generally should start with Sutherland’s “Irrationality” which is not specifically about finance but lays out the whole horrible mess in great detail.

    And, yeah, I’ve done a fair bit of reading (and writing) on the topic :)

    ReplyDelete
  11. Hi David Merkel

    I don’t think behavioural finance is broken because of this, but it looks a lot like other psychological theories which start out with a strong position “everyone acts like this” and are eventually forced to drop back to a more nuanced position “most people act like this most of the time”. Investigating the exceptions to prove the rule is what makes the subject interesting, I think.

    Personally I don’t believe High I.Q. makes a damn bit of difference – I know some very clever people who are simply useless investors. More likely it’s to do with having some very particular and at least mildly obsessive character traits coupled to reasonable smarts. Possibly – just possibly – the CRT is tapping into this: maybe the lower scorers are simply not interested in this type of problem?

    ReplyDelete
  12. Thre are 4 right answers. Do you know the fourth?

    ReplyDelete
  13. How does this help me decide what to do with my GE stock now trading at one third its purchase price?

    ReplyDelete
  14. A quick trick to solve any arbitrary combinations of problem #1 is to add the differential to the given total of both items, divided by 2 then subtracting the result with initial total and thus deriving $ for both items in question, i.e; $1.10 + $1.00 = $2.10 / 2 = $1.05 - $1.10 = -0.05 hence 0.05 & 1.05

    ReplyDelete
  15. The test has nothing to do with personal finance/investing. What's the difference between the private investor, the trader and the fund manager? How about putting the psych finance (PF) testing in perspective. Lastly, I would like to know the proportion of PFs who lost money in the stock market this past year.

    ReplyDelete
  16. I guess it depends on how often you do these types of puzzles / riddles. I love to solve these kinds of analytic problems all the time, so I was able to answer all three questions correctly within 10 seconds (in total).

    I would also like to contest this statement in the article: "once timelines stretch out this effect disappears, which is also sensible, since the longer you have to wait for your delayed reward the more likely it is to never occur.". Isn't it actually the opposite. As long as the overall probability over an infinite sample set is x, the likelihood of occurrence of an event increases the longer it doesn't occur.

    ReplyDelete
  17. "Stephen Jay Gould in The Mismeasure of Man lays out the whole sorry story."

    SJ Gould ... this pseudo-scientist (and his rubbish book) has been completely destroyed here :

    Lewis et al. : "The Mismeasure of Science: Stephen Jay Gould versus Samuel George Morton on Skulls and Bias"
    http://analyseeconomique.files.wordpress.com/2011/09/the-mismeasure-of-science-stephen-jay-gould-versus-samuel-george-morton-on-skulls-and-bias.pdf

    Rushton : "Race, Intelligence, and the Brain: The Errors and Omissions of the Revised Edition of S. J. Gould's The Mismeasure of Man (1996)"
    http://www.lrainc.com/swtaboo/stalkers/jpr_gould_paid.html

    Lynn : "Stephen Jay Gould was admired by journalists but not by scientists"
    http://analyseeconomique.files.wordpress.com/2011/09/science-in-the-service-of-ideology.pdf

    ReplyDelete
  18. I got all three questions right almost instantly, but then I've heard them all in various forms many, many times before as I suspect many in science and technology fields have as well.

    Another difference between $500 and a 15% chance at a million dollars is that the 15% chance might be a scam and those who choose the 15% don't take that into account. I mean, who offers you a 15% chance at a million dollars?

    ReplyDelete
  19. just read this again. great stuff. i use the bat and ball game in my book along with a few others to drive home the point that stopping to think can be useful - sometimes!
    But i restrict my ventures into the Pyschological stuff to just one part of a 5 part book.
    You really should put your ideas (and your superbly articulated thoughts on the ideas of others) into a book also TIMARR.
    it would be a great legacy.

    ReplyDelete
  20. Hi Paul

    You really should put your ideas (and your superbly articulated thoughts on the ideas of others) into a book

    Thank you. Give me, say, two weeks ;-)

    ReplyDelete
  21. "A quick trick to solve any arbitrary combinations of problem #1 is to add the differential to the given total of both items, divided by 2 then subtracting the result with initial total and thus deriving $ for both items in question, i.e; $1.10 + $1.00 = $2.10 / 2 = $1.05 - $1.10 = -0.05 hence 0.05 & 1.05"

    Are you serious? You have invented a formula to get "the answer". I got the other 2 questions, ALTHOUGH THERE ARE 2 answers to question 3 - 1 day or 47 days dependent on whether they are asking about the 1st or 2nd half of the lake - poor question design.

    The answer to this is definitely 10 cents - the bat is $1.00 more than the ball and both cost $1.10, obviously the cost of the ball is the total for both ($1.10) less the cost of the bat ($1.00).

    The only "trick" here is Timarr has given the wrong answer to see how many people want to say they are clever and got all 3 right. In the extreme we have you inventing a formula to rationalise an irrational answer.

    ReplyDelete