PsyFi Search

Thursday 26 November 2009

Rise of the Machines

High Frequency Trading

Proponents of high frequency trading tell us that they’re doing us all a service by improving liquidity in markets, which supposedly benefits everyone. However, to implement their strategy they’re prepared to pay a fee to get microscopically early access to prices. So, if you take the liquidity argument seriously, this suggests that these altruistic traders are paying stockmarket operators to give the rest of us a free ride, out of the kindness of their hearts.

Presumably no one really believes this. However, the trouble is that the rise of the machines exposes the fragile markets of the world to yet another possible source of mass extinction. Time for the Terminator. Again.

Automated and Algorithmic

The concept behind high frequency trading is simple. Practitioners pay a fee to markets to get access to real-time data a tiny fraction ahead of everyone else and then hugely powerful computer systems driven by automated trading software search out short term pricing anomalies and seek to exploit them.

Hidden deep behind this is the idea that although stock prices theoretically follow a random walk – which essentially means that a stock’s price in future is not predictable from its price now (which would thus instantly invalidate a thousand trading strategies) – there is evidence of autocorrelation in markets which says that this isn’t quite true. As Campbell, Lo and MacKinley put it in The Econometrics of Financial Markets:
"The fine structure of securities markets and frictions in the trading process can generate predictability. Time-varying expected returns due to changing business conditions can generate predictability. A certain degree of predictability may be necessary to reward investors for bearing certain dynamic risks."
It may be a wild shot in the dark but I'm guessing the authors think that there may be some predictability in market behaviour. The evidence they quote supports them.

Short-term trading techniques aim to exploit this asymmetry in markets. However, the ultra-fast reaction times of the latest market networks and the power of the supercomputers employed has changed the rules. These systems give the high frequency traders a tiny and fleeting advantage over other market participants. Tiny and fleeting, however, can be enough to make lots and lots of money if you do it often enough.

Asymmetric but Liquid

However, what’s got everyone suddenly hopping around is that it’s been spotted that the systems allow asymmetric, differential price discovery. So, for instance, they can rapidly submit small fill or cancel orders with escalating prices to discover the price lower frequency traders – i.e. the man, woman or extra-terrestrial in the street – is prepared to pay. This is a bit like entering a negotiation in which your opposite number can discover the maximum price you’re prepared to pay in return for a paying a fee to a third-party.

Although everyone's getting excited about whether this is wrong the more important question is whether it matters. The proponents of the approach – primarily the people making money out it – argue that they’re performing a public benefit by driving up market liquidity. Some estimates suggest that half of all trades in New York are driven by the rise of the machines. Greater liquidity drives down the cost of fees so, "obviously", this is good for everyone.

Of course, it isn’t: the idea that a bunch of financial engineers have anyone’s interest at heart but their own is so ludicrous that it makes a mockery of any media outlet that even advances the argument. Lower fees may result but the only people who benefit are the high frequency traders. For everyone else what's gained in lower fees is lost in higher prices as they're sampled by the bots testing our preparedness to pay a higher price.

An Unfair Advantage?

However, the problem doesn't seem to be with differential access to pricing information per se – this happens already where active traders are prepared to pay a fee for real-time prices and market maker information while us more sedentary types are quite happy with delayed prices. You don’t find many traders suggesting that practise should be banned because some people are too poor to afford access or computers.

No, the problem is that in order to use this differential pricing information you need gazillions of MIPS of computing power, which is not available to mere mortals. Only the superstars of the global securities industry can play this game.

To which one has to ask – why is anyone surprised?

Who Cares?

Logically any private investor who’s disadvantaged by the high frequency bots is already shackled by their own faulty psychology. Taking a wide angled view the idea that any small time practitioner can exploit short-term pricing anomalies when up against the might and money of the global securities industries is a bit like asking someone to go powerboat racing in a pedalo. No matter how good they are they're not going to win.

So playing the short-term trading game is, for the most part, simply delivering cash into the ever gaping maws of the investment industry's whales. If it wasn't self-evident before surely the rise of high-frequency trading clarifies that the securities industry has short-term trading weapons private traders can only goggle at? Complaining that it's unfair is to miss the point that if you insist on taking on the US Navy in a rowing boat armed with a herring then getting the United Nations to ban the use of submarines isn't going to make it an even fight. Oops, there goes another nuclear depth charge.

The securities industry will continue to extract cash from those investors foolish enough to believe they can beat it at its own short term game while simultaneously using its profits to pump out the message that short-term trading is profitable. And, it is – for them.

Fundamentally Patient

As far as most private investors are concerned high frequency trading should be irrelevant most of the time. The miniscule differences in prices that the high frequency trading bots are exploiting will make no difference to longer-term returns and the longer-term is the only perspective from which the private investor has any hope of consistently outperforming.

The general inability of the investment industry to think beyond the end of the next quarter means that the long-term investor focused on fundamentals rather than behavioural trading patterns at least has a chance on a playing field slanted in their direction. You can’t beat a bunch of supercomputers at their own game but not even a surfeit of supercharged silicon circuits can detect intrinsic value. In truth the value led investor should scarcely need an abacus as long as they’re in possession of a full set of digits.

Although Excel will do at a pinch.

Terminator’s Not Coming

There are wider problems with the rise of the machines. Firstly, the idea behind the joint stock company wasn’t this type of high speed flipping which divorces ownership and responsibility. Arguably these companies are too important to the world economy to allow them to be destabilised by this type of disenfranchised virtual ownership.

Secondly the use of computer trading mechanisms like this is fundamentally dangerous. These automated models are engaged in an escalating arms race as the boy wizards on either side seek to rapidly code algorithms to gain an advantage over their competitors. Yet these systems are, once again, picking up pennies from in front of the steamroller. It’s a matter of time before a program slips over on an unexpected environmental banana skin and gets squished.

The failure of one of these systems, either due to a set of unforeseen external circumstances or a software bug, will have unpredictable consequences. The results of such a happenstance is problematic, but history is not an encouraging guide. Human mediated systems handled at human speeds are prone to all sorts of unexpected failures. As the high frequency trading programs step out of the shadows we enter another world; and if these machines rise up against us there’ll be no leather clad saviour hiding behind designer shades from the future to save our sorry asses this time.

Related Articles: Quibbles With Quants, Econophysics, Consciousness and Cosmic Karma, It's Not Different This Time


  1. I apologize for this comment being off-topic, but I e-mailed you last week and haven't heard back. In the event my e-mail went to your junk file, I am reproducing it here.

    My company recently launched a site offering tools and ideas for short sellers, including a tool that screens for companies predicted to go bankrupt by their Altman Z-scores. The site,, may be of interest to those of your readers who short stocks (or buy puts on them) as part of their overall portfolio strategy.

    If you become an affiliate, your readers will get a discount on the site's regular membership fee, and you will earn recurring referral commissions on every premium member you refer to the site. The whole process would be seamless for you. You can read more on the site's affiliate program here:

    Please e-mail me if you have any questions or would like more information.

    Thank you.

  2. Your comments re the the unavoidable problems of computerized trading systems interacting with each other are prescient given the KGC problem. But this just addresses the expectable problems we will encounter as such systems evolve. I disagree with your conclusions about the effect of all the high frequency competitors on the little guy. As long as there is competition among the big boys who engage in high frequency price discovery, the results of their endeavors will be reduced to two things: finding out who is the lowest cost provider of high frequency price discovery; and conveyance of the overwhelming bulk of the benefit of high frequency price discovery to the little guy. It's business 101. All business people are self-interested and manically pursuing maximization of profits, assuming they are in a competitive business. The benefits that flow to their customers are a side effect but one that is strongly selected by the process. Walmart doesn't have me in mind when they establish complex supply chain relationships and computerized inventory management systems to deliver to me razor blades at a much cheaper price than I could ever accomplish all by myself. They are fighting like mad to do it better than Target, that's their focus. The result is that, due to competition, I get the razor blades at the absolute lowest price achievable given the state of current technology. I get a benefit many multiples of the profit margin of either competitor. Similarly, in HFT, the retail customer is getting prices slightly better, on average, than would have been available if HFT wasn't present. The benefit, multiplied over all market transactions, is enormously greater than the total profits of HFT. When a firm engaged in "best execution" (Themis comes to mind) is trying to move a large block of stock without changing the price, it's basically trying to legally fleece the other side of the transaction relative to the information they have that the other side doesn't have (to wit, the fact that they know they have much more price-moving volume to buy or sell). This is perfectly legal and understandable, more power to them. What irks them is that HFT can statistically estimate (also legally, without front running or any of the other bogus charges levied against HFT) whether such a buy or sell program is underway, and can slightly adjust prices for the hidden information that is now uncovered by their activities. It ends up giving a fairer price, given everything that's knowable at the time, than the other side of the transaction would have received absent that knowledge. This is the true savings of HFT to the retail investor. It's subtle and indirect, but it's also the simplest form of what every competitive business in the world does. If there weren't a benefit to HFT, the market would never pay for it. The market is a self organizing system that only pays for what it needs to survive. And better price discovery is something it is indeed willing to purchase. As long as their is fair competition amongst those willing to invest in the best possible HFT systems, the end user will benefit from their presence.