Chart of the day, HFT edition

[smagd]

I, as a sceptic, found this animation of trading frequency helpful in visualizing what's happening to today's stock exchanges:

This astonishing GIF comes from Nanex, and shows the amount of high-frequency trading in the stock market from January 2007 to January 2012. (Which means that the Knightmare craziness of last week is not included.)

The various colors, as identified in the legend on the right, are all the different US stock exchanges. You might think there are only two stock exchanges in the US, but you’d be wrong: there are only two exchanges where stocks are listed. There are many, many more exchanges where stocks are traded.

What we see here is relatively low levels of high-frequency trading through all of 2007. Then, in 2008, a pattern starts to emerge: a big spike right at the close, at 4pm, which is soon mirrored by another spike at the open. This is the era of traders going off to play golf in the middle of the day, because nothing interesting happens except at the beginning and the end of the trading day. But it doesn’t last long.

By the end of 2008, odd spikes in trading activity show up in the middle of the day, and of course there’s a huge flurry of activity around the time of the financial crisis. And then, after that, things just become completely unpredictable. There’s still a morning spike for most of 2009, but even that goes away eventually, to be replaced with sheer noise. Sometimes, like at the end of 2010, high-frequency trading activity is very low. At other times, like at the end of 2011, it’s incredibly high. Intraday spikes can happen at any time of day, and volumes can surge and fall back in pretty much random fashion.

It’s certainly fair to say that if you take a long, five-year view, then you can see a clear rise in trading activity. But it’s also fair to say that there’s something quite literally out of control going on here. Just as the quants at Knight found themselves unable to turn off their machines for 30 long minutes last week, the HFT world in aggregate seemingly has a mind of its own when it comes to trading patterns. Or, to put it another way, if there’s a pattern here, it’s one incomprehensible to human minds.

Back in 2007, I wasn’t a fan of a financial-transactions tax; today, I am. And this chart shows better than anything why my opinion has changed. The stock market is clearly more dangerous than it was in 2007, with much greater tail risk; meanwhile, in return for facing that danger, society as a whole has received precious little utility. Are spreads a tiny bit tighter than they might be otherwise? Perhaps. But that has no effect on stock-market returns for long-term or even medium-term investors.

The stock market today is a war zone, where algobots fight each other over pennies, millions of times a second. Sometimes, the casualties are merely companies like Knight, and few people have much sympathy for them. But inevitably, at some point in the future, significant losses will end up being borne by investors with no direct connection to the HFT world, which is so complex that its potential systemic repercussions are literally unknowable. The potential cost is huge; the short-term benefits are minuscule. Let’s give HFT the funeral it deserves.

So, should I be scared?

[SAI Peregrinus]

Yes, you should be scared. HFT adds nothing to the market, and increases volatility. The lie that it increases liquidity is common, but since all trades that HFTers execute are trades that would happen anyway they are simply unnecessary middle men.

[Aea]

Yes, you should be scared. HFT adds nothing to the market, and increases volatility. The lie that it increases liquidity is common, but since all trades that HFTers execute are trades that would happen anyway they are simply unnecessary middle men.

This is absolutely correct. For the most part HFT to add liquidity is unnecessary in the modern market due to the sheer volume of activity. There is no more need for a market maker to exist, since trades would happen regardless. The reason HFT exists is because most exchanges operate on a chronological basis. If my order beats another order by 1ms (often much much less) then it is processed first. If orders were simply batched (one batch every second) the whole allure of HFT would disappear. It is also a huge resource drain, instead of building something useful thousands of brilliant people are trying to figure out how to put a market order a few microseconds in front of their competitor to make some money.

[Zumwalt]

High frequency trading does increase volatility, and I think it edges out the smaller players. In fact, I would argue that it decreases liquidity because the machines enter/exit positions so quickly that they take shares (provided by the market maker) that could have been traded by human traders. This increased volatility then freaks the market makers, who pull their orders, and we get something similar to a flash crash, or the market crash that we saw last year.

On a side note, check this out. Algo traders can cause weird patterns when they trade.

http://blogs.wsj.com/marketbeat/2012...reaming_stream

[Bartholomeus Crane]

Although it will cost me money (not from trading), I should say you should be very scared. Little flash crashes already happen regularly. Trading algorithms running unsupervised costing the trading house lots of money aren't uncommon, but are ofcourse hidden away and not talked about. And now you big trading firms on wallstreet taking very small positions for just a millisecond. Not for trading, just to see who's biting and where is what.

HFT is bound to at one bring down a lot more than just some small fry. The system is becoming ever more complex and ever more uncontrollable. It is just waiting for one of the big boys to lose heavily because of an automated millisecond trade that somehow got away.

Transaction or Tobin tax is needed. Not just for nice government revenues, but simply to slow the market down, decrease volatility.

[KathDougans]

Reform of trading and talk of things like Tobin taxes will probably come about after the Chinese bots end up owning everything on Wall Street.

[Bartholomeus Crane]

Reform of trading and talk of things like Tobin taxes will probably come about after the Chinese bots end up owning everything on Wall Street.

More like after the massive crash of 2013 ...

[Zumwalt]

Reform of trading and talk of things like Tobin taxes will probably come about after the Chinese bots end up owning everything on Wall Street.

More like after the massive crash of 2013 ...

If a massive crash were to happen between now and November, it would be an important issue before the election that may get fixed!

[Rudolf Miller]

Reform of trading and talk of things like Tobin taxes will probably come about after the Chinese bots end up owning everything on Wall Street.

More like after the massive crash of 2013 ...

If a massive crash were to happen between now and November, it would be an important issue before the election that may get fixed!

Nope.

But I agree. It's scary what the pursuit of a dollar has led to. And the ones who will pay the most is the average Joe Beergut when his 401k accidentally 0's out because the algorithm facepalm'd.

Which of course will be Obama's fault in a red state, Republican's fault in a blue state.

[Bartholomeus Crane]

I seriously doubt there is going to be a massive crash in the US before November. There's just too much slack in the US economy for that to happen this soon. Especially with Europe on the brink in the meantime. After that all bets are off though ...

  Spoiler:

I firmly believe that something massive is coming. It seems to me all parties (not just political) in the US are trying to make this happen. What it is that will set it off I don't know, there are just too many possible candidates right now (Europe, China, more massive scandals to do with the banks, hidden liabilities that surface and kill off a too-big-to-fail bank, some algorithm going haywire, all of the above?). But once it will go down it will pretty much snowball into a failcascade and I can't think of something that will stop it. It just seems to me that none of the traditional negative feedback loops still function. Perhaps, just maybe, if the Republicans get beaten pretty harshly in November and lose both the house and the senate as well as the presidential race, there will be the political will or unity to do something to mitigate the fallout somewhat. But I doubt that'll happen actually. They're basically in just as deep. 2013 and/or 2014 are going to be very interesting.

[Nicho Void]

I seriously doubt there is going to be a massive crash in the US before November. There's just too much slack in the US economy for that to happen this soon. Especially with Europe on the brink in the meantime. After that all bets are off though ...

  Spoiler:

I firmly believe that something massive is coming. It seems to me all parties (not just political) in the US are trying to make this happen. What it is that will set it off I don't know, there are just too many possible candidates right now (Europe, China, more massive scandals to do with the banks, hidden liabilities that surface and kill off a too-big-to-fail bank, some algorithm going haywire, all of the above?). But once it will go down it will pretty much snowball into a failcascade and I can't think of something that will stop it. It just seems to me that none of the traditional negative feedback loops still function. Perhaps, just maybe, if the Republicans get beaten pretty harshly in November and lose both the house and the senate as well as the presidential race, there will be the political will or unity to do something to mitigate the fallout somewhat. But I doubt that'll happen actually. They're basically in just as deep. 2013 and/or 2014 are going to be very interesting.

How did this go from a discussion about HFT to a, "we're all doomed unless Democrats gain control of everything" statement?

Edit: Expanding: I think both sides of our political system are well aware and are in bed with the people driving these kind of trades.

[Rudolf Miller]

I stand by the genuine suffering of all the classes will lead to a road to redemption.

[punkboy101]

I seriously doubt there is going to be a massive crash in the US before November. There's just too much slack in the US economy for that to happen this soon. Especially with Europe on the brink in the meantime. After that all bets are off though ...

  Spoiler:

I firmly believe that something massive is coming. It seems to me all parties (not just political) in the US are trying to make this happen. What it is that will set it off I don't know, there are just too many possible candidates right now (Europe, China, more massive scandals to do with the banks, hidden liabilities that surface and kill off a too-big-to-fail bank, some algorithm going haywire, all of the above?). But once it will go down it will pretty much snowball into a failcascade and I can't think of something that will stop it. It just seems to me that none of the traditional negative feedback loops still function. Perhaps, just maybe, if the Republicans get beaten pretty harshly in November and lose both the house and the senate as well as the presidential race, there will be the political will or unity to do something to mitigate the fallout somewhat. But I doubt that'll happen actually. They're basically in just as deep. 2013 and/or 2014 are going to be very interesting.

Wow

I highly doubt that the Democrats will be able to do anything even if they gain control of the house and senate. . As both you and Nicho said, they are in just as deep with the people causing this problem, and those people make sure that they own whoever is in office. There is just to much risk and money involved to back one or the other, so they give to both.

These guys don't lose, no matter what the situation they profit. I agree with the whole "shit is about to go tit's up in a big way", and it pisses me off that for the most part the Governments in power have done next to nothing to solve the issues that caused the shitstorm in the first place, nor held anyone accountable for any number of failures or what amounts to flat out fraud/criminal behavior. If some of these people had gone to jail, if some of these banks had been given huge fines (in the low billions, not low hundred millions), and if legislation with teeth had been introduced and "too-big-to-fail" banks had been broken up, maybe, just maybe we would've been able to avoid the coming crash.

Instead we blame the poor/unemployed for being lazy, cut benefits to those that need them most, take peoples house's away from them (some are deserving but most are not) and continue to throw billions of money at the people who caused the issue. In reality we are the ones that will suffer in any coming finacialarmageddon, and we will be the ones to blame because we didn't hold politicians or business leaders accountable, we sat back and took the cuts, watched friends lose everything they worked for (jobs, houses, pensions, savings) , and got tricked into demonizing the poor while the global elite continue to make off like bandits.

Expecting or relying on the Government to look after your interests (Democrat or Republican, Labour or Conservative, Left wing or Right wing) is like asking the local fox to look after your chicken coop while you're away, it's not going to end well.

[Bartholomeus Crane]

I seriously doubt there is going to be a massive crash in the US before November. There's just too much slack in the US economy for that to happen this soon. Especially with Europe on the brink in the meantime. After that all bets are off though ...

  Spoiler:

I firmly believe that something massive is coming. It seems to me all parties (not just political) in the US are trying to make this happen. What it is that will set it off I don't know, there are just too many possible candidates right now (Europe, China, more massive scandals to do with the banks, hidden liabilities that surface and kill off a too-big-to-fail bank, some algorithm going haywire, all of the above?). But once it will go down it will pretty much snowball into a failcascade and I can't think of something that will stop it. It just seems to me that none of the traditional negative feedback loops still function. Perhaps, just maybe, if the Republicans get beaten pretty harshly in November and lose both the house and the senate as well as the presidential race, there will be the political will or unity to do something to mitigate the fallout somewhat. But I doubt that'll happen actually. They're basically in just as deep. 2013 and/or 2014 are going to be very interesting.

Wow

I highly doubt that the Democrats will be able to do anything even if they gain control of the house and senate. . As both you and Nicho said, they are in just as deep with the people causing this problem, and those people make sure that they own whoever is in office. There is just to much risk and money involved to back one or the other, so they give to both.

These guys don't lose, no matter what the situation they profit. I agree with the whole "shit is about to go tit's up in a big way", and it pisses me off that for the most part the Governments in power have done next to nothing to solve the issues that caused the shitstorm in the first place, nor held anyone accountable for any number of failures or what amounts to flat out fraud/criminal behavior. If some of these people had gone to jail, if some of these banks had been given huge fines (in the low billions, not low hundred millions), and if legislation with teeth had been introduced and "too-big-to-fail" banks had been broken up, maybe, just maybe we would've been able to avoid the coming crash.

Instead we blame the poor/unemployed for being lazy, cut benefits to those that need them most, take peoples house's away from them (some are deserving but most are not) and continue to throw billions of money at the people who caused the issue. In reality we are the ones that will suffer in any coming finacialarmageddon, and we will be the ones to blame because we didn't hold politicians or business leaders accountable, we sat back and took the cuts, watched friends lose everything they worked for (jobs, houses, pensions, savings) , and got tricked into demonizing the poor while the global elite continue to make off like bandits.

Expecting or relying on the Government to look after your interests (Democrat or Republican, Labour or Conservative, Left wing or Right wing) is like asking the local fox to look after your chicken coop while you're away, it's not going to end well.

That's quite a wall of text you got going there.

But my point with the Democratic win in the US is actually quite simple.

As long as the Republicans retain enough seats it is quite obvious that nothing will ever be done. You said it yourself: nothing proper has been done by the US government to prevent the next big fallout from happening. But look who blew up the talks on the economy? Who pushed so much shit that the US got downgraded? Who is always rooting for the wealthy and the expense of the middleclass (let alone the poor)? You blame the government collectively for this. I just draw the line that little bit finer: The Republicans have been sabotaging everything the government could have done since 2008. Because it suits them politically. It makes their point: Obama is not just black but ineffective as well, and government is the problem. Jadajadajada.

That doesn't let the Democrats off the hook either. They're not pushing back hard enough. But in the end, a vast proportion of the things that could be done, for example to curtail high frequency trade, automatic trading, derivatives; they all fail because of the intransigence of the Republicans to, you know, actually govern responsibly. And as long as they can derail every government action it is quite obvious that nothing is going to change and that the US economy will blindly drive off that cliff.

Having said that, even in the unlikely case of the Democrats winning big in November, the possibility of them being able to then mitigate the impending meltdown is slim to non-existant. Because I agree: things are too far gone. But perhaps with that political unity they will be able to mitigate some of the fallout. Or, you know, actually govern. Perhaps.

[Zeekar]

Barth there is some irony in you accusing any body of posting a wall of text.

[Bartholomeus Crane]

Barth there is some irony in you accusing any body of posting a wall of text.

There is

[Frug]

That's quite a wall of text you got going there.

This should be my new signature, c/d?

No but on topic, this stuff is weird. I remember last year we had a topic including the beauty of HFT and all the weird patterns that come out of their algorithms interacting with eachother.

How much money is actually generated with such trading going on? Is it worth it or are people just playing around?

[SAI Peregrinus]

That's quite a wall of text you got going there.

This should be my new signature, c/d?

No but on topic, this stuff is weird. I remember last year we had a topic including the beauty of HFT and all the weird patterns that come out of their algorithms interacting with eachother.

How much money is actually generated with such trading going on? Is it worth it or are people just playing around?

HFT does not generate money. It leeches money from various industries, on the order of tens to hundreds of billions of dollars, if not more. All the money coming out of HFT is money that would otherwise have been available as investments in businesses. It's certainly worth it to the people involved, to everyone else it's a net drain.

[Ophichius]

HFT is simply the 0.01 ISK game, played by computers a trillion times faster than it's played in Eve.

So expect similar returns and complete and utter lack of utility to anybody who actually does shit.

-O

[Bartholomeus Crane]

It is certainly worth it. For the big trading firm especially. This has been a trend for years now.

The simple fact is that human traders, in the long term average, are considered 'good' if they trade above 51% to maybe 52%. Some stake it out higher, but never in the long term. Note that ofcourse, this leaves out 'insight trading', which is basically where the real money is made. In the end, come what may, automatic or robot trading, nowadays, doesn't really, on average, do any worse or better than 'human trading'. And this is why the big traders on wall street (but also in London etc.) have been putting shitloads of money into this thing.

It isn't just the model side of things (greenlighting what is good or not based on a collectivation of information no human could ever approximate). By now the whole process/mechanism/whatever has been algorithmatised. Human traders are now making bets on a green light they have no, and will never have any, clue what it means or why it is green. The idea of: "Computer says so" is no longer outlandish, big ego or not.

The parallels are striking. As much as society has been financialised, in the same measure the financial sector has been algorithmatised. By now, some companies are spending millions of dollars just to be within an extra 100 yards closer to the digital backbone of Wall Street. Apparently that fraction of a millisecond is enough to justify buying prime real estate in the most expensive part of the the US.

All in all, unless you're a shmuck who'll invest in a Facebook IPO, this shit is pretty big.

The question now is: when is enough enough?