Early investigation shows software issues and flood of data caused three-hour trading freeze
A perfect storm of software troubles and massive data flow prompted last week’s NASDAQ trading outage, according to a preliminary investigation by NASDAQ OMX Group.
The three-hour trading halt on August 22 came about as traffic into NASDAQ’s Securities Information Processor (SIP) from the NYSE Arca spiked to 26 times the normal level and a slew of incorrect quotes, ‘connect and disconnect’ sequences and other issues flooded the exchange operator’s system. At the same time, a series of flaws in SIP software prevented the reversion to a back-up program.
‘The catalyst for the SIP failure was a confluence of unprecedented events that overwhelmed the processing capacity of the SIP,’ NASDAQ says in a press statement. ‘A sequence of events combined to create an unprecedented volume of message traffic into the SIP; well beyond the system’s tested capacity of 10,000 messages per port, per second.’
The exchange operator admits that ‘a number of these issues were clearly within the control of NASDAQ OMX’ and states that they will be resolved after further internal investigation.
‘NASDAQ OMX is currently identifying potential design changes to further strengthen the SIP’s resiliency, including architectural improvements, information security, disaster recovery plans and capacity parameters,’ it says.
Trading in thousands of stocks, including Google, Facebook, Apple and other big tech names, was halted from shortly after noon until half an hour before the close of the trading day on August 22.
The outage prompted an initial probe by the SEC, and Mary Jo White, the chairman of the agency, has said she will call a meeting of exchange leaders and major market participants to ‘accelerate ongoing efforts to further strengthen our markets’.
The trading freeze, ‘while resolved before the end of the day, was nonetheless serious and should reinforce our collective commitment to addressing technological vulnerabilities of exchanges and other market participants,’ White says in a statement.
