Written by Ziba Siddiqui
SAN FRANCISCO (Reuters) – A routine update by CrowdStrike Inc (NASDAQ: CST) to its widely used cybersecurity software that caused its clients’ computer systems to crash worldwide on Friday did not appear to have undergone adequate quality checks before it was deployed, security experts said.
The latest version of Falcon Sensor was supposed to make CrowdStrike customers’ systems more secure against hacks by updating the threats they defend against. But flawed code in the update files led to one of the most widespread technology outages in recent years for companies using Microsoft’s (NASDAQ: ) Windows operating system.
Global banks, airlines, hospitals and government offices were disrupted. CrowdStrike has released information to fix the affected systems, but experts said it will take time to get them back up and running because it requires manually removing the faulty code.
“It seems like the scan or protection they do when they look at the code, maybe that file wasn’t included in that or it slipped through somehow,” said Steve Cobb, chief security officer at Security Scorecard, some of whose systems were also affected by the issue.
The problems surfaced quickly after the update was rolled out on Friday, with users posting photos on social media of computers with blue screens displaying error messages. These are known in the industry as “blue screens of death.”
Patrick Wardle, a security researcher who specializes in studying threats against operating systems, said his analysis identified the code responsible for the outage.
He said the issue with the update was “with a file that contains either configuration information or signatures.” These signatures are code that detects certain types of malicious code or malware.
“It is very common for security products to update their signatures, once a day… because they are constantly monitoring new malware and because they want to make sure their customers are protected from the latest threats,” he said.
He said the pace of updates “may be why CrowdStrike hasn’t tested it much.”
It is unclear how this buggy code made its way into the update and why it was not discovered before it was released to customers.
“Ideally, this technology would have been rolled out to a limited group first,” said John Hammond, principal security researcher at Huntress Labs. “This is a safer approach to avoid a major mess like this.”
Other security companies have faced similar incidents in the past. A flawed McAfee antivirus update in 2010 took hundreds of thousands of computers offline.
But the global impact of the outage reflects CrowdStrike’s dominance. More than half of Fortune 500 companies and many government agencies, including the nation’s largest cybersecurity agency, the Cybersecurity and Infrastructure Security Agency, use the company’s software.
