Some 40,000 computers in New York City were down on Tuesday following a global outage caused by a faulty software update from CrowdStrike. Initially, officials reported that 90,000 devices were affected, but this number quickly ballooned to 300,000 as the true impact of the error became apparent. The city faced challenges in identifying the affected devices due to the nature of the propagation of the issue, requiring IT workers to individually assess each device rather than apply an automated solution across all machines.
Mayor Eric Adams attributed the delayed and inaccurate reporting of the number of affected devices to the tedious process of identifying them. The remediation process involved IT staffers starting the computers in safe mode, deleting the corrupted program, and then rebooting, which could take anywhere from two minutes to half an hour per device. New York City CTO Matthew Fraser explained that the city had to assess each device individually to determine if it had been affected, rather than implementing a blanket fix. He estimated that it would take until the end of the week to rectify the issue and bring all the broken devices back online.
Fraser mentioned that CrowdStrike had introduced an automated fix that could potentially expedite the remediation process, but he emphasized the importance of testing it in a lab environment first to ensure it wouldn’t cause any further issues. While tens of thousands of devices remained offline, the city had to prioritize essential services to ensure critical operations continued running smoothly. Mayor Adams noted that emergency operations, such as 911 services, were never interrupted during the crisis, despite the significant impact on other city agencies.
The global outage, which affected 8.5 million devices worldwide, was triggered by a faulty software update from CrowdStrike, a cybersecurity company that produces software for Microsoft computers. CrowdStrike CEO George Kurtz expressed regret over the situation, stating that it could take some time before it is completely rectified. Kurtz received a congressional letter requesting his testimony on what was described as the largest IT outage in history. The fallout from the incident led to a significant drop in CrowdStrike’s shares, with a 13% decline on Monday following an 11% decrease on Friday.
As the city worked to bring all affected devices back online, officials were faced with the challenge of prioritizing critical services while dealing with the aftermath of the global outage. The process of identifying and fixing the affected devices was labor-intensive, requiring individual assessment and remediation. While the impact of the outage was significant, emergency services like 911 remained operational throughout the crisis, ensuring that essential city services continued to function smoothly. The situation highlighted the importance of robust IT infrastructure and comprehensive contingency plans to address unexpected technical failures in the future.
