CrowdStrike Outage: What Happened & Next Steps

Ad News Live 

July 19 2024



CrowdStrike: A recent CrowdStrike update is causing Windows computers to crash, showing the dreaded Blue Screen of Death. This issue has left companies worldwide, including Sky News, unable to reboot their systems.


Frustrated users are turning to Reddit to share their woes, with one person saying, "Wow, stuck in a boot loop, and the entire org taken out."

   

CrowdStrike outage explanation and next steps guide.


So if you went to work this morning and encountered literal carnage, know that you're not alone. Here’s what went down and what you should do next.


What happened

As you may have guessed, an issue with CrowdStrike is causing a widespread global problem. CrowdStrike engineers are working hard to resolve the problem that's affecting their Falcon sensor product. CrowdStrike describes Falcon as "the CrowdStrike platform purpose-built to prevent breaches through an integrated set of cloud-delivered technologies that prevent all types of attacks – including malware and more."


READ ALSO 

According to the Sky News website, the IT outage has affected airports, businesses and broadcasters. Planes have been grounded in the U.S., trains have been affected in the U.K., as well as boarding scanners at Edinburgh Airport in Scotland.


Microsoft says it is taking "mitigation actions" following service problems, which it said began at about 6 a.m. Eastern time. The company says it is investigating problems with cloud services in the U.S. Sky News points out that this is an issue affecting various apps and services.


I've contacted CrowdStrike and Microsoft for comment and will update this article when the firms respond.


While initial reports focused on a suspicious update, a user named Brody, who is the director of CrowdStrike Overwatch, posted on X, a Twitter first, that it's "a faulty channel file, so it's not an update."


He suggested a workaround: 


1. Reboot your computer and choose Safe Mode or the Windows Recovery Environment.

2. Navigate to C:\Windows\System32\drivers\CrowdStrike.

3. Find and remove the file that matches 'C-00000291*.sys'.

4. Restart Windows normally.


What To Do

It's not easy to say what to do next, because while there is a solution, it's not scalable because it has to be applied manually, system by system. In a large company, this could mean hours or longer to get back up and running.


READ ALSO 

Adam Harrison, managing director at FTI Cybersecurity, says it would be very difficult to fix the problem once the system is in a reboot loop. "Fixing this will take some time for system admins because CrowdStrike can’t just send a remote update to fix it. It would require manual intervention on each system."


Harrison explains that while some might have the chance to revert to previous stable states, the majority will lack the resources or support to do so. "The fix itself works pretty quickly, but when you scale it up to thousands of servers and/or thousands of workstations, it's going to be a bad day at the office for a lot of people."


It's going to be a bad day for CrowdStrike, too. What can the firm do to help people?


"They'll be communicating that fix as quickly and as widely as they possibly can," Harrison says. "I believe the update has already been applied, so any systems that haven’t been updated for some reason are likely still running the problematic version." – Ian Thornton-Trump, CISO of Cyjax, says CrowdStrike "will certainly do their best to roll back the update and instruct older agents not to update until they fix it."


READ ALSO 

However, he says, "What's been done to the blue screen machines can't be undone. If the machines can be booted into safe mode they may be able to release out of band updates or patches. This is time consuming - if the machines are critical, they may actually consider restoring from backups or shadow copies (MSFT's built in recovery facility).


Whatever avenues they have, they'll try to fix as quickly as possible." Harrison says CrowdStrike could produce a tool that would apply the fix at the disk level, such as bootable media. "This could be really useful for anyone dealing with thousands of systems to fix. It's still not a solution that completely solves the problem remotely or at scale, but it could reduce recovery times."




Follow Us

AD News Live

Post a Comment

0 Comments