CrowdStrike IT meltdown a nightmare scenario that may have been inevitable

American cybersecurity company CrowdStrike has become a household name overnight.

But that’s not because of its decade spent securing and protecting a bulk of the world’s computers – rather because of a problematic update that caused many of those same computers to crash on Friday afternoon in what is being described as an unprecedented IT fail.

Checkouts at some Coles stores remained offline on Saturday morning after the CrowdStrike outage.Credit: Cameron Atfield

Many have understandably jumped to the conclusion that some kind of incompetence at CrowdStrike is responsible, or that someone at the company acted outside the normal checks and balances. Such a conclusion may turn out to be warranted, but it may also miss a crucial fact about cybersecurity; pushing out updates to extraordinarily tight timelines is not just common but necessary, and this global meltdown may be awful but inevitable.

We might consider that every Windows enterprise computer in the world – from banks to airports to corporations – going down for multiple hours represents a worst-case scenario. And, obviously, it’s very bad.

But there are worse scenarios: like a nation state or well-resourced criminal syndicate infiltrating every Windows enterprise computer in the world. Those are the kinds of threats that can often only be counteracted with immediate updates and fixes.

It’s not exactly clear so far whether that’s the kind of update that went wrong here, but it’s at least worth considering that CrowdStrike rolls out updates without days of extensive testing for a reason, rather than due to incompetence.

New threats to the computers we rely on emerge every hour daily, and bad actors are far more motivated to take advantage of them than end users are to protect themselves. That’s why your work computer likely has some kind of real-time protection running in the background, and your IT people likely set it, so your computer has to be rebooted occasionally to implement important updates.

Updates vary in their urgency, but they all go through some kind of testing and have the potential to cause unexpected issues.

What happened on Friday is almost certainly the result of a perfect storm: an update too urgent to wait for further testing, or an issue that couldn’t be perceived within the limitations of the testing environment, caused a problem that couldn’t be addressed by the usual mitigations. Some bad updates can be fixed with another update or a rollback, but this one snowballed.

CrowdStrike chief executive George Kurtz has been in damage control since the issue became apparent, but in time, governments and companies globally will and should expect a full explanation of what happened and how it can be avoided in the future.

The global IT outage caused chaos at Sydney and other airports around the world.Credit: Louise Kennerley

But we may end up having to reckon with the fact that these issues are unavoidable as long as cybercrime exists and millions of machines rely on the same few vendors for their software and protection.

Tony Anscombe, chief security evangelist at global security company ESET, said CrowdStrike shouldn’t necessarily be criticised for the problem. But the incident does highlight some issues in need of addressing.

“We should not lose sight of who is to blame when an incident such as this happens. If cybercriminals and nation state attackers did not create cyberthreats, then we would not need protection in real-time,” he said.

“Cybersecurity vendors are likely to be reviewing their update processes to ensure there are no gaps and to see how they can strengthen them.

“For me, the real learning comes that when a company reaches a significant market position, their dominance can cause a semi-monoculture event, one issue will then affect many.”

Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.