[LINK] The Great IT Outage of 2024

[Scott Howard]

The second comment is simply wrong.  Crowdstrike have a staged release for
"Sensor" content, but not for "Rapid Response" content.  The update that
caused this issue was "Rapid Response" content, so doesn't use the staging
mentioned.  That side of things functioned exactly as advertised.  (I'm not
saying they shouldn't have a staged rollout for this content, just that
they don't).

Crowdstrikes Preliminary Post Incident Review
<https://www.crowdstrike.com/falcon-content-update-remediation-and-guidance-hub/>
released several days above covers this fact.

  Scott


On Sat, Jul 27, 2024 at 12:42 PM Bernard Robertson-Dunn <brd at iimetro.com.au>
wrote:

> Dave explains the Crowdstrike IT outage, focusing in on its role as a
> kernel mode driver.
>
> https://www.youtube.com/watch?v=wAzEJxOo1ts
>
> Here is are two comments
>
> First Comment
>
> "The company I work at got bought by a bigger one. They required us to
> install Crowdstrike on all servers. We found a memory leak, that
> Crowdstrike still hasn't fixed after 6 months so I have refused to
> install it until then. I was on vacation when I saw all URGENT emails
> from other divisions.
>
> Thank you Crowdstrike for not fixing your memory leaks, it saved my
> vacation. =P"
>
> Second Comment
>
> "While this is technically what crashed machines it isn't the worst part.
>
> CS Falcon has a way to control the staging of updates across your
> environment. businesses who don't want to go out of business have a N-1
> or greater staging policy and only test systems get the latest updates
> immediately. My work for example has a test group at N staging, a small
> group of noncritical systems at N-1, and the rest of our computers at N-2.
>
> This broken update IGNORED our staging policies and went to ALL machine
> at the same time. CS informed us after our business was brought down
> that this is by design and some updates bypass policies.
>
> So in the end, CS caused untold millions of dollars in damages not just
> because they pushed a bad update, but because they pushed an update that
> ignored their customers' staging policies which would have prevented
> this type of widespread damage. Unbelievable."
>
>
> --
>
> Regards
> brd
>
> Bernard Robertson-Dunn
> Canberra Australia
> email: brd at iimetro.com.au
>
> _______________________________________________
> Link mailing list
> Link at anu.edu.au
> https://mailman.anu.edu.au/mailman/listinfo/link
>