news

Two global blue screens, the culprit is the same person? 14 years later, Thanos CEO caused another IT disaster

2024-07-22

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina


New Intelligence Report

Editor: Aeneas is so sleepy

【New Wisdom Introduction】It's amazing. Foreign media just discovered that the CEO of CrowdStrike, who caused the Microsoft blue screen disaster this time, had crashed devices around the world in the Windows XP era. It was also an update, and the devices were disconnected from the Internet, and they also had to be repaired manually. He caused global IT disasters twice, and this guy can "go down in history."

Microsoft's global blue screen incident has been solved!


A system logic error triggered by the "C-00000291*.sys" configuration file instantly destroyed about 1 billion computers around the world, and subsequently triggered all the second-order and third-order effects.

As AI master Karpathy said, any single point instantaneous failure that still exists in the field of technology will pose a huge hidden danger to human society.


The instigator of this global TI disaster, CrowdStrike CEO, was found by foreign media to have a criminal record.

It was him who crashed all the devices in the world with an update at McAfee in 2010!


Logical error triggers global collapse

As soon as the failure occurred, some netizens warned everyone - Stop all CrowdStrike updates! Stop all CrowdStrike updates!


Patrick Wardle, founder of the Objective-See Foundation, also conducted a detailed investigation into the cause of the incident as soon as possible.

First, he looked at the fault location - mov r9d, [r8], where R8 is an unmapped address.

This location is taken from the pointer array (stored in RAX) and the index RDX (0x14 * 0x8) stores an invalid memory address.



Other "drivers" (such as "C-00000291-...32.sys") appear to be obfuscated data and are x-ref'd by "CSAgent.sys".

Therefore, it may be this invalid (configuration/signature) data that triggers the failure in CSAgent.sys.


This can be determined more easily through debugging.

Obviously, the most important unanswered question in this incident is, what exactly is this "C-00000291-...xxx.sys" file?

Once CSAgent.sys references them, it crashes immediately; and deleting them can fix the crash.

On VT, he also reverse-engineered CSAgent.sys as well as data from a single crash dump.



Finally, Wardle shared several versions of CSAgent.sys (+idb), as well as various "C-....sys" files (including the latest one that he believes already contains the "fix").

He said that since he does not have any Windows system or virtual machine, he hopes that netizens can continue to dig.

Just yesterday, malware expert Malware Utkonos discovered more details:

At the address 37c78ba2eac468941a80f4e12aa390a00cb22337fbf87a94c59cee05473d1c66, there seems to be a file magic check for 0xaaaaaaaa.

This mode is also the first four bytes of the "Channel Files". A file with all NULLs may cause the cmp to fail.


As you can see, the value in rcx that is compared with 0xaaaaaaaa is allocated at the top by ExAllocatePoolWithTagPriority, which is the buffer that receives the data read by ZwReadFile.

This value is then passed to the function using cmp (Utkonos named these functions in the diagram as internal wdm.h function calls).


A sanity check shows that the 0xaaaaaaaa byte pattern only appears once at offset 0 of the "channel file" checked here.


The following is the address that executes something like cmp.


As you can see, only 0xaaaaaaaa looks different.


CrowdStrike official explanation

Soon, CrowdStrike released an explanation on its official blog, clarifying the questions that netizens were confused about:

On July 19, 2024, at 04:09 UTC, CrowdStrike released a sensor configuration update to Windows systems as part of the Falcon platform protection mechanism as part of ongoing operations. This configuration update triggered a logic error that caused a crash and blue screen (BSOD) on the affected systems. The update that caused the system crash was fixed on July 19, 2024, at 05:27 UTC.


Report address: https://www.crowdstrike.com/blog/falcon-update-for-windows-hosts-technical-details/

The technical details are as follows:

In Windows systems, channel files are located in the following directory: C:WindowsSystem32driversCrowdStrike, and the file name begins with "C-". Each channel file has a unique number as an identifier.

The affected channel file in this incident is 291, the file name starts with "C-00000291-" and ends with the .sys extension. Although channel files end with the SYS extension, they are not kernel drivers.

Channel file 291 affects how Falcon evaluates named pipe execution on Windows systems. These named pipes are the mechanism used for normal inter-process or inter-system communication in Windows.

Friday's update, which was intended to target a newly discovered malicious named pipe used in a common C2 framework in cyberattacks, actually triggered a logical error in the system, causing the crash.

This has nothing to do with the null byte problem in channel file 291 or any other channel file, though.


This incident has been made into a song by netizens using Suno

To recover, you have to boot the machine in safe mode, log in as a local administrator and delete the contents - something that cannot be automated.

Therefore, the impact of this paralysis is so great and difficult to recover from.

It was him last time.

Although CrowdStrike acknowledged its mistake and issued an apology and solution on Friday.

But they have yet to explain how this destructive update was released without testing and other safety measures.

Naturally, many critical voices began to focus on the central figure of the incident: CrowdStrike CEO George Kurtz.


Technology industry analyst Anshel Sag pointed out that this is not the first time Kurtz has played a major role in a major IT event.



Familiar recipe, familiar taste

On April 21, 2010, antivirus software McAfee released a software update for corporate customers.

The updated software deleted a critical Windows system file, causing millions of computers around the world to crash and restart repeatedly.

Similar to the CrowdStrike error, the McAfee issue also requires manual repair (device offline).

Kurtz was the chief technology officer of McAfee at the time.

Kurtz founded CrowdStrike in 2012 and remains its CEO today.



What happened in 2010?

At 6 a.m. on April 21, 2010, McAfee released a "problematic" virus definition update to its corporate customers.

Then, these automatically updated Windows XP computers will fall directly into an "infinite reboot" loop until technical support personnel arrive and manually repair it.

The reason behind this is actually very simple - after receiving the new definition, the antivirus software will identify a regular Windows binary file "svchost.exe" as a virus "W32/Wecorl.a" and destroy it.


One university IT worker reported that 1,200 computers on his network were down.

Another email from a US business said they had "hundreds of users" affected:

This problem affects a large number of users, and simply replacing svchost.exe does not solve the problem. You must boot into safe mode, install the extra.dat file, and then run the vsca console manually. After that, you also need to delete the quarantined files. Each user has at least two files quarantined, and some users have as many as 15. Unfortunately, using this method, you cannot be sure which of the files you recover are important system files and which are virus files.

There was also a report from Australia that 10% of the cash registers at the country's largest supermarket chain were paralyzed, forcing 14 to 18 stores to close.

The impact of this incident at the time was so great that many people were amazed: "Even hackers who focus on developing viruses probably can't create malware that can quickly 'take down' so many machines like McAfee did today."


Here is how the SANS Internet Storm Center described the incident:

McAfee version 5958 of the "DAT" file is causing problems with a large number of Windows XP SP3. Affected systems will enter a reboot loop and lose all network connectivity. This problematic DAT file can infect single workstations as well as workstations connected to a domain. Using "ePolicy Orchestrator" to update virus definition files appears to have accelerated the spread of this problematic DAT file. ePolicy Orchestrator is commonly used to update "DAT" files in enterprises, but because the affected systems will lose network connectivity, it cannot revoke this problematic signature.

Svchost.exe is one of the most important files in Windows system, it carries the services of almost all system functions. Without Svchost.exe, Windows cannot start at all.

Although the two incidents occurred 14 years apart, they raise the same question: how did such an update flow from the test lab and enter the production server? In theory, such problems should have been discovered and resolved in the early stages of testing.

Who is it?

George Kurtz grew up in Parsippany-Troy Hills, New Jersey, and attended Parsippany High School.

Kurtz said he started programming video games on a Commodore computer in fourth grade, and in high school he built an early online communication platform, a bulletin board system.

He graduated from Seton Hall University with a degree in accounting.


He subsequently founded Foundstone and served as Chief Technology Officer of McAfee.

Currently, George Kurtz serves as CEO of CrowdStrike, a cybersecurity company he co-founded with Dmitri Alperovitch.

In addition to his business achievements, he is also a racing driver.


Price Waterhouse and Foundstone

After graduating from college, Kurtz began his career at Price Waterhouse as a Certified Public Accountant (CPA).

In 1993, Price Waterhouse made Kurtz one of the first hires in its newly formed security group.


In 1999, he co-authored Hacking Exposed, a network security book for network administrators, with Stuart McClure and Joel Scambray. The book has sold over 600,000 copies and has been translated into more than 30 languages.


Later that year, he founded Foundstone, a cybersecurity company that was one of the first dedicated security consulting firms. Foundstone focused on vulnerability management software and services and developed a well-recognized incident response business, with many Fortune 100 companies as customers.


McAfee

McAfee acquired Foundstone for $86 million in August 2004, and Kurtz became McAfee's senior vice president and general manager of risk management. During his tenure, he helped develop the company's security risk management strategy.

In October 2009, McAfee appointed him Global Chief Technology Officer and Executive Vice President.


Over time, Kurtz became frustrated with the slowness of existing security technologies, which he believed were not keeping up with the pace of new threats.

He was inspired to found CrowdStrike after he saw a passenger sitting next to him on a plane wait 15 minutes for McAfee software to load on his laptop.


CrowdStrike

In November 2011, Kurtz joined private equity firm Warburg Pincus as an entrepreneur-in-residence and began working on his next project, CrowdStrike.

In February 2012, he teamed up with former Foundstone CFO Gregg Marston and Dmitri Alperovitch to formally form CrowdStrike.


CrowdStrike shifted its focus from anti-malware and antivirus products, McAfee's approach to cybersecurity, to identifying techniques used by hackers in order to spot incoming threats. It also developed a "cloud-first" model to reduce the software burden on customers' computers.

In May 2017, CrowdStrike was valued at over $1 billion. In 2019, the company went public on the Nasdaq for $612 million, with a valuation of $6.6 billion.


In July 2020, an IDC report ranked CrowdStrike as the fastest growing endpoint security software vendor.

In 2024, Kurtz will still be president and CEO of CrowdStrike.

Sure enough, the world is just a huge makeshift team.

References:

https://x.com/MalwareUtkonos/status/1814777806145847310

https://www.businessinsider.com/crowdstrike-ceo-george-kurtz-tech-outage-microsoft-mcafee-2024-7

https://www.crowdstrike.com/blog/falcon-update-for-windows-hosts-technical-details/

https://www.zdnet.com/article/defective-mcafee-update-causes-worldwide-meltdown-of-xp-pcs/