News Windows 3.1 saves the day during CrowdStrike outage — Southwest Airlines scrapes by with archaic OS

[Ralston18]

No idea really....

I.e. "pencil-pen story"

I was on a NASA Space Station related project (scheduling - PERTS/GANTTS) at the time (long before the Space Station ever launched) and simply remember that story/rumor making the rounds.

@ NASA HQ in DC, Goddard, Cape Canveral/Kennedy, Houston (I was there when the Challenger exploded).

The story seemed to come from politicians demanding expensive module construction side trips to their represented states to do some very minor work at even more great expense.

Along with threats to vote against future Space Station funding.... Wreaked havoc on budgets and schedules.

I do not remember anyone refuting pencil/pen budget issues but that was well above my paygrade....

I do remember getting into a bit of trouble on that contract. I naively exceeded contract requirements - an apparent "no-no" as it was viewed by some.

I can live with "isn't true".

That story/those memories (?) are becoming the least of my worries.

Thanks.

 

[USAFRet]

No one used regular graphite pencils in spacecraft. Graphite flakes and dust floating around in zero g was not a good idea.

They all used regular pens or grease pencils. We used those on the flightline.

The Fisher Pen Co designed and tested the pressurized cartridge pen on their own dime.
Then sold them to NASA at cost. Flown on every mission starting with Apollo 7 (1968).

And eventually the Russians and Chinese.
https://www.spacepen.com/our-story

But yes...political pork is a thing...

 

[derekullo]

https://www.vogons.org/viewtopic.php?t=87617

Rust and Window 3.51

Now no more crashes

 

[Kondamin]

So what about those that just didn’t use CrowdStrike instead of ancient windows or is it just bashing on Microsoft

 

[jeremyj_83]

Reminds me of a long ago story about the US spending millions on a ink pen that could write in space sans gravity.

The other space faring country at the time used pencils.

Really do not know despite caring.

Will leave research etc. to others willing to pursue the matter.

Or some cold, rainy, nothing better to do day when the thought manifests in my old tired mind.....

The reason for the zero g ink pen is due to not wanting graphite shards floating in the air. Those little shards of graphite could get into the electronics and short them out. Hence an ink pen that can write in zero g is the best solution. Sure it sounds like finding a solution to something that doesn't have a problem but graphite in a space vehicle is potentially dangerous.

 

[jeremyj_83]

If all these incompetent companies where relying on Linux systems rather than the piece of trash code that is Windows, we would not even know about the issue because it would not have happened.

As someone who administers both Windows and Linux I can tell you that Windows wasn't the problem. Windows Server is rock solid. The issue could also have happened on Linux, which has its own quirks.

 

[jeremyj_83]

Not really, for two reasons.

First, switching from Win16 to anything else is decidedly non-trivial in an enterprise context.
Second and more importantly, from what I know this is only for their scheduling system.

So the entire premise of "changing an enterprise" is a shameful non-sequitor; perhaps even a strawman. Changing just their scheduling system should be fairly straightforward, at least, much easier than switching the entire enterprise good grief.

Upgrading from Windows 3.1/95 to Server 2016 is a MAJOR change. The software won't run on the modern OS even in compatibility mode. It would be as major a change as switching to Linux.

 

[Giroro]

There is no such thing as security by obscurity

 

[bit_user]

https://www.vogons.org/viewtopic.php?t=87617

Rust and Window 3.51

Now no more crashes

I remember doing some Windows programming in the early/mid-2000's and occasionally reading in the WinAPI documentation about features that were available only in newer versions of Windows. It makes me wonder just how many hacks and how compromised software would be, when you back-port it so far. Unless you can modify the kernel, many of those features probably can't be retrofitted.

And let's not forget that such old versions Windows didn't necessarily have the best scheduler or things like proper NUMA support. Their network stack must've been much more basic, etc. Not to mention 32-bit and limiting you to only 2 GB (I think later upped to 3 GB?) of userspace memory.

 

[Bobnla]

Story is not true

Redditer did digging and found a Dallas Morning news story that basically said that their software looks so old that it looks like it was created using windows 3.1. Not that Southwest is using, 3.1, just that the interface looks old.
No tables turned heading on SWAirlines sub

I expect more research from Tom's. Especially in something as fantastically outrageous as this. Did you even ask Southwest? Could it be they use Linux for most of their stuff? Be better.

 

[matt2367]

Not really, for two reasons.

First, switching from Win16 to anything else is decidedly non-trivial in an enterprise context.
Second and more importantly, from what I know this is only for their scheduling system.

So the entire premise of "changing an enterprise" is a shameful non-sequitor; perhaps even a strawman. Changing just their scheduling system should be fairly straightforward, at least, much easier than switching the entire enterprise good grief.

That's because it was.

https://twitter.com/x/status/1814367821943713960

View: https://twitter.com/ArtemR/status/1814367821943713960


All the author of this article had to do is scroll down to see the poster admit it. Southwest runs old software but it doesn't literally run on Windows 3.1.

 

[bit_user]

They have no idea what they are talking about. Linux would have also crashed if a kernel mode driver didn't handle an exception.

Well, it seems I need to amend what I said about that. I forgot about eBPF, which is a "virtual machine" feature of the Linux kernel, that allows filters and a subset of drivers to run in a sandboxed environment.

https://www.oligo.security/blog/recent-crowdstrike-outage-emphasizes-the-need-for-ebpf-based-sensors
​

I don't know for sure that their Linux solution only uses eBPF, as I think there are still some limitations on what you with it, but I've seen some indications that they are.

https://www.reddit.com/r/crowdstrike/comments/1ag8y0m/comment/kofcw96/
​

That's not to say you can't unintentionally take a machine offline with a buggy or misconfigured eBPF service. However, at least it won't kernel panic.

 

[NinjaNerd56]

Using windows 3.1 and windows 95 for the systems.

Taking mental note never fly with these guys ever.

I have been in their IT shop in Dallas. They are delusional folks, and I quit flying with them as well.

Of course, it may have had more to do with their aircrew not setting the flaps correctly for takeoff and stalling a 737. Yeah, that happened.

 

[JeffreyP55]

Southwest Airlines was able to keep operating even when all other airlines were crippled by the CrowdStrike update because their systems use Windows 3.1, a 32-year-old operating system that no longer receives updates.

Windows 3.1 saves the day during CrowdStrike outage — Southwest Airlines scrapes by with archaic OS : Read more

Maybe some are still using 80 column keypunch cards. I actually built, trouble shot, repaired 150lbs keypunch machines back in the day. That was just a few years ago!No PC's or MAC's yet...

 

[afoggy]

If all these incompetent companies where relying on Linux systems rather than the piece of trash code that is Windows, we would not even know about the issue because it would not have happened.

Actually not many months ago, an update to Crowdstrike has lead to kernel panics in Linux.

 

[ezst036]

All the author of this article had to do is scroll down to see the poster admit it. Southwest runs old software but it doesn't literally run on Windows 3.1.

Thank you. It's common for new information to appear.

So once again, people are infighting because of journalists.

Perhaps this time because of pure accident or laziness(not fully reading) but very often millions of people end up arguing and fighting amongst each other because of missing information. One side seeking more information, the other side furious about what scant little is currently available and digging in on that scant little.

 

[bit_user]

Actually not many months ago, an update to Crowdstrike has lead to kernel panics in Linux.

Do you know when? It would be interesting to see if this is before/after their adoption of eBPF. Also, older kernels wouldn't have all the eBPF features needed, meaning they might be limited to using natively-compiled modules, for those versions. So, we should also like to have some details about which kernel versions were affected.

 

[binba]

I find it funny they get credit for not being hit by an outage because they don’t run newer systems. It’s because they don’t run Crowdstrike.

This.
This whole story making the round seems patently dumb, making some senseless assumption that every airline in the US uses Crowdstrike for no reason. The claims about SW using Win3.1/95 I find hard to believe too... but at least provide a source. What I found is one source using it as a metaphor -- that their scheduling ERP software, SkySolver, dates back to the days of Windows 3.1.

It totally seems like the case of news outlets picking up stories from each other and running with it.

 

[CmdrShepard]

Do you know when? It would be interesting to see if this is before/after their adoption of eBPF. Also, older kernels wouldn't have all the eBPF features needed, meaning they might be limited to using natively-compiled modules, for those versions. So, we should also like to have some details about which kernel versions were affected.

What does it matter? A crash is a crash.

If they don't have proper exception handling in Windows kernel mode driver they most certainly don't have it in Linux either. It all smells like a bunch of junior C++ devs working on kernel stuff for the first time without knowing how the underlying hardware works. Wouldn't be the first time.

 

[mikeebb]

If all these incompetent companies where relying on Linux systems rather than the piece of trash code that is Windows, we would not even know about the issue because it would not have happened.

Crowdstrike did something similar to Linux systems a few months ago, but nobody noticed: https://www.techspot.com/news/103899-crowdstrike-also-broke-debian-rocky-linux-earlier-year.html

Maybe Crowdstrike is a problem?

Edit: ninja'd of course.

 

[bit_user]

What does it matter? A crash is a crash.

It matters to the extent that it's predictive. If we can establish that it was on kernels that CrowdStrike only supported via natively-compiled modules, then it would suggest that eBPF is indeed a sound path forward.

If they don't have proper exception handling in Windows kernel mode driver they most certainly don't have it in Linux either.

It sounds like you're unfamiliar with eBPF technology.

https://en.wikipedia.org/wiki/EBPF
​

 

[mitch074]

I remember doing some Windows programming in the early/mid-2000's and occasionally reading in the WinAPI documentation about features that were available only in newer versions of Windows. It makes me wonder just how many hacks and how compromised software would be, when you back-port it so far. Unless you can modify the kernel, many of those features probably can't be retrofitted.

And let's not forget that such old versions Windows didn't necessarily have the best scheduler or things like proper NUMA support. Their network stack must've been much more basic, etc. Not to mention 32-bit and limiting you to only 2 GB (I think later upped to 3 GB?) of userspace memory.

Forget about that, mere UTF-8 support was lacking - "international"? Whoopsies !

 

[eye4bear]

It's not like those are what run the airplanes themselves lol

Is Southwest still flying Boeing 737s? If so, enough said...

 

[CmdrShepard]

It matters to the extent that it's predictive. If we can establish that it was on kernels that CrowdStrike only supported via natively-compiled modules, then it would suggest that eBPF is indeed a sound path forward.

I don't think eBPF is such a panacea as you describe it.

From your wiki link I see that it's static code analysis and it only checks whether pointers are checked before dererferencing and whether loops are guaranteed to have an exit condition -- if it passes then executable doesn't have any further restrictions and can still cause havoc because static code analysis most certainly is bound to have some unhandled corner case hiding somewhere.

Also, it doesn't seem to be of much use for stuff not related to networking and EDR solution also needs access to pipes, IOCTLs of other drivers, and file handles.

There's even a version for Windows if you are interested on GitHub. You could rewrite say Peer Guardian in it, but that's about it.

 

[cjbreisch]

Of course, this article is completely wrong in its entire premise.

Yes, these ancient versions of Windows aren't getting updates, but since it wasn't a Windows Update that caused the problem, that's immaterial.

It was a Crowdstrike update that caused the problem. My guess is that Crowdstrike has not created a version of their app for these archaic versions of Windows (why would they?), so there was nothing to update.