Question Frequent BSODs are back

[kurapan]

Hi!
I came here asking for help with random BSODs in March and after that I was free from any BSODs for about a month. However, now they're back.

I get various error codes, with KERNEL_SECURITY_FAILURE being probably the most frequent one. But I also get others like APC_INDEX_MISMATCH, DRIVER_IRQL_NOT_LESS_OR_EQUAL, SYSTEM_SERVICE_EXCEPTION, PAGE_FAULT_IN_NONPAGED_AREA, DPC_WATCHDOG_VIOLATION, etc.

The IMAGE_NAME showed when analyzing a minidump in windbg is usually ntkrnlmp.exe, but it also was npfs.sys a few times.
It probably isn't really telling much but it has never crashed when I was playing a game. It's always been either when the system is idle or not under heavy load (like watching a video on youtube, browsing on the internet, etc.).

SMART says Good and health percentage is 98~100% for all of the three drives in my PC.

I've done a virus and malware scan. There wasn't any issue detected.

I did a memtest when the BSOD issues first started and it passed. But I plan to let a test run again when I go to work tomorrow or on Thursday.

Here are the most recent minidumps: https://drive.google.com/file/d/1AvBH8KEWMOnjC6poA1ti3PPuT1KmOzs6/view?usp=sharing

Here are my PC specs:
CPU: Ryzen 5 3500
GPU: GTX 1660 SUPER (Gigabyte)
MB: ASRock B450M Steel Legend
PSU: Corsair RM550x
RAM: G.Skill Flare X 2x8GB DDR4-3200
SSD1: WD Blue SN550 1TB NVMe
SSD2: WD Blue SATA SSD M.2
SSD3: Samsung SSD 850 EVO 250GB
OS: Windows 10 Education, version 2004

I built the PC in September 2020. All parts were new except for the Samsung SSD which I previously used for about 2 weeks and then it lay dormant for about 2 years. I haven't done any mining on the PC. The PC isn't usually under heavy load for a prolonged time.

I'm aware I don't have the latest bios but I didn't wanna update it unless there doesn't seem to be any other possible solution because afaik it's not recommended to update bios as long as you're not experiencing any issues.

Here's also a screenshot from the event viewer.

Thanks in advance for any help or advice.

 

[Grobe]

I came here asking for help with random BSODs in March and after that I was free from any BSODs for about a month. However, now they're back.

I suggest you put a link to that particular thread.

SMART says Good and health percentage is 98~100% for all of the three drives in my PC.

Please upload all s.m.a.r.t. data along with name of program used.

I've done a virus and malware scan. There wasn't any issue detected.

What software and version ?

I did a memtest when the BSOD issues first started and it passed. But I plan to let a test run again when I go to work tomorrow or on Thursday.

Remember to let it run for enough time, overnight if possible.

I'm aware I don't have the latest bios but

I suggest you take the time and read about what issues the newer bios versions solves, for all you know it may be that one thing that make your system unstable.

 

[kurapan]

Thanks for the reply.

I suggest you put a link to that particular thread.

Here's the past thread from March.

[SOLVED] - Various random BSODs

Hi! I've been getting a lot of different BSODs for the past few days. Several a day. The system always reboots and works fine until it crashes again some time later. It happens at random times, even when I'm not doing anything on the PC. I've been having a few BSODs per month for the past few...

forums.tomshardware.com

Please upload all s.m.a.r.t. data along with name of program used.

Screenshots from Crystal Disk Info

https://imgur.com/PYOKZQu

View: https://i.imgur.com/PYOKZQu.png

https://imgur.com/EQe9aF7

View: https://i.imgur.com/EQe9aF7.png

https://imgur.com/rZ6AeYo

View: https://i.imgur.com/rZ6AeYo.png

What software and version ?

MalwareBytes 4.4.11.149
Windows Defender

Remember to let it run for enough time, overnight if possible.

I did 8 passes when I ran it last time. I plan to let it run when I go to work in the morning.

I suggest you take the time and read about what issues the newer bios versions solves, for all you know it may be that one thing that make your system unstable.

It seems to be mostly support of newer CPUs and Windows 11 according to the download page of the manufacturer. I wonder if I can find some more detailed information anywhere.

 

[Grobe]

In this post:
https://forums.tomshardware.com/threads/various-random-bsods.3751461/post-22632259
You suggest three storage devices, while provided s.m.a.r.t. data for two ssd's. What's about the third one?

Have you tried to re-install Windows anytime lately ?

 

[kurapan]

In this post:
https://forums.tomshardware.com/threads/various-random-bsods.3751461/post-22632259
You suggest three storage devices, while provided s.m.a.r.t. data for two ssd's. What's about the third one?

Have you tried to re-install Windows anytime lately ?

I put screenshots from Crystal Disk for C: (WD Blue NVMe), D: (Samsung 850 EVO), and E: (WD Blue SATA SSD) drive in my previous post.

I haven't done any OS reinstallation lately.

 

[Grobe]

Can you say something about how often - this times - you expect the error to occur? Like, once per hour, once per day? Are all errors the same? Is there a common task (or program) you're running when the error occurs ?

 

[kurapan]

The frequency is random. I had several (about 2 to 4) BSODs per day at the beginning of March. Then I didn't get any BSOD for about a month until mid April. In the past days, I've been having several BSODs per day. Sometimes they occur every 10~20 minutes (usually 2 to 3 in succession), and then it can run again without any BSOD for several hours.

Error codes are different - I listed a few of them in the first post - but the service(?) stated in Windbg when analyzing a minidump usually says ntkrnlmp.exe. It was also npfs.sys a few times.
There isn't any common task except for all tasks running in the background from startup. It can crash even when the PC is idle and when there are no programs running except for tasks that start automatically at startup.

 

[Grobe]

From what you write about the randomness of the bsod's I'll lean toward hardware malfunction.

However - that is probably also how far it's possible to go before starting to replacing one by one component until the bsod's doesn't occurs any more. But if you expect having periods on months in good periods, then there is no way to tell if any component replaced really was a faulty one.

However - it isn't hopeless - yet. Here is some measurements I think you can and should take before spending money on new hardware:

• Do that memtest run overnight, see if the computer stay running that whole time.
• Use the OCCT benchmark software and make it run. Make sure you get the graphs for voltage and also temperatures. Sometimes if the PSU is really bad, one can tell from those graphs.
• Re-install Windows. Try to install as few apps and drivers as possible afterwards and see how it behaves over time. Remember to backup important/personal files first.

 

[johnbl]

Hi!
I came here asking for help with random BSODs in March and after that I was free from any BSODs for about a month. However, now they're back.

I get various error codes, with KERNEL_SECURITY_FAILURE being probably the most frequent one. But I also get others like APC_INDEX_MISMATCH, DRIVER_IRQL_NOT_LESS_OR_EQUAL, SYSTEM_SERVICE_EXCEPTION, PAGE_FAULT_IN_NONPAGED_AREA, DPC_WATCHDOG_VIOLATION, etc.

The IMAGE_NAME showed when analyzing a minidump in windbg is usually ntkrnlmp.exe, but it also was npfs.sys a few times.
It probably isn't really telling much but it has never crashed when I was playing a game. It's always been either when the system is idle or not under heavy load (like watching a video on youtube, browsing on the internet, etc.).

SMART says Good and health percentage is 98~100% for all of the three drives in my PC.

I've done a virus and malware scan. There wasn't any issue detected.

I did a memtest when the BSOD issues first started and it passed. But I plan to let a test run again when I go to work tomorrow or on Thursday.

Here are the most recent minidumps: https://drive.google.com/file/d/1AvBH8KEWMOnjC6poA1ti3PPuT1KmOzs6/view?usp=sharing

Here are my PC specs:
CPU: Ryzen 5 3500
GPU: GTX 1660 SUPER (Gigabyte)
MB: ASRock B450M Steel Legend
PSU: Corsair RM550x
RAM: G.Skill Flare X 2x8GB DDR4-3200
SSD1: WD Blue SN550 1TB NVMe
SSD2: WD Blue SATA SSD M.2
SSD3: Samsung SSD 850 EVO 250GB
OS: Windows 10 Education, version 2004

I built the PC in September 2020. All parts were new except for the Samsung SSD which I previously used for about 2 weeks and then it lay dormant for about 2 years. I haven't done any mining on the PC. The PC isn't usually under heavy load for a prolonged time.

I'm aware I don't have the latest bios but I didn't wanna update it unless there doesn't seem to be any other possible solution because afaik it's not recommended to update bios as long as you're not experiencing any issues.

Here's also a screenshot from the event viewer.

Thanks in advance for any help or advice.

you have a driver that is corrupting kernel memory. most of the time it looks like a timer table is getting corrupted. one time it looked like it was coming thru a filter driver and the named pipe file system. Maybe it is your google cloud driver. you do have a old driver from 2009 installed (partition wizard)
I would remove it.

you might just set up verifier.exe and force a bugcheck to name the bad driver.
you have many windows files where the checksum and time out has been removed. Maybe because it is an education build? I skip debugging these systems since i can not tell if the system is malware hacked using the debugger.

so, old partition wizard, google cloud driver (my best guess as cause)
or your vpn software. if you run verifier tests on these drivers it should bugcheck on the corruption rather than later when the corrupted timer expires.

 

[kurapan]

you have a driver that is corrupting kernel memory. most of the time it looks like a timer table is getting corrupted. one time it looked like it was coming thru a filter driver and the named pipe file system. Maybe it is your google cloud driver. you do have a old driver from 2009 installed (partition wizard)
I would remove it.

you might just set up verifier.exe and force a bugcheck to name the bad driver.
you have many windows files where the checksum and time out has been removed. Maybe because it is an education build? I skip debugging these systems since i can not tell if the system is malware hacked using the debugger.

so, old partition wizard, google cloud driver (my best guess as cause)
or your vpn software. if you run verifier tests on these drivers it should bugcheck on the corruption rather than later when the corrupted timer expires.

Thanks for the advice. I got rid of the partition wizard and vpn and left google drive installed for now. I think I'll be able to see if it had any effect in about a day or two (it was less frequent again the past days and I had about two days without a bsod).

Is there any guide how to run driver verifier in the safest way possible? I used it last year when my problems started upon an advice, but my PC kept crashing in a loop on windows startup and I couldn't even start it in safe mode, but I somehow managed to turn off verifier through command line after at least the windows advanced repair options finally loaded after a lot of repetitive crashes.

 

[kurapan]

From what you write about the randomness of the bsod's I'll lean toward hardware malfunction.

However - that is probably also how far it's possible to go before starting to replacing one by one component until the bsod's doesn't occurs any more. But if you expect having periods on months in good periods, then there is no way to tell if any component replaced really was a faulty one.

However - it isn't hopeless - yet. Here is some measurements I think you can and should take before spending money on new hardware:

• Do that memtest run overnight, see if the computer stay running that whole time.
• Use the OCCT benchmark software and make it run. Make sure you get the graphs for voltage and also temperatures. Sometimes if the PSU is really bad, one can tell from those graphs.
• Re-install Windows. Try to install as few apps and drivers as possible afterwards and see how it behaves over time. Remember to backup important/personal files first.

I didn't know about OCCT. I'll try it out. Though, I hope that the PSU isn't the cause of the issue because that's probably the only part in my PC that I didn't cheap out on when I built it.
Thanks!

 

[johnbl]

Thanks for the advice. I got rid of the partition wizard and vpn and left google drive installed for now. I think I'll be able to see if it had any effect in about a day or two (it was less frequent again the past days and I had about two days without a bsod).

Is there any guide how to run driver verifier in the safest way possible? I used it last year when my problems started upon an advice, but my PC kept crashing in a loop on windows startup and I couldn't even start it in safe mode, but I somehow managed to turn off verifier through command line after at least the windows advanced repair options finally loaded after a lot of repetitive crashes.

when you run verifier you need to make sure you know how to get into safe mode and know how to turn it off verifier.exe /reset some people have to turn off the fast start up or go thru 3 or 4 boot crashes before windows gives them the option of a safe mode startup. verifier will call a bugceck on the first driver bug it sees. So you have to fix the driver problem then retest to see if another driver has a issue. (then repeat until you don't get any more bugchecks) when you are done testing you have to issue the reset command or your system will run slowly until you do. Often when people can not get into safe mode they panic and reinstall windows before windows automatically goes into safe mode after the 3 failed bugcheck during boot. You might want to disable fast startup before you run verifier if you are worried about it.

 

[kurapan]

I'm back!

It was fine again for about 10 days and then BSODs started again.

I ran memtest for 8 passes again. No error.

Then I ran driver verifier and it crashed when it started booting. It also revealed the cause of the BSOD. It's RzDev_007a.sys, Razer drivers.
I have uninstalled Razer Synapse, the software that also installs Razer drivers. The drivers itself still seemed to be in the system though when I checked the list of installed drivers, and it seems to be the case because I got another BSOD.
After that I uninstalled the mouse from Device Manager, but I can still see RzDev_007a.sys listed among installed drivers.
I'm not really sure what's the safest way of removing those drivers. Is it safe to use brute force and simply delete them from the drivers directory in system32? Or is there any other better way?

It might not be necessary because driver verifier seems to have already revealed the cause of the BSODs, but I've uploaded the minidump created when I ran driver verifier.
https://drive.google.com/file/d/1kOgzNU9XKGivtbb1VPGp73xSj1OQhfCL/view?usp=sharing

 

[SkyNetRising]

Turn off driver verifier. It is crashing, because you're using it.
Driver verifier is meant for developers, who're developing drivers (not for everyday use).

 

[kurapan]

I did turn off driver verifier. I wouldn't even be able to boot into Windows without using safe mode without turning it off.

 

[SkyNetRising]

And you're still getting driver verifier BSODs after that?
Run system restore and restore to time before you enabled driver verifier.

 

[kurapan]

I'm not getting driver verifier BSODs. I'm getting BSODs with random error codes that I've been getting at random intervals in the past weeks.
I just used driver verifier to try to detect the cause of the BSODs because a faulty driver was the suspected cause. See johnbl's comment above https://forums.tomshardware.com/threads/frequent-bsods-are-back.3760446/post-22677252

 

[johnbl]

bugcheck was caused by
RzDev_007a.sys file date aug 17 2020

driver allocated driver memory that code could run in.
it should have allocated driver memory for data only
(should allocate NonPagedPoolNx)
This is needed to lock down locations where malware can run.


you would want to remove this driver or see if you can update it.
otherwise you will have to exclude this driver from being tested by verifier.exe

ie
verifier.exe /all /driver.exclude rzDev_007a.sys
this will allow you to continue testing other drivers for problems and will prevent a bugcheck on this driver.

https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/verifier

you can download and run autoruns from microsoft to disable or remove the driver: https://docs.microsoft.com/en-us/sysinternals/downloads/autoruns

just find the menu item to hide microsoft entries, then look at the list to find the driver. you can uncheck the driver name to make the system skip loading of the driver at boot. checking the driver will allow it to load on the next boot. Deleting the entry will remove it until you reinstall the software.

again, using the exclude option allows the driver to load but not be checked by verifier. you would do this if you really need the driver and want to check other drivers for errors. verifier bugcheck on the first error it finds and it can be picky. razer drivers do a lot of unrecommended stuff and tend to get flagged a lot. Often these checks start out as recommendations then later become requirements for windows . even the cpu can now do this type of checking and prevent this type of activity. IE running data as code.

note; this recommendation for driver data to be in non execute memory has been in place since windows 8.
https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/no-execute-nonpaged-pool

 

[kurapan]

I tried to remove it by uninstalling it from Device Manager, but I can still see it in System32/Drivers.
I also tried to reinstall it (uninstalled from Device Manager and uninstalled Razer Synapse, and then installed Razer Synapse again) and now I'm waiting if it will crash again. However, the driver dated August 2020 seems to be the latest one. It says created and modified August 24, 2020 when I open Properties, though. Not August 17. I couldn't find any list of drivers on Razer's website, but they have a firmware download on their drivers download page for the mouse dated August 25, 2020. Maybe it includes the driver too. However, the latest firmware version, the one I have installed, is from September 2021.

I didn't know that you can exclude a certain driver from a driver verifier test. I'll try that if it crashes again. But first I'll try to uncheck the driver from autoruns.

Thanks a lot for the advice. I'll post an update when I have some new information.

 

[johnbl]

I tried to remove it by uninstalling it from Device Manager, but I can still see it in System32/Drivers.
I also tried to reinstall it (uninstalled from Device Manager and uninstalled Razer Synapse, and then installed Razer Synapse again) and now I'm waiting if it will crash again. However, the driver dated August 2020 seems to be the latest one. It says created and modified August 24, 2020 when I open Properties, though. Not August 17. I couldn't find any list of drivers on Razer's website, but they have a firmware download on their drivers download page for the mouse dated August 25, 2020. Maybe it includes the driver too. However, the latest firmware version, the one I have installed, is from September 2021.

I didn't know that you can exclude a certain driver from a driver verifier test. I'll try that if it crashes again. But first I'll try to uncheck the driver from autoruns.

Thanks a lot for the advice. I'll post an update when I have some new information.

the debugger shows the internal file date, sometimes it does not match the file date on the file. date of compile of the object verses the day the compiled objects were linked into the final file. (boils down it is the same file)

 

[kurapan]

I managed to get completely rid of Razer drivers, but I got another BSOD after that.
I set up driver verifier again and this time Windows booted normally.
But now I got another BSOD, which however wasn't triggered by driver verifier. Btw I followed the instructions here and selected all drivers that are not provided by Microsoft.
I've uploaded dump files from the two BSODs that just happened, but I guess they won't provide much new info as they aren't triggered by driver verifier. https://drive.google.com/file/d/10e7-SAzhkdmSgB212fUoIoN_smDAyLak/view?usp=sharing

Is it safe to assume now that the issue isn't caused by drivers? Or that if it's drivers, it's drivers provided in Windows and I should reinstall Windows?
What would you recommend as my next course of action? Reinstall windows? Update bios? Or are there any more tests I should do?

 

[johnbl]

problem is some driver
---------
first bug check
APC_INDEX_MISMATCH (1)
common driver bug, with mismatched calls to a certain routine.

maybe get rid of this driver:
pwdrvio.sys Mon Jun 15 18:43:45 2009
(just based on the file date the file is suspect, no other reason to suspect this driver)

debugger would not let me read the bios info. Is this bios up to date?

-------
the second bugcheck was a watchdog timeout.
you were running some drive test utility, and got a watchdog timeout.
The timer was for a network function which would lead to suspect a network adaptor.
Most likely this driver just needs to be updated:
rt640x64.sys Tue Aug 28 01:09:35 2018 realtek network driver

you can look at your motherboard vendors website or google for the realtek website.
(came up in Chinese for me this time)
This driver could have caused both bugchecks but did cause the last one.
update driver and retest.

-(did not check for other network drivers)
-could not look at log info since this was a minidump
this link might work for you:
one is for a usb device the other for motherboard network device:
files updated on 2022/05/04
Downloads - REALTEK

 

[kurapan]

I got rid of pwrdvio.sys. I think you've actually mentioned it before so I uninstalled the application responsible for the driver (MiniTool Partition Wizard) but I didn't notice that the driver had stayed active in the system.

It's not the latest bios version. Should I get the latest one just in case? The latest ones seemed to focus on support for Windows 11 and newer AMD cpus (5000 series) according to the info on the maker's website.

I'll try to update the network adapter driver. Thanks for the link.

Should I activate kernel memory or automatic memory dump to get more information in the dump files?
Btw I also deactivated driver verifier after those two BSODs whose dumps I posted. Should I keep it activated for further debugging?

 

[johnbl]

I got rid of pwrdvio.sys. I think you've actually mentioned it before so I uninstalled the application responsible for the driver (MiniTool Partition Wizard) but I didn't notice that the driver had stayed active in the system.

It's not the latest bios version. Should I get the latest one just in case? The latest ones seemed to focus on support for Windows 11 and newer AMD cpus (5000 series) according to the info on the maker's website.

I'll try to update the network adapter driver. Thanks for the link.

Should I activate kernel memory or automatic memory dump to get more information in the dump files?
Btw I also deactivated driver verifier after those two BSODs whose dumps I posted. Should I keep it activated for further debugging?

if you provide kernel dumps then there is a lot less guessing involved in reading the dump. more logs are included in kernel dumps also allows more of the debugger functions to work.

you can keep the verifier on, it should not slow the system down too much and it will pick up common errors in drivers and bugcheck on the error.

you should update the bios, often the vendors do not even mention many of the fixes they do. with the debugger not being able to read teh bios info from the memory dump indicates that there has been some major bios spec change in some tables in the bios. My debugger is updated so It will not read the tables for a old version of standard tables in the bios.

it is pretty common for uninstallers not to remove the drivers.

 

[kurapan]

Okay! I've changed the settings to kernel memory dumps.

I've installed the latest Realtek LAN driver, but I got another BSOD.
I'm going to update the bios now.