Question High nvlddmkm.sys DPC count and execution ms

[Ethonodon]

Hi, I've been having this issue for the past few months which comes on and off. More often than not, it's off. I just installed a new set of ram (2x16gb corsair vengeance rgb pro) and now it's back again. What I notice in task manager is that whenever I try to do ANY simple task (scrolling on discord, scrolling on steam, watching a youtube video), my pc stutters a ton as my gpu usage skyrockets. Cpu, ram, everything else seems to generally stay pretty low. When this happens while I'm playing a game, ram usage usually stays high as well. I just ran latencymon while watching a couple seconds of a youtube video to try and get some answers out of that, and here were the results:

_
CONCLUSION
_
Your system appears to be having trouble handling real-time audio and other tasks. You are likely to experience buffer underruns appearing as drop outs, clicks or pops. One or more DPC routines that belong to a driver running in your system appear to be executing for too long. One problem may be related to power management, disable CPU throttling settings in Control Panel and BIOS setup. Check for BIOS updates.
LatencyMon has been analyzing your system for 0:00:15 (h🇲🇲ss) on all processors.


_
SYSTEM INFORMATION
_
Computer name: DESKTOP-CUU43JP
OS version: Windows 10, 10.0, version 2009, build: 19045 (x64)
Hardware: System Product Name, System manufacturer
BIOS: 3802
CPU: AuthenticAMD AMD Ryzen 7 3700X 8-Core Processor
Logical processors: 16
Processor groups: 1
Processor group size: 16
RAM: 32680 MB total


_
CPU SPEED
_
Reported CPU speed (WMI): 360 MHz
Reported CPU speed (registry): 360 MHz

Note: reported execution times may be calculated based on a fixed reported CPU speed. Disable variable speed settings like Intel Speed Step and AMD Cool N Quiet in the BIOS setup for more accurate results.


_
MEASURED INTERRUPT TO USER PROCESS LATENCIES
_
The interrupt to process latency reflects the measured interval that a usermode process needed to respond to a hardware request from the moment the interrupt service routine started execution. This includes the scheduling and execution of a DPC routine, the signaling of an event and the waking up of a usermode thread from an idle wait state in response to that event.

Highest measured interrupt to process latency (µs): 21001.90
Average measured interrupt to process latency (µs): 149.245925

Highest measured interrupt to DPC latency (µs): 20999.10
Average measured interrupt to DPC latency (µs): 144.449758


_
REPORTED ISRs
_
Interrupt service routines are routines installed by the OS and device drivers that execute in response to a hardware interrupt signal.

Highest ISR routine execution time (µs): 206.660
Driver with highest ISR routine execution time: Wdf01000.sys - Kernel Mode Driver Framework Runtime, Microsoft Corporation

Highest reported total ISR routine time (%): 0.000555
Driver with highest ISR total time: Wdf01000.sys - Kernel Mode Driver Framework Runtime, Microsoft Corporation

Total time spent in ISRs (%) 0.000696

ISR count (execution time <250 µs): 2659
ISR count (execution time 250-500 µs): 0
ISR count (execution time 500-1000 µs): 0
ISR count (execution time 1000-2000 µs): 0
ISR count (execution time 2000-4000 µs): 0
ISR count (execution time >=4000 µs): 0


_
REPORTED DPCs
_
DPC routines are part of the interrupt servicing dispatch mechanism and disable the possibility for a process to utilize the CPU while it is interrupted until the DPC has finished execution.

Highest DPC routine execution time (µs): 23076.020
Driver with highest DPC routine execution time: nvlddmkm.sys - NVIDIA Windows Kernel Mode Driver, Version 528.49 , NVIDIA Corporation

Highest reported total DPC routine time (%): 0.203498
Driver with highest DPC total execution time: nvlddmkm.sys - NVIDIA Windows Kernel Mode Driver, Version 528.49 , NVIDIA Corporation

Total time spent in DPCs (%) 0.340589

DPC count (execution time <250 µs): 19297
DPC count (execution time 250-500 µs): 0
DPC count (execution time 500-10000 µs): 21
DPC count (execution time 1000-2000 µs): 4
DPC count (execution time 2000-4000 µs): 151
DPC count (execution time >=4000 µs): 48


_
REPORTED HARD PAGEFAULTS
_
Hard pagefaults are events that get triggered by making use of virtual memory that is not resident in RAM but backed by a memory mapped file on disk. The process of resolving the hard pagefault requires reading in the memory from disk while the process is interrupted and blocked from execution.

NOTE: some processes were hit by hard pagefaults. If these were programs producing audio, they are likely to interrupt the audio stream resulting in dropouts, clicks and pops. Check the Processes tab to see which programs were hit.

Process with highest pagefault count: steam.exe

Total number of hard pagefaults 8
Hard pagefault count of hardest hit process: 3
Number of processes hit: 5


_
PER CPU DATA
_
CPU 0 Interrupt cycle time (s): 3.790012
CPU 0 ISR highest execution time (µs): 8.890
CPU 0 ISR total execution time (s): 0.000589
CPU 0 ISR count: 934
CPU 0 DPC highest execution time (µs): 23076.020
CPU 0 DPC total execution time (s): 0.806894
CPU 0 DPC count: 18047
_
CPU 1 Interrupt cycle time (s): 0.127153
CPU 1 ISR highest execution time (µs): 8.810
CPU 1 ISR total execution time (s): 0.000170
CPU 1 ISR count: 325
CPU 1 DPC highest execution time (µs): 34.520
CPU 1 DPC total execution time (s): 0.002932
CPU 1 DPC count: 344
_
CPU 2 Interrupt cycle time (s): 0.611673
CPU 2 ISR highest execution time (µs): 9.930
CPU 2 ISR total execution time (s): 0.000105
CPU 2 ISR count: 45
CPU 2 DPC highest execution time (µs): 98.060
CPU 2 DPC total execution time (s): 0.000869
CPU 2 DPC count: 168
_
CPU 3 Interrupt cycle time (s): 0.455404
CPU 3 ISR highest execution time (µs): 0.0
CPU 3 ISR total execution time (s): 0.0
CPU 3 ISR count: 0
CPU 3 DPC highest execution time (µs): 19.810
CPU 3 DPC total execution time (s): 0.000117
CPU 3 DPC count: 36
_
CPU 4 Interrupt cycle time (s): 0.627825
CPU 4 ISR highest execution time (µs): 0.0
CPU 4 ISR total execution time (s): 0.0
CPU 4 ISR count: 0
CPU 4 DPC highest execution time (µs): 46.440
CPU 4 DPC total execution time (s): 0.000078
CPU 4 DPC count: 4
_
CPU 5 Interrupt cycle time (s): 0.523052
CPU 5 ISR highest execution time (µs): 0.0
CPU 5 ISR total execution time (s): 0.0
CPU 5 ISR count: 0
CPU 5 DPC highest execution time (µs): 22.010
CPU 5 DPC total execution time (s): 0.000076
CPU 5 DPC count: 26
_
CPU 6 Interrupt cycle time (s): 0.572392
CPU 6 ISR highest execution time (µs): 0.0
CPU 6 ISR total execution time (s): 0.0
CPU 6 ISR count: 0
CPU 6 DPC highest execution time (µs): 69.830
CPU 6 DPC total execution time (s): 0.001594
CPU 6 DPC count: 95
_
CPU 7 Interrupt cycle time (s): 0.565038
CPU 7 ISR highest execution time (µs): 0.0
CPU 7 ISR total execution time (s): 0.0
CPU 7 ISR count: 0
CPU 7 DPC highest execution time (µs): 45.20
CPU 7 DPC total execution time (s): 0.000220
CPU 7 DPC count: 47
_
CPU 8 Interrupt cycle time (s): 0.530598
CPU 8 ISR highest execution time (µs): 0.0
CPU 8 ISR total execution time (s): 0.0
CPU 8 ISR count: 0
CPU 8 DPC highest execution time (µs): 0.80
CPU 8 DPC total execution time (s): 0.000002
CPU 8 DPC count: 2
_
CPU 9 Interrupt cycle time (s): 0.527440
CPU 9 ISR highest execution time (µs): 0.0
CPU 9 ISR total execution time (s): 0.0
CPU 9 ISR count: 0
CPU 9 DPC highest execution time (µs): 3.760
CPU 9 DPC total execution time (s): 0.000004
CPU 9 DPC count: 1
_
CPU 10 Interrupt cycle time (s): 0.616380
CPU 10 ISR highest execution time (µs): 0.0
CPU 10 ISR total execution time (s): 0.0
CPU 10 ISR count: 0
CPU 10 DPC highest execution time (µs): 25.420
CPU 10 DPC total execution time (s): 0.000202
CPU 10 DPC count: 41
_
CPU 11 Interrupt cycle time (s): 0.606182
CPU 11 ISR highest execution time (µs): 0.0
CPU 11 ISR total execution time (s): 0.0
CPU 11 ISR count: 0
CPU 11 DPC highest execution time (µs): 25.570
CPU 11 DPC total execution time (s): 0.000050
CPU 11 DPC count: 12
_
CPU 12 Interrupt cycle time (s): 0.780488
CPU 12 ISR highest execution time (µs): 206.660
CPU 12 ISR total execution time (s): 0.000737
CPU 12 ISR count: 1202
CPU 12 DPC highest execution time (µs): 730.880
CPU 12 DPC total execution time (s): 0.004502
CPU 12 DPC count: 421
_
CPU 13 Interrupt cycle time (s): 0.424921
CPU 13 ISR highest execution time (µs): 1.220
CPU 13 ISR total execution time (s): 0.000044
CPU 13 ISR count: 89
CPU 13 DPC highest execution time (µs): 23.590
CPU 13 DPC total execution time (s): 0.001399
CPU 13 DPC count: 238
_
CPU 14 Interrupt cycle time (s): 0.613787
CPU 14 ISR highest execution time (µs): 0.940
CPU 14 ISR total execution time (s): 0.000005
CPU 14 ISR count: 8
CPU 14 DPC highest execution time (µs): 34.810
CPU 14 DPC total execution time (s): 0.000116
CPU 14 DPC count: 21
_
CPU 15 Interrupt cycle time (s): 0.625090
CPU 15 ISR highest execution time (µs): 0.770
CPU 15 ISR total execution time (s): 0.000024
CPU 15 ISR count: 56
CPU 15 DPC highest execution time (µs): 26.480
CPU 15 DPC total execution time (s): 0.000101
CPU 15 DPC count: 18
_


I noticed specifically that under the drivers tab, "nvlddmkm.sys" and "dxgkrnl.sys" had especially high DPC counts and execution ms. Not entirely sure what it means, it's just what I notice.

PC Specs:
EVGA GeForce RTX 3070 Ti FTW3, Ryzen 7 3700X, Antec Earthwatts Gold Pro EA750G, Asus Prime B450-M, Hitachi HUA722020ALA331, Samsung 870 EVO 1TB, 32GB Corsair Vengeance RGB Pro, 4 case fans


Any help is appreciated!

 

[Ethonodon]

Bit of an update:

I've tried installing to the newest version of Nvidia drivers. I used safe mode, ddu, all of that to make sure I had a proper uninstallation of the old driver. The problem still persists.

I've also noticed that in task manager it lists many programs using GPU 0 - 3D- which I think is unusual because I don't remember it normally having more than one or two processes using that.

Here are some things I notice in HWINFO64:

Disclaimer: I am not super technologically literate, so these are wild guesses at what could be problematic.

-DRAM Read/Write Bandwidth is low(?) hovering from 0.060 Gbps - 0.650 Gbps. It says the "Maximum" was 1.436 Gbps which is when I assume my pc was running normally

-Cpu usage stays very low while the gpu usage skyrockets. Cpu usually stays under 20% while the gpu spikes to nearly 100% just trying to watch a youtube video

-Page file usage is at 0%. No idea what this is though

-Gpu memory controller usage stays low, even when under heavy load

-Gpu bus load stays around 0%, even when under load. Sometimes goes up to 1-2%. Not sure what this is normally supposed to be

 

[Ethonodon]

An additional update: I disabled HPET in device manager, which improved the situation a small amount, but not nearly enough.

I'm using the game Mordhau's main menu as a testing grounds for my performance, as I know roughly where it should be with my parts. Before disabling HPET, I'd get maybe 40-60 fps in the Mordhau main menu, which is absolutely terrible for my rig. After disabling it, I now get in the 60-80 range, which is still quite bad for my pc, but better nonetheless. It should be sitting around 120-144 fps, as that's where it sat with my old ram, and my new ram is both faster and twice the size.

It's also worth noting that after installing my new ram, I immediately ran Mordhau to test it, and the game ran beautifully. 120-144 fps the whole way, for roughly 10 minutes. Best performance I've ever seen out of this pc. Then I decided to try Star Citizen, where it ran in the single digits of fps. After closing Star Citizen, the problem has been present since.

Hope this additional information is helpful.

 

[johnbl]

try the following:
delete the system virtual memory, reboot and turn it back on to dump the pagefile.sys
run cmd.exe as an admin then run
dism.exe /online /cleanup-image /restorehealth

update the bios and chipset drivers from your motherboard vendors website.

google how to force a kernel memory dump using a keyboard, make the registry settings.

run the updated system for a while then force a kernel memory dump of the running system

put the kernel dump (memory.dmp) file on a cloud server, share it for public access and post a link.
I will take a look at the drivers and the internal logs and see if anything looks wrong in the debugger.

(bios and chipset update just because that is the most common problem and is a common fix)

note: if your motherboard has a realtek sound card, be sure to update the driver to a version made in 2020 or newer.

 

[Ethonodon]

try the following:
delete the system virtual memory, reboot and turn it back on to dump the pagefile.sys
run cmd.exe as an admin then run
dism.exe /online /cleanup-image /restorehealth

update the bios and chipset drivers from your motherboard vendors website.

google how to force a kernel memory dump using a keyboard, make the registry settings.

run the updated system for a while then force a kernel memory dump of the running system

put the kernel dump (memory.dmp) file on a cloud server, share it for public access and post a link.
I will take a look at the drivers and the internal logs and see if anything looks wrong in the debugger.

(bios and chipset update just because that is the most common problem and is a common fix)

note: if your motherboard has a realtek sound card, be sure to update the driver to a version made in 2020 or newer.

Here is a MEGA folder with the MEMORY.DMP file: https://mega.nz/folder/d1lSHIzK#6sxbMWhOFIUhZZV_W-ulSw

I very recently updated my BIOS to fix another issue, as well as my motherboard drivers too. I can try reinstalling my mobo drivers AGAIN, if it comes to it. If that's the case which ones should I do? Just the chipset? Or others as well?

I've reinstalled my realtek audio drivers to a newer version, from 2020. The problem still persists.

 

[johnbl]

Here is a MEGA folder with the MEMORY.DMP file: https://mega.nz/folder/d1lSHIzK#6sxbMWhOFIUhZZV_W-ulSw

I very recently updated my BIOS to fix another issue, as well as my motherboard drivers too. I can try reinstalling my mobo drivers AGAIN, if it comes to it. If that's the case which ones should I do? Just the chipset? Or others as well?

I've reinstalled my realtek audio drivers to a newer version, from 2020. The problem still persists.

failed install of this device:
InstancePath is "USBSTOR\Other&Ven_WD&Prod_SES_Device&Rev_1020\575853324541303331303441&1"
error code was:
Error code: (NTSTATUS) 0xc0000490 (3221226640) - There are no compatible drivers available for this device.

this device is disabled for some reason:
InstancePath is "ACPI\PNP0103\2&daba3ff&0" (battery controller?)

I would remove the device, then go into windows control panel, device manager, find the menu item to show hidden devices, check it then look for greyed out usb devices and remove them. (or hunt down the required software, I would skip the install if this is a old device)

InstancePath is "ACPI\PNP0A08\0"
This device has some problem, log indicates it is in a unknown state.
looks like it tried to start up but then just got confused

modified windows files:
SystemRoot\System32\win32k.sys * Invalid
SystemRoot\System32\win32kfull.sys * Invalid
see if you can update this driver:
\SystemRoot\System32\drivers\rt640x64.sys Tue May 26 01:25:13 2020
debugger says there is a memory leak.
maybe try the version from here:
https://www.realtek.com/en/componen...0-1000m-gigabit-ethernet-pci-express-software


lots of errors in the usb3 log.
FxDevice:😀eleteDeviceFromFailedCreateNoDelete - WDFDEVICE 00005AFE9A976FD8 !devobj 0000000000000000 created, but EvtDriverDeviceAdd returned status 0xc0000035(STATUS_OBJECT_NAME_COLLISION) or failure in creation


start cmd.exe as an admin then run
dism.exe /online /cleanup-image /restorehealth

turn off the system virtual memory, reboot and turn it back on. This should delete your pagefile.sys and dump the modified windows files.

give these a shot, I will try to look at the dump some more in the morning.

: kd> !sysinfo machineid
Machine ID Information [From Smbios 3.3, DMIVersion 0, Size=2716]
BiosMajorRelease = 5
BiosMinorRelease = 17
BiosVendor = American Megatrends Inc.
BiosVersion = 3802
BiosReleaseDate = 04/28/2022
SystemManufacturer = System manufacturer
SystemProductName = System Product Name
SystemFamily = To be filled by O.E.M.
SystemVersion = System Version
SystemSKU = SKU
BaseBoardManufacturer = ASUSTeK COMPUTER INC.
BaseBoardProduct = PRIME B450M-A
BaseBoardVersion = Rev X.0x
: kd> !sysinfo cpuinfo
[CPU Information]
~MHz = REG_DWORD 3600
Component Information = REG_BINARY 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
Configuration Data = REG_FULL_RESOURCE_DESCRIPTOR ff,ff,ff,ff,ff,ff,ff,ff,0,0,0,0,0,0,0,0
Identifier = REG_SZ AMD64 Family 23 Model 113 Stepping 0
ProcessorNameString = REG_SZ AMD Ryzen 7 3700X 8-Core Processor
Update Status = REG_DWORD 1
VendorIdentifier = REG_SZ AuthenticAMD

 

[johnbl]

you might consider disabling nvidia share if you are not actively using it. (just because of various issues with it in the past)
lots of errors in the usb extension logs.
USBXHCI

maybe you need custom usb drivers for your machine. looks like all the usb drivers are generic ones from microsoft.
(check your motherboard vendor to see if they have special usb drivers for your machine.)
you might go directly to amd for the drivers, you could see if you can get updated GPIO drivers as well as the chipset drivers.

You might also google how to have windows delete your pagefile.sys on system shutdown. make the changes and reboot.
it would reduce the impact of a driver with a memory leak, also helps when something is making a modification to the windows core files in the pagefile.sys.

you can start cmd.exe as an admin then disable plug and play via
net.exe stop "plug and play"

just in case it keeps trying to install a driver over and over.