Question LatencyMon: high end machine has trouble handling real-time audio ?

[3di]

I have a decent spec machine (see below report), but I'm getting bad audio performance. I'm using a focusrite 4i4 external audio interface, connected to a usb c 3.2 2x2 port.

Here's the report. I've made sure there's no cpu throttling in the control panel, but I'm not sure which setting to modify in Bios to disable throttling there, or how safe it is to do so. Any tips, or advice for alternative solutions based on the below latencymon report:
_________________________________________________________________________________________________________
CONCLUSION
_________________________________________________________________________________________________________
Your system appears to be having trouble handling real-time audio and other tasks. You are likely to experience buffer underruns appearing as drop outs, clicks or pops. One or more DPC routines that belong to a driver running in your system appear to be executing for too long. One problem may be related to power management, disable CPU throttling settings in Control Panel and BIOS setup. Check for BIOS updates.
LatencyMon has been analyzing your system for 0:12:15 (h:mm:ss) on all processors.


_________________________________________________________________________________________________________
SYSTEM INFORMATION
_________________________________________________________________________________________________________
Computer name: 3DIWORKSTATION
OS version: Windows 11, 10.0, version 2009, build: 26100 (x64)
Hardware: Z790 Nova WiFi, ASRock
BIOS: 7.01
CPU: GenuineIntel Intel(R) Core(TM) i7-14700K
Logical processors: 28
Processor groups: 1
Processor group size: 28
RAM: 98088 MB total


_________________________________________________________________________________________________________
CPU SPEED
_________________________________________________________________________________________________________
Reported CPU speed (WMI): 340 MHz
Reported CPU speed (registry): 3418 MHz

Note: reported execution times may be calculated based on a fixed reported CPU speed. Disable variable speed settings like Intel Speed Step and AMD Cool N Quiet in the BIOS setup for more accurate results.


_________________________________________________________________________________________________________
MEASURED INTERRUPT TO USER PROCESS LATENCIES
_________________________________________________________________________________________________________
The interrupt to process latency reflects the measured interval that a usermode process needed to respond to a hardware request from the moment the interrupt service routine started execution. This includes the scheduling and execution of a DPC routine, the signaling of an event and the waking up of a usermode thread from an idle wait state in response to that event.

Highest measured interrupt to process latency (µs): 4278.70
Average measured interrupt to process latency (µs): 8.498138

Highest measured interrupt to DPC latency (µs): 150.40
Average measured interrupt to DPC latency (µs): 1.864909


_________________________________________________________________________________________________________
REPORTED ISRs
_________________________________________________________________________________________________________
Interrupt service routines are routines installed by the OS and device drivers that execute in response to a hardware interrupt signal.

Highest ISR routine execution time (µs): 189.239906
Driver with highest ISR routine execution time: Wdf01000.sys - Kernel Mode Driver Framework Runtime, Microsoft Corporation

Highest reported total ISR routine time (%): 0.000576
Driver with highest ISR total time: Wdf01000.sys - Kernel Mode Driver Framework Runtime, Microsoft Corporation

Total time spent in ISRs (%) 0.000576

ISR count (execution time <250 µs): 67737
ISR count (execution time 250-500 µs): 0
ISR count (execution time 500-1000 µs): 0
ISR count (execution time 1000-2000 µs): 0
ISR count (execution time 2000-4000 µs): 0
ISR count (execution time >=4000 µs): 0


_________________________________________________________________________________________________________
REPORTED DPCs
_________________________________________________________________________________________________________
DPC routines are part of the interrupt servicing dispatch mechanism and disable the possibility for a process to utilize the CPU while it is interrupted until the DPC has finished execution.

Highest DPC routine execution time (µs): 1161.313341
Driver with highest DPC routine execution time: nvlddmkm.sys - NVIDIA Windows Kernel Mode Driver, Version 566.14 , NVIDIA Corporation

Highest reported total DPC routine time (%): 0.011788
Driver with highest DPC total execution time: ntoskrnl.exe - NT Kernel & System, Microsoft Corporation

Total time spent in DPCs (%) 0.036115

DPC count (execution time <250 µs): 760600
DPC count (execution time 250-500 µs): 0
DPC count (execution time 500-10000 µs): 95
DPC count (execution time 1000-2000 µs): 1
DPC count (execution time 2000-4000 µs): 0
DPC count (execution time >=4000 µs): 0


_________________________________________________________________________________________________________
REPORTED HARD PAGEFAULTS
_________________________________________________________________________________________________________
Hard pagefaults are events that get triggered by making use of virtual memory that is not resident in RAM but backed by a memory mapped file on disk. The process of resolving the hard pagefault requires reading in the memory from disk while the process is interrupted and blocked from execution.

NOTE: some processes were hit by hard pagefaults. If these were programs producing audio, they are likely to interrupt the audio stream resulting in dropouts, clicks and pops. Check the Processes tab to see which programs were hit.

Process with highest pagefault count: blender.exe

Total number of hard pagefaults 20704
Hard pagefault count of hardest hit process: 10414
Number of processes hit: 72

Spoiler

_________________________________________________________________________________________________________
PER CPU DATA
_________________________________________________________________________________________________________
CPU 0 Interrupt cycle time (s): 5.271126
CPU 0 ISR highest execution time (µs): 9.963136
CPU 0 ISR total execution time (s): 0.000035
CPU 0 ISR count: 5
CPU 0 DPC highest execution time (µs): 943.966355
CPU 0 DPC total execution time (s): 0.156593
CPU 0 DPC count: 21391
_________________________________________________________________________________________________________
CPU 1 Interrupt cycle time (s): 3.488419
CPU 1 ISR highest execution time (µs): 19.224108
CPU 1 ISR total execution time (s): 0.003395
CPU 1 ISR count: 669
CPU 1 DPC highest execution time (µs): 255.098011
CPU 1 DPC total execution time (s): 0.030942
CPU 1 DPC count: 2746
_________________________________________________________________________________________________________
CPU 2 Interrupt cycle time (s): 2.089683
CPU 2 ISR highest execution time (µs): 23.777940
CPU 2 ISR total execution time (s): 0.002619
CPU 2 ISR count: 947
CPU 2 DPC highest execution time (µs): 187.434465
CPU 2 DPC total execution time (s): 0.025847
CPU 2 DPC count: 12406
_________________________________________________________________________________________________________
CPU 3 Interrupt cycle time (s): 2.212069
CPU 3 ISR highest execution time (µs): 0.0
CPU 3 ISR total execution time (s): 0.0
CPU 3 ISR count: 0
CPU 3 DPC highest execution time (µs): 244.139848
CPU 3 DPC total execution time (s): 0.060706
CPU 3 DPC count: 10484
_________________________________________________________________________________________________________
CPU 4 Interrupt cycle time (s): 2.190855
CPU 4 ISR highest execution time (µs): 1.849912
CPU 4 ISR total execution time (s): 0.000003
CPU 4 ISR count: 4
CPU 4 DPC highest execution time (µs): 286.490345
CPU 4 DPC total execution time (s): 0.055928
CPU 4 DPC count: 15479
_________________________________________________________________________________________________________
CPU 5 Interrupt cycle time (s): 1.881944
CPU 5 ISR highest execution time (µs): 0.0
CPU 5 ISR total execution time (s): 0.0
CPU 5 ISR count: 0
CPU 5 DPC highest execution time (µs): 206.689292
CPU 5 DPC total execution time (s): 0.002323
CPU 5 DPC count: 1652
_________________________________________________________________________________________________________
CPU 6 Interrupt cycle time (s): 2.129189
CPU 6 ISR highest execution time (µs): 57.116442
CPU 6 ISR total execution time (s): 0.013778
CPU 6 ISR count: 6051
CPU 6 DPC highest execution time (µs): 818.928613
CPU 6 DPC total execution time (s): 0.045862
CPU 6 DPC count: 16554
_________________________________________________________________________________________________________
CPU 7 Interrupt cycle time (s): 2.273575
CPU 7 ISR highest execution time (µs): 0.0
CPU 7 ISR total execution time (s): 0.0
CPU 7 ISR count: 0
CPU 7 DPC highest execution time (µs): 403.029257
CPU 7 DPC total execution time (s): 0.064422
CPU 7 DPC count: 16272
_________________________________________________________________________________________________________
CPU 8 Interrupt cycle time (s): 11.653675
CPU 8 ISR highest execution time (µs): 38.125219
CPU 8 ISR total execution time (s): 0.000773
CPU 8 ISR count: 211
CPU 8 DPC highest execution time (µs): 1161.313341
CPU 8 DPC total execution time (s): 2.416085
CPU 8 DPC count: 299564
_________________________________________________________________________________________________________
CPU 9 Interrupt cycle time (s): 8.254375
CPU 9 ISR highest execution time (µs): 32.957870
CPU 9 ISR total execution time (s): 0.041230
CPU 9 ISR count: 27122
CPU 9 DPC highest execution time (µs): 262.135459
CPU 9 DPC total execution time (s): 1.630009
CPU 9 DPC count: 81210
_________________________________________________________________________________________________________
CPU 10 Interrupt cycle time (s): 9.775990
CPU 10 ISR highest execution time (µs): 189.239906
CPU 10 ISR total execution time (s): 0.051936
CPU 10 ISR count: 31218
CPU 10 DPC highest execution time (µs): 369.357812
CPU 10 DPC total execution time (s): 1.865596
CPU 10 DPC count: 136512
_________________________________________________________________________________________________________
CPU 11 Interrupt cycle time (s): 8.376887
CPU 11 ISR highest execution time (µs): 13.322411
CPU 11 ISR total execution time (s): 0.000088
CPU 11 ISR count: 15
CPU 11 DPC highest execution time (µs): 298.180807
CPU 11 DPC total execution time (s): 0.989804
CPU 11 DPC count: 112595
_________________________________________________________________________________________________________
CPU 12 Interrupt cycle time (s): 2.094831
CPU 12 ISR highest execution time (µs): 0.0
CPU 12 ISR total execution time (s): 0.0
CPU 12 ISR count: 0
CPU 12 DPC highest execution time (µs): 685.545348
CPU 12 DPC total execution time (s): 0.022667
CPU 12 DPC count: 10612
_________________________________________________________________________________________________________
CPU 13 Interrupt cycle time (s): 1.880763
CPU 13 ISR highest execution time (µs): 0.0
CPU 13 ISR total execution time (s): 0.0
CPU 13 ISR count: 0
CPU 13 DPC highest execution time (µs): 193.272674
CPU 13 DPC total execution time (s): 0.004618
CPU 13 DPC count: 1399
_________________________________________________________________________________________________________
CPU 14 Interrupt cycle time (s): 1.998405
CPU 14 ISR highest execution time (µs): 21.035108
CPU 14 ISR total execution time (s): 0.004710
CPU 14 ISR count: 1495
CPU 14 DPC highest execution time (µs): 195.306319
CPU 14 DPC total execution time (s): 0.024185
CPU 14 DPC count: 10530
_________________________________________________________________________________________________________
CPU 15 Interrupt cycle time (s): 1.821946
CPU 15 ISR highest execution time (µs): 0.0
CPU 15 ISR total execution time (s): 0.0
CPU 15 ISR count: 0
CPU 15 DPC highest execution time (µs): 191.909889
CPU 15 DPC total execution time (s): 0.003829
CPU 15 DPC count: 1635
_________________________________________________________________________________________________________
CPU 16 Interrupt cycle time (s): 0.963813
CPU 16 ISR highest execution time (µs): 0.0
CPU 16 ISR total execution time (s): 0.0
CPU 16 ISR count: 0
CPU 16 DPC highest execution time (µs): 146.40550
CPU 16 DPC total execution time (s): 0.005791
CPU 16 DPC count: 1632
_________________________________________________________________________________________________________
CPU 17 Interrupt cycle time (s): 0.778760
CPU 17 ISR highest execution time (µs): 0.0
CPU 17 ISR total execution time (s): 0.0
CPU 17 ISR count: 0
CPU 17 DPC highest execution time (µs): 37.058806
CPU 17 DPC total execution time (s): 0.000428
CPU 17 DPC count: 169
_________________________________________________________________________________________________________
CPU 18 Interrupt cycle time (s): 0.777064
CPU 18 ISR highest execution time (µs): 0.0
CPU 18 ISR total execution time (s): 0.0
CPU 18 ISR count: 0
CPU 18 DPC highest execution time (µs): 26.799590
CPU 18 DPC total execution time (s): 0.000403
CPU 18 DPC count: 147
_________________________________________________________________________________________________________
CPU 19 Interrupt cycle time (s): 0.768594
CPU 19 ISR highest execution time (µs): 0.0
CPU 19 ISR total execution time (s): 0.0
CPU 19 ISR count: 0
CPU 19 DPC highest execution time (µs): 74.975424
CPU 19 DPC total execution time (s): 0.0020
CPU 19 DPC count: 568
_________________________________________________________________________________________________________
CPU 20 Interrupt cycle time (s): 0.955488
CPU 20 ISR highest execution time (µs): 0.0
CPU 20 ISR total execution time (s): 0.0
CPU 20 ISR count: 0
CPU 20 DPC highest execution time (µs): 86.346987
CPU 20 DPC total execution time (s): 0.003434
CPU 20 DPC count: 985
_________________________________________________________________________________________________________
CPU 21 Interrupt cycle time (s): 0.836421
CPU 21 ISR highest execution time (µs): 0.0
CPU 21 ISR total execution time (s): 0.0
CPU 21 ISR count: 0
CPU 21 DPC highest execution time (µs): 39.157109
CPU 21 DPC total execution time (s): 0.000966
CPU 21 DPC count: 205
_________________________________________________________________________________________________________
CPU 22 Interrupt cycle time (s): 0.842959
CPU 22 ISR highest execution time (µs): 0.0
CPU 22 ISR total execution time (s): 0.0
CPU 22 ISR count: 0
CPU 22 DPC highest execution time (µs): 53.083382
CPU 22 DPC total execution time (s): 0.000833
CPU 22 DPC count: 176
_________________________________________________________________________________________________________
CPU 23 Interrupt cycle time (s): 0.853992
CPU 23 ISR highest execution time (µs): 0.0
CPU 23 ISR total execution time (s): 0.0
CPU 23 ISR count: 0
CPU 23 DPC highest execution time (µs): 173.508484
CPU 23 DPC total execution time (s): 0.009322
CPU 23 DPC count: 2423
_________________________________________________________________________________________________________
CPU 24 Interrupt cycle time (s): 0.926726
CPU 24 ISR highest execution time (µs): 0.0
CPU 24 ISR total execution time (s): 0.0
CPU 24 ISR count: 0
CPU 24 DPC highest execution time (µs): 184.015799
CPU 24 DPC total execution time (s): 0.010712
CPU 24 DPC count: 2246
_________________________________________________________________________________________________________
CPU 25 Interrupt cycle time (s): 0.797842
CPU 25 ISR highest execution time (µs): 0.0
CPU 25 ISR total execution time (s): 0.0
CPU 25 ISR count: 0
CPU 25 DPC highest execution time (µs): 68.939438
CPU 25 DPC total execution time (s): 0.003155
CPU 25 DPC count: 579
_________________________________________________________________________________________________________
CPU 26 Interrupt cycle time (s): 0.781722
CPU 26 ISR highest execution time (µs): 0.0
CPU 26 ISR total execution time (s): 0.0
CPU 26 ISR count: 0
CPU 26 DPC highest execution time (µs): 195.739614
CPU 26 DPC total execution time (s): 0.001694
CPU 26 DPC count: 427
_________________________________________________________________________________________________________
CPU 27 Interrupt cycle time (s): 0.713501
CPU 27 ISR highest execution time (µs): 0.0
CPU 27 ISR total execution time (s): 0.0
CPU 27 ISR count: 0
CPU 27 DPC highest execution time (µs): 40.810415
CPU 27 DPC total execution time (s): 0.000560
CPU 27 DPC count: 98
_________________________________________________________________________________________________________

 

[Colif]

Highest DPC routine execution time (µs): 1161.313341
Driver with highest DPC routine execution time: nvlddmkm.sys - NVIDIA Windows Kernel Mode Driver, Version 566.14 , NVIDIA Corporation

its normal for Nvidia to be the top.

blender might just hit hard drive more than other applications, I don't know if it being top Page faults is a clue.

can you show us the drivers page next time?

Often the biggest source of latency can be lan drivers, do you use Ethernet or Wifi? Disable which ever controller you don't use and/or update the drivers

https://pg.asrock.com/mb/Intel/Z790 Nova WiFi/index.asp#Download

 

[ubuysa]

This statement leapt out at me...

RAM: 98088 MB total

Can you please list exactly what RAM you have installed? Mixing different RAM makes and types is a well-known cause of problems, so I'd like to see what RAM you have installed first.

 

[Colif]

could be 4 x 24gb with some rounding up?
24.5 x 4 = 98
Pretty sure they don't sell 24.5gb sticks though?
I know there are 48gb but that is too big?
too many size choices now. Sure looks like a collection.

I assume that is total ram and not in use. Curious, latencymon won't run on my PC so I can't check to see if its total or used.

 

[ubuysa]

That's really why I asked, it looks suspiciously like mixed RAM to me.

 

[3di]

Thank for the responses. The problem happens with no software running. Am I correct in thinking that Wdf01000.sys is indicating that there is an issue communicating with a piece of hardware? If so, maybe a GPU driver issue?

Ram is 2x 48GB matched RAM, tried at 4400 and 6600 (xmp)

Full PC spec: https://pasteboard.co/VD4JS6qW0KFb.png


drivers: https://pasteboard.co/PI1MM2ujokzM.png


cpu page: https://pasteboard.co/2LTCySszvmYK.png


I've tried turning off XMP, and I've also turned off speed step.

 

[3di]

resource mon shows no cpu throttling, and the spikes don't seem to tally with any activity on disk, memory, or network activity. I did think possibly I saw a few spikes in hard faults/sec. So I tried setting my min page file to 16000, and max to 32000. But that hasn't helped.

 

[Colif]

Guess the program can't add up? Not sure where it would get the extra 2.088gb then?

It might be GPU but every time I seen latencymon report it shows Nvidia - mainly as how much its used

windows driver framework is also fairly common on reports.

hard page faults happen if the CPU needs to get data off the drive. limiting the page file isn't a solution, as it may not be page file. If program has to access data that isn't in ram, it will cause a hard page fault. They aren't actually problems, just badly named normal operations of windows.

did you try the lan driver idea?

 

[3di]

Thanks. I did try disabling wifi, lan, bluetooth, etc and uninstalled the audio interface: https://pasteboard.co/9FFzPzr1HuDO.png

But I still got a monstrous spike within a few mins: https://pasteboard.co/xYI3GL1mXGb9.png

processes: https://pasteboard.co/PM7c4rTzuS8d.png

drivers: https://pasteboard.co/XBe3VePK5FBQ.png

cpus: https://pasteboard.co/Q8ISf0V5Cw5Z.png


new report:

Spoiler

_________________________________________________________________________________________________________
CONCLUSION
_________________________________________________________________________________________________________
Your system appears to be having trouble handling real-time audio and other tasks. You are likely to experience buffer underruns appearing as drop outs, clicks or pops. One problem may be related to power management, disable CPU throttling settings in Control Panel and BIOS setup. Check for BIOS updates.
LatencyMon has been analyzing your system for 0:03:21 (h:mm:ss) on all processors.


_________________________________________________________________________________________________________
SYSTEM INFORMATION
_________________________________________________________________________________________________________
Computer name: 3DIWORKSTATION
OS version: Windows 11, 10.0, version 2009, build: 26100 (x64)
Hardware: Z790 Nova WiFi, ASRock
BIOS: 7.01
CPU: GenuineIntel Intel(R) Core(TM) i7-14700K
Logical processors: 28
Processor groups: 1
Processor group size: 28
RAM: 98088 MB total


_________________________________________________________________________________________________________
CPU SPEED
_________________________________________________________________________________________________________
Reported CPU speed (WMI): 340 MHz
Reported CPU speed (registry): 3418 MHz

Note: reported execution times may be calculated based on a fixed reported CPU speed. Disable variable speed settings like Intel Speed Step and AMD Cool N Quiet in the BIOS setup for more accurate results.


_________________________________________________________________________________________________________
MEASURED INTERRUPT TO USER PROCESS LATENCIES
_________________________________________________________________________________________________________
The interrupt to process latency reflects the measured interval that a usermode process needed to respond to a hardware request from the moment the interrupt service routine started execution. This includes the scheduling and execution of a DPC routine, the signaling of an event and the waking up of a usermode thread from an idle wait state in response to that event.

Highest measured interrupt to process latency (µs): 13998.80
Average measured interrupt to process latency (µs): 12.723189

Highest measured interrupt to DPC latency (µs): 113.20
Average measured interrupt to DPC latency (µs): 2.399403


_________________________________________________________________________________________________________
REPORTED ISRs
_________________________________________________________________________________________________________
Interrupt service routines are routines installed by the OS and device drivers that execute in response to a hardware interrupt signal.

Highest ISR routine execution time (µs): 27.509655
Driver with highest ISR routine execution time: Wdf01000.sys - Kernel Mode Driver Framework Runtime, Microsoft Corporation

Highest reported total ISR routine time (%): 0.000047
Driver with highest ISR total time: Wdf01000.sys - Kernel Mode Driver Framework Runtime, Microsoft Corporation

Total time spent in ISRs (%) 0.000047

ISR count (execution time <250 µs): 814
ISR count (execution time 250-500 µs): 0
ISR count (execution time 500-1000 µs): 0
ISR count (execution time 1000-2000 µs): 0
ISR count (execution time 2000-4000 µs): 0
ISR count (execution time >=4000 µs): 0


_________________________________________________________________________________________________________
REPORTED DPCs
_________________________________________________________________________________________________________
DPC routines are part of the interrupt servicing dispatch mechanism and disable the possibility for a process to utilize the CPU while it is interrupted until the DPC has finished execution.

Highest DPC routine execution time (µs): 309.662961
Driver with highest DPC routine execution time: ntoskrnl.exe - NT Kernel & System, Microsoft Corporation

Highest reported total DPC routine time (%): 0.007519
Driver with highest DPC total execution time: ntoskrnl.exe - NT Kernel & System, Microsoft Corporation

Total time spent in DPCs (%) 0.021746

DPC count (execution time <250 µs): 138492
DPC count (execution time 250-500 µs): 0
DPC count (execution time 500-10000 µs): 1
DPC count (execution time 1000-2000 µs): 0
DPC count (execution time 2000-4000 µs): 0
DPC count (execution time >=4000 µs): 0


_________________________________________________________________________________________________________
REPORTED HARD PAGEFAULTS
_________________________________________________________________________________________________________
Hard pagefaults are events that get triggered by making use of virtual memory that is not resident in RAM but backed by a memory mapped file on disk. The process of resolving the hard pagefault requires reading in the memory from disk while the process is interrupted and blocked from execution.

NOTE: some processes were hit by hard pagefaults. If these were programs producing audio, they are likely to interrupt the audio stream resulting in dropouts, clicks and pops. Check the Processes tab to see which programs were hit.

Process with highest pagefault count: msmpeng.exe

Total number of hard pagefaults 272
Hard pagefault count of hardest hit process: 247
Number of processes hit: 6


_________________________________________________________________________________________________________
PER CPU DATA
_________________________________________________________________________________________________________
CPU 0 Interrupt cycle time (s): 1.602792
CPU 0 ISR highest execution time (µs): 0.0
CPU 0 ISR total execution time (s): 0.0
CPU 0 ISR count: 0
CPU 0 DPC highest execution time (µs): 211.239614
CPU 0 DPC total execution time (s): 0.055384
CPU 0 DPC count: 7231
_________________________________________________________________________________________________________
CPU 1 Interrupt cycle time (s): 0.884261
CPU 1 ISR highest execution time (µs): 0.0
CPU 1 ISR total execution time (s): 0.0
CPU 1 ISR count: 0
CPU 1 DPC highest execution time (µs): 163.112932
CPU 1 DPC total execution time (s): 0.003115
CPU 1 DPC count: 548
_________________________________________________________________________________________________________
CPU 2 Interrupt cycle time (s): 1.183609
CPU 2 ISR highest execution time (µs): 13.045348
CPU 2 ISR total execution time (s): 0.000668
CPU 2 ISR count: 228
CPU 2 DPC highest execution time (µs): 194.326799
CPU 2 DPC total execution time (s): 0.023305
CPU 2 DPC count: 6572
_________________________________________________________________________________________________________
CPU 3 Interrupt cycle time (s): 0.911677
CPU 3 ISR highest execution time (µs): 0.0
CPU 3 ISR total execution time (s): 0.0
CPU 3 ISR count: 0
CPU 3 DPC highest execution time (µs): 203.474254
CPU 3 DPC total execution time (s): 0.007792
CPU 3 DPC count: 2618
_________________________________________________________________________________________________________
CPU 4 Interrupt cycle time (s): 1.040130
CPU 4 ISR highest execution time (µs): 0.0
CPU 4 ISR total execution time (s): 0.0
CPU 4 ISR count: 0
CPU 4 DPC highest execution time (µs): 220.256290
CPU 4 DPC total execution time (s): 0.016957
CPU 4 DPC count: 3750
_________________________________________________________________________________________________________
CPU 5 Interrupt cycle time (s): 0.663029
CPU 5 ISR highest execution time (µs): 0.0
CPU 5 ISR total execution time (s): 0.0
CPU 5 ISR count: 0
CPU 5 DPC highest execution time (µs): 0.0
CPU 5 DPC total execution time (s): 0.0
CPU 5 DPC count: 0
_________________________________________________________________________________________________________
CPU 6 Interrupt cycle time (s): 0.612986
CPU 6 ISR highest execution time (µs): 0.0
CPU 6 ISR total execution time (s): 0.0
CPU 6 ISR count: 0
CPU 6 DPC highest execution time (µs): 62.477180
CPU 6 DPC total execution time (s): 0.001881
CPU 6 DPC count: 649
_________________________________________________________________________________________________________
CPU 7 Interrupt cycle time (s): 0.523220
CPU 7 ISR highest execution time (µs): 0.0
CPU 7 ISR total execution time (s): 0.0
CPU 7 ISR count: 0
CPU 7 DPC highest execution time (µs): 0.0
CPU 7 DPC total execution time (s): 0.0
CPU 7 DPC count: 0
_________________________________________________________________________________________________________
CPU 8 Interrupt cycle time (s): 3.759475
CPU 8 ISR highest execution time (µs): 0.0
CPU 8 ISR total execution time (s): 0.0
CPU 8 ISR count: 0
CPU 8 DPC highest execution time (µs): 236.222937
CPU 8 DPC total execution time (s): 0.626013
CPU 8 DPC count: 73584
_________________________________________________________________________________________________________
CPU 9 Interrupt cycle time (s): 1.892073
CPU 9 ISR highest execution time (µs): 15.502341
CPU 9 ISR total execution time (s): 0.000132
CPU 9 ISR count: 22
CPU 9 DPC highest execution time (µs): 201.319193
CPU 9 DPC total execution time (s): 0.087777
CPU 9 DPC count: 6963
_________________________________________________________________________________________________________
CPU 10 Interrupt cycle time (s): 2.051476
CPU 10 ISR highest execution time (µs): 27.509655
CPU 10 ISR total execution time (s): 0.001822
CPU 10 ISR count: 564
CPU 10 DPC highest execution time (µs): 309.662961
CPU 10 DPC total execution time (s): 0.186929
CPU 10 DPC count: 20594
_________________________________________________________________________________________________________
CPU 11 Interrupt cycle time (s): 2.167798
CPU 11 ISR highest execution time (µs): 0.0
CPU 11 ISR total execution time (s): 0.0
CPU 11 ISR count: 0
CPU 11 DPC highest execution time (µs): 232.778233
CPU 11 DPC total execution time (s): 0.207653
CPU 11 DPC count: 14691
_________________________________________________________________________________________________________
CPU 12 Interrupt cycle time (s): 0.48310
CPU 12 ISR highest execution time (µs): 0.0
CPU 12 ISR total execution time (s): 0.0
CPU 12 ISR count: 0
CPU 12 DPC highest execution time (µs): 61.580749
CPU 12 DPC total execution time (s): 0.000722
CPU 12 DPC count: 186
_________________________________________________________________________________________________________
CPU 13 Interrupt cycle time (s): 0.463062
CPU 13 ISR highest execution time (µs): 0.0
CPU 13 ISR total execution time (s): 0.0
CPU 13 ISR count: 0
CPU 13 DPC highest execution time (µs): 186.848449
CPU 13 DPC total execution time (s): 0.000969
CPU 13 DPC count: 68
_________________________________________________________________________________________________________
CPU 14 Interrupt cycle time (s): 0.643727
CPU 14 ISR highest execution time (µs): 0.0
CPU 14 ISR total execution time (s): 0.0
CPU 14 ISR count: 0
CPU 14 DPC highest execution time (µs): 161.544178
CPU 14 DPC total execution time (s): 0.001890
CPU 14 DPC count: 393
_________________________________________________________________________________________________________
CPU 15 Interrupt cycle time (s): 0.627968
CPU 15 ISR highest execution time (µs): 0.0
CPU 15 ISR total execution time (s): 0.0
CPU 15 ISR count: 0
CPU 15 DPC highest execution time (µs): 52.655061
CPU 15 DPC total execution time (s): 0.001227
CPU 15 DPC count: 184
_________________________________________________________________________________________________________
CPU 16 Interrupt cycle time (s): 0.230187
CPU 16 ISR highest execution time (µs): 0.0
CPU 16 ISR total execution time (s): 0.0
CPU 16 ISR count: 0
CPU 16 DPC highest execution time (µs): 53.518139
CPU 16 DPC total execution time (s): 0.000192
CPU 16 DPC count: 31
_________________________________________________________________________________________________________
CPU 17 Interrupt cycle time (s): 0.215225
CPU 17 ISR highest execution time (µs): 0.0
CPU 17 ISR total execution time (s): 0.0
CPU 17 ISR count: 0
CPU 17 DPC highest execution time (µs): 11.242247
CPU 17 DPC total execution time (s): 0.000030
CPU 17 DPC count: 7
_________________________________________________________________________________________________________
CPU 18 Interrupt cycle time (s): 0.223251
CPU 18 ISR highest execution time (µs): 0.0
CPU 18 ISR total execution time (s): 0.0
CPU 18 ISR count: 0
CPU 18 DPC highest execution time (µs): 63.598011
CPU 18 DPC total execution time (s): 0.000716
CPU 18 DPC count: 106
_________________________________________________________________________________________________________
CPU 19 Interrupt cycle time (s): 0.218457
CPU 19 ISR highest execution time (µs): 0.0
CPU 19 ISR total execution time (s): 0.0
CPU 19 ISR count: 0
CPU 19 DPC highest execution time (µs): 24.287010
CPU 19 DPC total execution time (s): 0.000347
CPU 19 DPC count: 67
_________________________________________________________________________________________________________
CPU 20 Interrupt cycle time (s): 0.235098
CPU 20 ISR highest execution time (µs): 0.0
CPU 20 ISR total execution time (s): 0.0
CPU 20 ISR count: 0
CPU 20 DPC highest execution time (µs): 73.210942
CPU 20 DPC total execution time (s): 0.000969
CPU 20 DPC count: 138
_________________________________________________________________________________________________________
CPU 21 Interrupt cycle time (s): 0.195632
CPU 21 ISR highest execution time (µs): 0.0
CPU 21 ISR total execution time (s): 0.0
CPU 21 ISR count: 0
CPU 21 DPC highest execution time (µs): 6.906085
CPU 21 DPC total execution time (s): 0.000012
CPU 21 DPC count: 2
_________________________________________________________________________________________________________
CPU 22 Interrupt cycle time (s): 0.193654
CPU 22 ISR highest execution time (µs): 0.0
CPU 22 ISR total execution time (s): 0.0
CPU 22 ISR count: 0
CPU 22 DPC highest execution time (µs): 0.0
CPU 22 DPC total execution time (s): 0.0
CPU 22 DPC count: 0
_________________________________________________________________________________________________________
CPU 23 Interrupt cycle time (s): 0.178113
CPU 23 ISR highest execution time (µs): 0.0
CPU 23 ISR total execution time (s): 0.0
CPU 23 ISR count: 0
CPU 23 DPC highest execution time (µs): 0.0
CPU 23 DPC total execution time (s): 0.0
CPU 23 DPC count: 0
_________________________________________________________________________________________________________
CPU 24 Interrupt cycle time (s): 0.182338
CPU 24 ISR highest execution time (µs): 0.0
CPU 24 ISR total execution time (s): 0.0
CPU 24 ISR count: 0
CPU 24 DPC highest execution time (µs): 65.994441
CPU 24 DPC total execution time (s): 0.000769
CPU 24 DPC count: 69
_________________________________________________________________________________________________________
CPU 25 Interrupt cycle time (s): 0.170328
CPU 25 ISR highest execution time (µs): 0.0
CPU 25 ISR total execution time (s): 0.0
CPU 25 ISR count: 0
CPU 25 DPC highest execution time (µs): 13.061439
CPU 25 DPC total execution time (s): 0.000085
CPU 25 DPC count: 11
_________________________________________________________________________________________________________
CPU 26 Interrupt cycle time (s): 0.173813
CPU 26 ISR highest execution time (µs): 0.0
CPU 26 ISR total execution time (s): 0.0
CPU 26 ISR count: 0
CPU 26 DPC highest execution time (µs): 13.870685
CPU 26 DPC total execution time (s): 0.000029
CPU 26 DPC count: 8
_________________________________________________________________________________________________________
CPU 27 Interrupt cycle time (s): 0.173846
CPU 27 ISR highest execution time (µs): 0.0
CPU 27 ISR total execution time (s): 0.0
CPU 27 ISR count: 0
CPU 27 DPC highest execution time (µs): 47.436220
CPU 27 DPC total execution time (s): 0.000146
CPU 27 DPC count: 23
_________________________________________________________________________________________________________

 

[3di]

resource mon and hardware monitor at the moment of a spike: https://pasteboard.co/GwtSt4y82KOk.png

 

[ubuysa]

It would be wise to remove one RAM stick for a few days and then swap sticks for a few days. See whether the latency changes on just one stick? One thing I don't fully understand is your hard page fault rate - these are pages that were page-faulted in from the pagefile...

Total number of hard pagefaults                       20704
Hard pagefault count of hardest hit process:          10414    (blender.exe)
Number of processes hit:                              72

In a 12min 15sec run that works out at over 28 page faults per second. With 96GB of RAM installed why are you seeing such a high page fault rate? Even taking blender.exe out of the numbers you're still seeing 14 hard page faults per second there. That's curious. The second latencymon output has a much more reasonable hard page fault rate.

Both the longest ISR and the longest DPC in both runs is the Wdf01000.sys driver. This is the root driver for the Windows Driver Foundation (WDF). Any drivers coded using the Windows Driver Foundation libraries are managed by Wdf01000.sys. The driver actually causing this issue isn't seen, only Wdf01000.sys.

You might want to upload the full kernel dump, it's just possible we might be able to identify the driver in there. It's the file C:\Windows\Memory.dmp and it will be large. I can also do an analysis of the long-running DPCs from that dump.

 

[Colif]

Process with highest pagefault count: msmpeng.exe

I just thought I quote that and then thought of a reason. - I guess defender might make sense if it scanned drive.

I thought hard page faults just meant it hitting drive, how do we know its the page file and not some other files on drive?

I still don't get how its seeing him with 98gb of ram. (its not a known bug of program). Its ram... so its not working set. it didn't add on any page file.
Its consistent, 2 scans, same amount.

 

[3di]

Thanks @ubuysa . I don't have a Memory.dmp file, is there some other tool I can use to generate the data you need? I have one report in LiveKernelReports, but that was just my network adapter after turning on fast boot.

@Colif Ram is reported correctly in system information, so it must be a problem with latencymon

OS Name Microsoft Windows 11 Pro
Version 10.0.26100 Build 26100
Other OS Description Not Available
OS Manufacturer Microsoft Corporation
System Name 3DIWORKSTATION
System Manufacturer ASRock
System Model Z790 Nova WiFi
System Type x64-based PC
System SKU To Be Filled By O.E.M.
Processor Intel(R) Core(TM) i7-14700K, 3400 Mhz, 20 Core(s), 28 Logical Processor(s)
BIOS Version/Date American Megatrends International, LLC. 7.01, 27/09/2024
SMBIOS Version 3.5
Embedded Controller Version 255.255
BIOS Mode UEFI
BaseBoard Manufacturer ASRock
BaseBoard Product Z790 Nova WiFi
BaseBoard Version
Platform Role Desktop
Secure Boot State Off
PCR7 Configuration Elevation Required to View
Windows Directory C:\WINDOWS
System Directory C:\WINDOWS\system32
Boot Device \Device\HarddiskVolume5
Locale United Kingdom
Hardware Abstraction Layer Version = "10.0.26100.1"
Username 3DIWORKSTATION\3diWorkstation
Time Zone GMT Standard Time
Installed Physical Memory (RAM) 96.0 GB
Total Physical Memory 95.8 GB
Available Physical Memory 81.3 GB
Total Virtual Memory 111 GB
Available Virtual Memory 94.8 GB
Page File Space 15.6 GB
Page File C:\pagefile.sys
Kernel DMA Protection On
Virtualisation-based security Running
Virtualisation-based security required security properties
Virtualisation-based security available security properties Base Virtualisation Support, DMA Protection, UEFI Code Readonly, SMM Security Mitigations 1.0, Mode Based Execution Control, APIC Virtualisation
Virtualisation-based security services configured Hypervisor enforced Code Integrity
Virtualisation-based security services running Hypervisor enforced Code Integrity, Hypervisor-Enforced Paging Translation
App Control for Business policy Enforced
App Control for Business user mode policy Off
Automatic Device Encryption Support Elevation Required to View
A hypervisor has been detected. Features required for Hyper-V will not be displayed.

 

[Colif]

I don't have a Memory.dmp file, is there some other tool I can use to generate the data you need?

That is unusual.

there should be one. Its set up be default

Do you use cccleaner as it can delete it automatically if allowed to.

https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/task-manager-live-dump << shows how to create one

something has been using page file, unless you set a fixed size, as with your ram amount it should never be used unless you using more ram than you have.
I have 32gb ram, and page file auto set and only 2gb.
Yours is 15.6gb so something uses it.

 

[3di]

@Colif thanks. I did set a manual page file to 16gb min 32 max, to see if that would resolve the spike, because the msmpeng.exe was rolling out page faults like they were going out of fashion.


@ubuysa Here's my dump: https://www.dropbox.com/scl/fi/zsg2...ey=mhl7mzev3b5gu8pa0hl7ametg&st=zkoyxijx&dl=0

 

[3di]

ignore the net adapter crash by the way, that was just because I'd tried turning on fast startup. Had to clear the cmos to get the computer to boot after that.

 

[ubuysa]

Do you run a garbage cleaning tool? Something like Disk Cleanup, Ccleaner, etc? If you're getting minidumps there should be a memory.dmp file too.

1. Please enter the Run command sysdm.cpl
2. Click the Advanced tab
3. Click the bottom Settings button
4. Post a screenshot of that window.

 

[3di]

I did run advanced system care a few days ago, it probably removed it.

https://pasteboard.co/v0SJhApAwtB7.png

 

[Colif]

What storage drives do you have?
Just thinking, take different tack on this... its all drive activity so maybe check drives health and see if its not cause of any of it. It probably isn't, just best not to ignore anything

a page fault is when CPU needs data from Ram
a Hard page fault is when it needs it off Drive.

Defender scanning drive is likely to cause lots of them, by its very actions.
So changing the page file size and limiting it to just 32gb doesn't help when you have 98gb of ram, its meant to be able to grow to the size of your ram IF it needed all of it. Right now, limiting it could lead to Out of Memory Errors.

windows won't use it unless it needs it, hence mine is only 2gb. If I needed it, it could grow.

it appears to be set up right, you just need to create one now.

 

[3di]

@Colif I have three 2TB m.2 drives: Western Digital WD_Black SN850X 2 TB M.2-2280 PCIe 4.0 X4 NVME Solid State Drive I'll run a disk check.

Here's the dump: https://www.dropbox.com/scl/fi/zsg2...ey=mhl7mzev3b5gu8pa0hl7ametg&st=zkoyxijx&dl=0

 

[ubuysa]

Your dump settings are fine. The dump must have been removed. TBH it's counter-productive cleaning up error recording information (like logs and dumps) because sooner or later you'll want them.

When was that live dump taken? It really needs to have been taken at the exact same time as you were having audio latency issues. A dump taken at any other time will be of little use.

However, there are a couple of things I've learned from that dump....

I can see a loaded driver called EL_Display_vhidbus.sys which is quite old, dating from 2021...

0: kd> lmDvmEL_Display_vhidbus
Browse full module list
start             end                 module name
fffff802`2e9a0000 fffff802`2e9ad000   EL_Display_vhidbus   (deferred)       
    Image path: \SystemRoot\System32\drivers\EL_Display_vhidbus.sys
Image name: EL_Display_vhidbus.sys
    Browse all global symbols  functions  data  Symbol Reload
    Timestamp:        Tue Jun 29 04:16:03 2021 (60DA7453)
    CheckSum:         0000E8F8
    ImageSize:        0000D000
    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4
    Information from resource tables:

This isn't a common driver we see, so I looked it up. It appears to be a component of WindowexeAllKiller? Please tell me you don't have that installed? It seems to market itself as a junk file cleaner, but it looks EXTREMELY dangerous to me and I would never advise anyone to use anything like this 'tool'. What have you used it to do? Just from reading its instructions it's clear that you can unintentionally do a lot of serious damage with this tool. If it has an undo feature I would use that and undo everything you have changed with it and then uninstall it. Injudicious use may well be the root cause of your latency problems.

The other thing, which may be irrelevant depending on whether this dump was taken as audio latency problems were occurring, are the longest running DPC routines at the time of the dump. We can extract the WMI trace records from the dump and save them in a form that the Windows Performance Analyzer (WPA) can read, buy using WPA to display the DPCs that were running we can identify those that were long running and thus potentially contributing to your latency. The longest running DPCs in that dump are....

nvlddmkm.sys          8.122167 ms
MpKslDrv.sys          6.398402 ms
tcpip.sys             2.775793 ms
Wdf01000.sys          1.346372 ms
dxgkrnl.sys           1.054072 ms
storport.sys          0.359704 ms

Microsoft recommend that no DPC should run longer than 100μs (0.1ms) and all of the above run for longer than that. However, it's common to see dxgkrnl.sys and nvlddmkm.sys running longer than recommended so we can discount those. The tcpip.sys and storport.sys drivers (we do also see stornvme.sys) show that you were probably streaming something at the time of the dump and it's common to see these DPCs running long when streaming. The Wdf01000.sys driver is the root driver for all drivers using the Windows Driver Foundation libraries, so this could be any third-party driver that uses WDF libraries. We do often see Wdf01000.sys running slightly long so I'm tempted to ignore this too.

The outlier then is the MpKslDrv.sys driver, this is a component of Windows Defender and it's not common to see that driver running that long. Do you have any other antivirus product installed (Avast, Malwarebytes, etc.)? I don't see any of their drivers loaded but that doesn't man they're not installed. Multiple real-time antivirus products is a known cause of problems.

This may just be a coincidence, and it will will depend in large part of when this dump was taken. It will also depend on what Windows Defender was analysing at the time.

Having seen a WindowsexeAllKiller driver installed I'm inclined to wonder whether this issue may have been self-inflicted?

 

[3di]

@ubuysa thanks for that. I believe the EL_Display_vhidbus.sys is the driver for the easy canvas app on the ipad. It makes the ipad behave as a graphics tablet with display. The software is from https://easynlight.com/en/easycanvas

The easy and light display hub is also disabled in the device manager.

I only have the windows defender and windows firewall installed, and no other antivirus/firewall.

The spike happens constantly, so I think it should have shown up, but I'll do another dump with nothing at all running, and then create a dump immediately after a spike.

 

[Colif]

This may just be a coincidence, and it will will depend in large part of when this dump was taken. It will also depend on what Windows Defender was analysing at the time.

msmpeng.exe was also showing on one of the latency reports - its also Defender.

would running autoruns and showing screenshots help? Thinking the driver tab?
Autoruns shows everything that loads with windows

download https://learn.microsoft.com/en-us/sysinternals/downloads/autoruns
run as admin as its safe.
click on driver tab
it may look something like this:

now I wouldn't do anything now, just show a screenshot

Image posting via Imgur

go here - https://imgur.com/ you might need to set up a user. click on New Post which will display next screen where you can select image(s) - hold Ctrl while selecting images in explorer to get multiple Once uploaded, images will show on the next page right clicking an image (once its shown...

forums.tomshardware.com

Autoruns can be used to remove drivers or just stop them running with windows.
It might help, I don't know how much Ubuysa can see. So thats one more...

note: rspLLL64.sys is the driver of latencymon. I was curious as I don't like unknown drivers on PC. Sure explains why it crops up in high latency posts... people trying to blame the tool lol.

Or


You can download and run Driverview - http://www.nirsoft.net/utils/driverview.html

All it does is looks at drivers installed; it won't install any (this is intentional as 3rd party driver updaters often get it wrong)

When you run it, go into view tab and set it to hide all Microsoft drivers, will make list shorter.

Now its up to you, you can look through the drivers and try to find old drivers, or you can take a screenshot from (and including)Driver name to (and including)Creation date.

upload it to an image sharing website and show link here It shows every driver on PC, not necessarily the ones running. Not all drivers are needed at startup. Only if necessary.


Note: Forum software is dumb, there is no max page length in characters, its more post count so I am putting spoilers around the latency reports or otherwise page just be too long

 

[Colif]

@Colif I have three 2TB m.2 drives: Western Digital WD_Black SN850X 2 TB M.2-2280 PCIe 4.0 X4 NVME Solid State Drive I'll run a disk check.

try testing them with this
https://support-en.wd.com/app/answe...ive-and-update-firmware-using-western-digital

 

[ubuysa]

Thanks for the info on EL_Display_vhidbus.sys, it sounds less dangerous now!

I wouldn't bother taking more manual dumps, the chances of you catching anything useful is pretty low.

I now nothing about that Focusrite audio device but the ISR code and DPC code (which is what leads to 'latency') are in the device drivers. The ISR code (the front-end of device interrupt processing) is given control when a device presents an interrupt, the ISR designed to run quickly, ideally in 25μs. All the ISR does is store the buffer location, schedule a DPC, flag the device interrupt complete, and then return. The DPC (the back-end of device interrupt processing) is placed on a queue with other DPCs, the queue is executed when a processor is otherwise idle. The DPC accesses the buffer and handles all the necessary processing to move data etc. to complete the device interrupt. The DPC is where the bulk of interrupt processing is done.

In your Latencymon report the DPC times are reasonable, only a handful run longer than 100µs...

Total time spent in DPCs (%) 0.036115

DPC count (execution time <250 µs): 760600
DPC count (execution time 250-500 µs): 0
DPC count (execution time 500-10000 µs): 95
DPC count (execution time 1000-2000 µs): 1
DPC count (execution time 2000-4000 µs): 0
DPC count (execution time >=4000 µs): 0

There are only 96 DPCs that run long and 760,600 that run in a reasonably short time. Those 96 outliers could be causing problems, but I don't think that 0.012% of DPCs running long is going to generate the sort of bad audio issues you report. Incidentally, ntoskrnl.exe is neither a driver nor a DPC even though it appears in these reports. It's the Windows kernel and we expect it to run long.

It's a similar picture with the ISRs, the reported ISR times are generally reasonable...

Total time spent in ISRs (%) 0.000576

ISR count (execution time <250 µs): 67737
ISR count (execution time 250-500 µs): 0
ISR count (execution time 500-1000 µs): 0
ISR count (execution time 1000-2000 µs): 0
ISR count (execution time 2000-4000 µs): 0
ISR count (execution time >=4000 µs): 0

All of the ISRs seen ran within the normal limit (although 250µs is long for an ISR). I'm starting to think that this isn't an ISR/DPC issue necessarily. That means it's either an issue with the Focusrite device or its drivers, or it's those hard page faults I mentioned already.

In case you don't know, a page fault occurs when a reference to a virtual storage page is made and that page is no longer active. If the page is still in RAM - on the Standby list - then the page is simply re-activated and the process continues. These are known a soft page faults. A hard page fault is where the page is no longer in RAM and has been written to the pagefile. This requires an I/O to the storage device to recover the wanted page and that takes time, during which the process waiting for that page is delayed (more latency).

Pages are written to the pagefile for several reasons. An application can request that pages are paged out, it might do this if it's unlikely to need these pages again - pages containing error handling code or program initialisation code for example. The Windows memory manager will steal pages that it detects are being unused - Windows measures how often each RAM page is referenced, unreferenced pages are liable to be stolen. If there is plenty of RAM these stolen pages are typically left in RAM and placed on the Standby list but there are times when they can be directly paged out to the page file.

In your Latencymon report there are a large number of hard page faults, more than I would expect in a 96GB RAM system. These could easily be related to the poor audio performance if these pages were related to an audio app or the Focusrite device/app. You can open the Resource Monitor (enter resmon in the Run command box), expand the Memory section, and then click on the Hard Faults/sec header to sort on that column - so that the processes with the highest Hard Fault/sec rate are at the top. If you leave this display running you should be able to see which processes are taking hard page faults and whether these correlate with your audio glitches.