Yes they will spin up when windows seeks for devices and drives or any software that looks for drives even if its unreadable by windows, They will still spin down after a while, but the moment you open Computer, they will spin up.
Years ago when I had backup drive in my computer I used a double pole switch on a molex to sata connector, connected it to the drive, One side of the switch controlled the 12v and the other side controlled the 5v, and I enabled hot swap in the bios for that sata port and ran the switch to the back of the computer by the rear fan, case had a hole perfect for it, I used black wire so it wouldn't stick out.
When I wanted to access the drive, I could flip the switch on, and the drive would spin up and windows would eventually see it and use it, once I was done I could safely remove it like a flash drive and flip the switch off, it wouldn't spin up at all in windows as the only wires connected was just the ground wires.
That was old days, I don't do that anymore, but something like that could be helpful for something like this, especially if its a linux drive you don't need it to power on all the time when you are in windows, you do have to understand how electric and the wires work though, simply switching just the 12v wire, the drive will still get 5v power so you'd need to switch both 12v (Yellow) and 5v (Red) at the same time, and you really don't want to put a switch on the ground (black wires) side.
Also another way you could do this, have an external HDD that has a switch, hook it up to USB 3, and just boot from the drive via USB for linux, Windows don't like to run from USB. When you want to boot to windows, shut off the external HDD, Linux is fine with USB and USB 3 is pretty quick and an HDD shouldn't max out a USB 3 port, a sata SSD will, but it will still be plenty fast.