There may be two issues here, but description is not entirely clear.
First, a fundamental issue. You probably have at least TWO different ways for your system to send out audio. Your moboi has a Realtek audio output system that sends out signals via 3.5mm sockets on the mobo back panel. I expect that you also have a video card in a PCIe slot. Now, there is no way for an added video card to get an audio signal from a mobo Realtek system. So virtually all video cards have their own second audio chip on them, and very often when that card is first installed it sets itself up to send audio out through that card, and not through the mobo back panel.
But Windows can use only ONE audio output system at any time. YOU can change which output system Windows uses, and then YOU must ensure the correct output line is connected to your speakers. At bottom left type into the search window "sound settings", then let it open that window. The first item is the Default Sound Output device. Use the drop-down chooser to see what devices you have, and I be there are a Realtek system PLUS a system named like your video card. If you choose to use the video card system, then all audio output will come from that card and its ports. Both HDMI and DP ports can and do carry audio with them. So at your MONITOR if it is set to use audio from that video cable input it will work. BUT you say you have a separate cable from your MOBO back panel 3.5mm socket to the monitor. So if your monitor is set to use that input line for its audio signal, it is NOT getting anything if your system is set to send sound from the video card! Because you say the Realtek system reports there is nothing attached to it, that may be how your system is operating. So one option for you is to set the MONITOR to get sound from the video input cable and NOT try to use the separate sound cable.
On the other hand, IF you really want to use the mobo Realtek output system and its rear panel 3.5 mm socket, then in that Windows Sound Output screen you can select that. THEN you can set your MONITOR to use the separate audio cable from the mobo back panel into the monitor' audio input socket.
Second possible issue. You mention the "Realtek Digital Output" option in that system, and that may show up also in the Windows Sound Output screen as a choice. "Digital output" on an audio system often means output via an OPTICAL fibre port intended to go to an external amplfier, so it does NOT sent audio out through the 3.5 mm socket. IF you have chosen that Digital Output, change that in the Relatek system to use the Speaker output - that is the 3.5 mm socket.