Most of the octa-core SoCs are actually quad core. They have 4 regular cores, and 4 low-power cores. When there's not much going on (e.g. phone is idling with the screen off) it will run off the low-power cores. When a processor-heavy task is running (like a game) it will run off the regular cores. All 8 cores are never in use at the same time.
In this case however (video playback), the determining factor is going to be how good the hardware video decoder is. Most of the mainstream phone SoCs can decode 1080p or better without problems. But the off-brand ones may not be as capable, and limited to lower resolution as with earlier SoCs, or have spotty performance with 1080p if the SoC is heavily loaded.
I wouldn't invest too heavily in a set top streaming box right now. The AV1 video codec was just finalized in 2018, with a revision in 2019. It is a royalty-free competitor to h.265 with Amazon, Apple, Google, Facebook, Intel, Microsoft, Netflix, and many other big names behind it. Because it was finalized recently, you shouldn't expect to see widespread hardware decode support for it until 2021-2022.
When GPUs start coming with hardware AV1 decoders, I suspect many of the streaming services like Netflix and YouTube will switch to it to avoid having to pay royalties for h.265. When that happens, any streaming box without hardware AV1 decode will have to decode the streams in software. And many of them won't be fast enough to do that, meaning they won't be able to play streamed video from those services. The last time this happened is when YouTube switched from h.264 to VP9, resulting in many of the older smart TVs (which hard hardware h.264 decode but not VP9 decode) becoming unable to play YouTube videos.