phaelax :
I think the first few responses missed my question. I'm well aware of how to rip a dvd and decrypt it. My question is about what format to compress the video into. Like H.264. I think Cherry Blossoms might have answered it though. With 265 still being somewhat new, I wasn't sure the big differences with it over 264 or how compatible it was with current software. Last time I ripped a DVD I think DivX was the common choice.
After some googling as well, it seems like 264 is really the current modern choice these days.
Ahh my bad.
If the question is which codec to use then the answer would be the latest, most advanced codec that still supports the hardware you wish to play it on.
If you wish to play it on a 2010 "smart" tv than you would probably be limited to mp4 or avi.
If you wish to play it on a fully updated VLC and Windows 10 then you would go with h.265 and mkv.
H.265 having the highest quality, smallest file size files and mkv having the most versatile list of features
(Click the links at the bottom for more information on mkv and h.265)
The "h.265 Intel QSV" if offered by handbrake is the super fast Intel QuickSync encoder I mentioned earlier.
Unfortunately it doesn't support 2-pass encoding which allows you do save on space by reducing quality in scenes that aren't complex meaning you want to use the h.265(x265), h.265 10 bit(x265) or h.265 12 bit(x265)
During the 1st pass, statistics about the video are collected. Then in the second pass, those statistics are used to make bit-rate allocation decisions. (Copy pasted from handbrake)
For example if the credits are just words against a black background the black background would be encoded at lower quality due to the quality not being needed.
h.265 10 bit and 12 bit or available if you happen to find a dvd that is encoded with that color depth and you wish to preserve all those colors. (I dont recall seeing any dvds with 10 bit or 12 bit, but i could be wrong)
Having said all that, in all honesty, I rarely use the avg bitrate dual pass method due to the extra time it takes.
I most always just set it to h.265 Intel QSV and Constant Quality 18 and just have it do it because saving 10% in file size isn't crucial to me.
Feel free to try both methods.
Here are some articles i found on x265 versus x264 and mp4 versus mkv.
https://www.quora.com/What-is-the-difference-between-x265-x264-video-coding-Which-is-advantageous
https://www.quora.com/How-do-you-choose-between-MP4-or-MKV-format