scarp VGA. as smorizio said, its an analogue connection, so its will perform the worst.
between DVI and HDMI, DVI is usually more compatible for most monitors today (up to 1440p, most of them use DVI or DisplayPort as the main input). There are 2 types of DVI. DVI-I and DVI-D. the difference is that the D version carries only digital signal, but the I version carries also analogue connection, as well as the digital one. thats why GPU's come with DVI to VGA adapters. dual link DVI (which is the most popular nowdays) can go up to 2560x1600 @ 60Hz.
HDMI is useful if you are connecting your PC to a TV. most (or even all, with modern TV's) of them use HDMI as the main (or even the only) connection to the TV. thats why consoles use HDMI and the video source. there are also 2 versions on HDMI. HDMI 1.4a and HDMI 2.0. 2.0 is only available on new 900 series nvidia cards, and it can do up to 4k @ 60Hz. 1.4a is the most popular one today, and it can do up to 4k @ 30Hz. they are backwards compatible so you shouldn't worry about it (unless you have a 900 series card and a 4k TV that supports HDMI 2.0).
theres is another standard, and it's called DisplayPort. the main advantage of displayport (on version 1.2a, which is the newest), is that you can run variable refresh rate displays (G-Sync on nvidia and Free-Sync on AMD). it can do up to 4k @ 60Hz.