That's the big thing for me as well I'm at 4.2 because my jump to 4.4 was far to great in the voltage and heat department. I'm sure there's tweaking I could do but I've been too busy to deal with my oc. But I'm rambling, 100Mhz isn't going to be very noticeable, so if you're comfortable with where you're at with 4.4 I'd stick with that.
The dangers are mainly heat related, but if you're only getting 70 degrees, then you're okay. For haswell it seems like the place to get worried is high 80s.