Unless you have a specific drive designed to handle the cold and moisture, its a terrible idea.
A standard flash drive would have a far shorter life expectancy in the freezer.
Buy a fireproof safe if you are really interested in keeping something safe from fire. Because guess what, that fridge is going to turn into an oven when there is fire all around it. It may be insulated, but its not fireproof.