Question Unzipping 30GB file fills the hard drive.

[amagnu]

Hi there!

I'm trying to unzip a very large file, but it slowly fills the hard drive with cache files or something like that and fails when the HDD capacity reaches zero. I have 411GB of free space so I didn't really think it would be a problem. Is there any way to circumvent this problem or do I have to get more HDD space?

 

[USAFRet]

What file type is it? Some files can compress quite a lot, and with a 30GB zip, it is quite believable that the unzipped result would be over 400GB.

 

[amagnu]

What file type is it? Some files can compress quite a lot, and with a 30GB zip, it is quite believable that the unzipped result would be over 400GB.

Yeah, I probably should have mentioned that. It's a .bz2 file and it's a detailed GIS map of the entire planet so I'm guessing it's quite big. But I had no idea that files could be compressed that much. That would certainly leave me in a bit of a pickle if that's really the case.

 

[USAFRet]

What file type is it? Some files can compress quite a lot, and with a 30GB zip, it is quite believable that the unzipped result would be over 400GB.

Yeah, I probably should have mentioned that. It's a .bz2 file and it's a detailed GIS map of the entire planet so I'm guessing it's quite big. But I had no idea that files could be compressed that much. That would certainly leave me in a bit of a pickle if that's really the case.

A .bz2 is another type of file compression. Depending on the actual file contents...you may well be in a pickle.
Here are some comments on finding out how large the uncompressed result might be:
http://superuser.com/questions/53984/is-there-a-way-to-determine-the-decompressed-size-of-a-bz2-file

"I picked the smallest file for the first test, of course. 140 MB compressed --> 3 GB uncompressed. "

At that same compression ratio, your 30GB file would come out at around 650GB.

 

[noidea_77]

As far as i understand this, you don't need to unpack it. The GIS editor / software will work on the bz2 file directly: http://gis.stackexchange.com/questions/11144/using-planet-bz2-with-arcgis-editor-for-osm-syncing-changes