The only 2 I have never used are punch cards (though I have seen them, they were just before my time) and the Super Disc. Our old family computers growing up had 8" and 5.25 drives, and later we got into floppies when they became available.
My own first personal build back in 2000-1 was frustrating because I was determined to not install a floppy drive in it. But after I had the thing together I realized the BIOS did not natively support boot to CD (Thank you ABIT lol). So I had to borrow the floppy drive from the family PC, make a boot floppy that would load the CD drivers in order to boot to the CD drive for the first year. Then I did my first (and last thankfully) BIOS update in order to have native CD booting capabilities.
I never owned a Zip or Jazz drive because I heard so many nightmare stories about data corruption, and the ever present question of "where the hell do you back up that much information?" lol, now we say the same thing about multiple TB projects, but still, it was a huge problem back in the day.
I remember when the flash drives came out, and thinking "finally, something useful!". My first one was a USB 1.1 drive that had 128MB of space, and it was HUGE lol. I recently got a 16GB USB3 drive thinking that would be enough space, but am now finding that I should have sprung for a 64GB version as 16GB really isn't that much space in the grand scheme of things. I think flash drives will be with us for a good long time yet.
I also remember my first external (duel firewire) DVD burner. It was $500 (and that was a steal considering most were still in the $750+ range at the time). I burned the external interface (they call it fire wire for a reason) about a year in, but that drive lasted as an internal drive for about 5 more years. Considering I now can only get the dumb things to last 6mo to a year with consistant use I have to say that they had something going with those old drives.
Lastly, I wonder if Hologram DVD (or whatever they are calling them these days) will ever arrive, or if blue ray will be the last platter style optical media. With flash and other solid state memory technologies progressing so well, it is hard to imagine optical media lasting much longer. Sure, it will still hold a place for media distribution (audio and movies), but I think it's days are quite numbered for mainstream consumption.