It is like a memory card, but it is soldered to the board. It is part of the reason why these things can be so small.
Look at this Ipad mini 2 board. It contains almost everything.
Image from
iFixit
Yellow is the storage. Next to it is the SOC. it contains the cpu video card and ram(volatile memory) and more.
On most desktops and many laptops you CAN take out the hard drive or solid state drive to test it and try to recover files on another computer. If a drive is encrypted, it is not as easy to recover.
Modern tablets just do not allow this. Best to always keep a backup just in case they fail. Some failures(dead battery, broken screen[as long as you can...