Moving ETH and EC20 Coins to Nano

Hey guys, I currently have a bit of MATIC, LINK and ETH stored on shakepay/coinbase. As the price of these have started to rise I'd feel much safer knowing they were stored on a hardware wallet which is why I'm looking into the Nano Ledger.

My question is: if I want to transfer over any of these coins am I going to have to pay the gas fee every time which is currently a few hundred dollars if I'm understanding correctly? As that would basically eat away most of what I currently have in each coin which would make it not viable for me to transfer over to the hardware wallet. Or is the gas fee only for ETH itself and would be cheaper to transfer over the MATIC and LINK? Orrrr am I just completely not understanding how the gas fee works and I wouldn't really have to pay much to transfer those over to my hardware wallet?