ERC-721 scaling on layer 2. Nonsense, or am I missing something?

I've seen projects out there that talks about scaling ERC-721 or NFT on layer 2 using rollups. That wouldn't work, right?

In order to trade on layer 2, I'd have to transfer the tokens first. And since ERC-721 doesn't allow multi-token transfer, each token (unique by definition) would need at least 2 transfers on the mainnet, to and from layer 2. That doesn't seem like a complete scaling solution.

I also heard people talking about minting directly on layer 2, then allowing withdraw to mainnet when exit. That's ok if the token starts out in the layer 2 ecosystem and stays there. But doesn't work for existing tokens.

What am I missing?