That's intended. The "can't attack" is basically a status effect that will disappear like any other effect when the pokemon is switched.
I don't know if they'll nerf Leafeon specifically by changing this mechanic, but for now it's working as intended.
