And when you catch a fry there so fragile even though they swim off they probably wont make it far.
The fingerlings are fish you'll want around in a year or two so just leave them alone and up the size as tossing iron said or move on to bigger better fish.
I would only agree with the "moving on" part if you're concerned about damaging a bunch of YOY or yearlings. Upping the size of your offering doesn't necessarily make sense when you're using a size 16 nymph or size 22 dry or something like that. The hatch is what it is. Bigger trout don't just eat bigger nymphs and flies than are present in the stream. Fishing in the morning or evening with barbless hooks for young fish is fine when it's in season. Just handle them with great care.