If you start catching the little ones again, try catching as many of the little ones as you can. Then leave for about an hour. Come back and try again. It's possible that you wont fool the same fish small twice but you may catch the one that was not fooled the first time.