A June 4 stocking date would mean that the browns were 2.5-3.5 inches and the rainbows were 3.5-4.5 inches long in a typical year. In addition, if you were not fishing within a hundred yds of a stocking point, you might not have been fishing over those particular fish. Typically they are stocked at some locations within the greater fingerling stocked segment and allowed to spread out on their own over time. You could have easily been fishing over last year's June stocked fingerlings, however, which included Browns and rainbows. Those browns were adipose fin clipped so they would be easily identified and separated a year later from the wild browns of a similar size. With thousands of fingerlings having been stocked last year and similar annual stockings in previous years, plus the wild fish, it sounds as though you just had a relatively poor outing that day.
Note: After a few year break from rainbow stockings (browns continued) the mix was reinstated last year, but adipose fin clips were only given to last year's browns. The larger brown that you caught could have been wild or stocked. It is tough to tell by appearance in that river at times, despite years of practice there.
Glad to learn of the fly activity, as it is another positive sign of the river's gradual return from the dead.
Despite the worm containers, it is unlikely that many fish are being kept given the long-standing local reputation of the river. "I would never eat a fish from that river" is the typical local comment that's heard. Because of apparent relatively heavy fishing activity over freshly stocked fingerlings at one particular location, however, the stocking point was moved to another location so that the freshly stocked fingerlings would receive less immediate pressure and associated direct or delayed C&R angling mortality. I am very much looking forward to some trout trips to the river this summer based on past performance.