And late May/early June, if it's more about fishing and being in a nice area, man I love Penns Creek in the Poe Paddy area.
It's a forested remote valley in the mountains of central PA, and Penns Creek forms a gorge through it with beautiful overviews and a number of hiking trails lead to overlooks and such. Penns Creek, in the gorge, is of course a world famous trout fishery, and the mid-state trail runs along much of it. My wife always liked the area, FWIW.
There are other streams in the region as well, both stocked and unstocked.
As far as lodging, well, that's the tough part. That valley itself is very remote. From outside, it's at least a half hour on dirt roads to get in there. In the valley itself, there are 2 state parks. One is Poe Paddy, which is right on Penns Creek, and very pleasant IMO. But it's a campground, and a primitive one at that. No cabins. The other is Poe Valley, which is a few miles up the hollow. Poe Valley does advertise 3 "camping cottages", but I have no idea what those are like. There's a little lake there, and it's a bit more modernized.
You could also opt for somewhere like Hemlock Acres Campground, which has a number of cabins. Private campground that's fairly easy access to the Poe Valley, but still 20 minutes or so on dirt roads. But it's 10 minutes to Coburn and Millheim, which are small actual towns in the area. The towns themselves are rather quaint and often have things going on. Millheim has an excellent brew pub called Elk Creek Cafe that we all love. Good food and beer, and usually good music on weekend evenings. My wife and I often drove out there from State College on Saturday nights.
7 Mountains Camground is another option. It's got lots of nice cabins and is