I went there for a week every year for about 15 years straight with 4 friends. We went this time of the year, some years the week before, and other times the week after, Memorial Day. We always stayed at the Matagamon Wilderness Camoground, which is just outside the northeast corner of Baxter, on the East Branch of the Penobscot River.
We mainly fished for Landlocks in the East Branch, but usually spent a couple days each year fishing one or 2 of the walk in ponds inside Baxter, or traveling to fish the West Branch.
I’m sorry I don’t remember the specific ponds we fished in Baxter (I always just went where the other guys I was with took me, as they had probably made that trip for 10 years before I started going with them), but I do remember the fishing was always good wherever we went. And the black flies were always horrendous! 😃 Regardless, I’m sure you will enjoy it.