Clarity definitely affects spookiness. This is why small stream guys love hefty flows, you spook less.
Also a factor seems to be the productivity of the stream, maybe size too. I've noticed on big limestoners (Penns, Spring, LJR, etc.), fish are rising nearly at your feet. You can walk through an area and they recover fast. You can watch fish meander around, even spook em a little, and they'll wonder back in place and start feeding again, maybe just get a little more selective. They just aren't spooky fish. Yet on a smaller, infertile freestoner, they'll scoot like darts at 30 yards and that pool is affected for the rest of the day.
I think as far as the fish are concerned, amount of food available and susceptibility to predators plays a major role. A lot of food = a lot of incentive to feed = less spooky. And smaller streams with shallow water = more susceptibility to predators = more wary fish. Then of course nomatter how wary the fish, the water clarity, current, etc. effects whether they can see ya or not.