I own a lot of very short rods including one of Ed's rods built on a Lamiglass blank, a 5'0" 3wt Winston Retro, a 5'6" 3wt McFarland Spruce Creek, not to mention rods at 6'0" & 6'6" and a slew at 7'0".
All with the exception of a few bamboo rods at 6"6" & 7'0" are fiberglass, all cast just fine with the recommended line and all are wonderful rods for those incredibly tight places.
HOWEVER, the first thing I noticed when fishing a 5'0" or 5'6" rod is how much faster they feel versus their longer counterparts. That requires an immediate timing change to slow things down AND a very short leader.
As it seems Billyboats' rod is graphite and he's using a heavier line, it may be even faster than my short sticks meaning the troubles may be from not allowing the line (if there is any involved) to straighten out and/or the leader is entirely too long.
BTW - What is a #4 leader? If it doesn't mean a four foot leader, I'd start by getting one and trying to slow things down and not over power the cast to compensate. Another thing is make sure you are not using a long front taper line.
A long front tapered line (meaning anything much longer than 7') is a disaster on most short rods fished at short distances, especially the shortest rods fished with a longer leader. That's because there just isn't enough fly line mass out past the tip top, even if you overload the rod with a heavier line.
Good luck!