Ok I got it fixed, I put cam bolts on and fixed the caster would have been a 30min. job if the washers were the right size, it was a simple fix just ground them down a little. Probably took an hour even the grinding and setting to in. Drives great.
I've checked for loose joints and the steering shaft and can't find any slack but when your going down the hwy it walks bad, so I'm not sure what the problem is.