I just did some of this on my 2005 and learned it all on this forum.
Camber you cannot readily do anything about.
Caster you can, and it makes a huge difference. Think about a shopping cart. The caster is opposite a car, and the front wheels at speed rattle back and forth. Low or negative caster and you start to feel that, and the wheels go back and forth even slightly, it feels all over the road.
You need adjustable control arms. Then get a level on too of the ball joint and adjust your caster to a 6+ ish and test drive. There is no one exact number, and the main reason I have the level/angle finder on there is to know where I am, and try and get both sides of the axle the same. What the number is don’t get wrapped around the axle, but 3-4 for me and mine would drive all over the road.
A big tell take sign that Caster needs to be fixed, does your wheel return to center when you turn. My TJ I’d turn and that MFer would keep going in a freaking circle until I foxed it.
I know personally, if you follow all the steps and get it all tight, you can have a TJ with lift that drives like stock.