I've been obsessed with the weight topic for a while now. When building TJs my experience has been that getting one well built under 4200 lbs is almost impossible, unless you strip them down. Counting the weight of every modification is important, so using aluminum to the maximum extent possible is critical. For armor, the only place I can think to run steel would be on the rockers. This is the beauty of something like Savvy's rocker armor. It uses an aluminum rocker panel, combined with steel sliders. The steel slider allows you to pivot over obstacles better than with aluminum.
Drive-train wise, your biggest weight adders will be moving from a Dana 30/35 combination to Dana 44s front and rear. And, adding a 241 transfer case. This is why you see a 300 lb difference between the sport and Rubicon models. Wheel and tire choices, along with heavy bumpers are other things most don't think about. In the end keeping weight down can be done. It becomes a bit of a religion and is, in the end, very expensive. However, it's worth it. Overall, a lighter and well built rig will perform very well on and off-road. Braking, accelerating, and climbing will all improve.