The weight at the outer diameter acts just the same as weight of the tire overall, just moreso

because the further from the center, the greater the effective load [lever principle]). The heavier the tire, the more it will have rotational mass and change-of-direction mass (I'm sure there is a technical term for that). Any increase in mass will adversely affect your drivetrain because it will increase loads as speed changes and as trajectory (direction of travel) changes.
As for the total weight of your vehicle and how much you might "need to loose" to make a difference; quite frankly, that is impossible to answer. It would depend on your total vehicle's weight, its sprung weight vs. unsprung weight, its suspension compliance (softness), gearing, engine, tire grip, etc., etc., etc.
All I can say is, to repeat what others have stated; in general, lighter is better. Will 10 pounds of steel replaced by aluminum make a big difference?...almost certainly not. Will a cumulative 120 pounds of X replaced by Y make a difference?...probably so (though, it may not be really obvious, depending on your vehicle).
Yeah, I know, wishy-washy as hell, but that's the deal...just keep it as light
as you reasonably can.