I've consistently seen independent tests show modern catalytic converters (mid-90s and onward) only affect power by 1-3 hp and on most engine dynos that's within the margin of error and less than the fan, AC, alternator and maybe even the water pump. We didn't have it figured out in the 70s and 80s, but that was only part of the restriction in the systems of that era.
My wife and I were cruising a back road in the mountains and passed an old CJ2 or CJ3 and she commented about the smell. Heresy warning. Even if I had a pre-cat era car now, I'd think about installing a modern converter if viable just to not have that exhaust stink. Lots of compact configurations are available. I used to reek of exhaust after taking my old Fiat 850 spider or Honda CB750 out and personally don't really have a positive nostalgic feeling towards that aspect of those vehicles, fun as they were.