Yup - that's where I'm at. Drives my wife crazy because I blame my ignoring her (when I can hear her) on my hearing loss...
For me, it's a little of that and a lot of spending years at race tracks. I have always tried very hard to wear ear protection because my dad had early hearing loss (Korean War and 35 years in a noisy paper mill) and was pretty good at convincing me to protect mine. However, when you're working on a car in a quiet paddock and the car next to you fires up, you're exposed before you have a chance to don ear plugs. Too many of those did the damage for me.
Just in the last year or so, I've started to notice tinnitus, and it is not fun. Getting a good night's sleep is not easy these days. As I sit here typing this, my ears have a dull ringing...