I convinced my wife to let the hard top off of our 03 go in favor of a new soft top.
We hadn't put the hard top on in over 5 years, and it was taking up a bunch of space in the garage (stored on a rack and pushed to the side, but still taking up a lot of room).
The noise level w/ the hard top was much like riding in a typical car (we also have full doors).
With the soft top it is definitely letting in more sounds from the road, but it is not "too loud" and we don't have to yell or anything.