Think of it this way, even if the backlash was double where it should be, you still aren't going to feel .020 while driving. It is entirely possible that you are feeling some play due to the miles. But that would be the totality of all the worn parts; spider gears, axle shafts, bearings, carrier bores, etc.
On top of all that, it is a bad idea to tighten, or significantly change, the backlash on gears with those kind of miles. If you do anything, rebuild the differential.