Black is a bitch to keep clean.
Along with the dark blues, maroons, burgandies and even the dark greens. At least those dark greens go well with a perfect brown mud. Other mud, especially grey mud, not so much.
And white is also a bitch to keep clean, it shows just about everything on it except house dust.
My TJ is my second silver vehicle and the wife has had two others over the years. As much as I dislike (almost hate) silver vehicles, I've never considered color as a high point when buying a used car and I learned with our first silver car, a 2001 New Beetle TDI, that the silver somehow "hides" quite a bit of what really should be washed off.
Back to the topic of yellow. Here's my CJ5 back in the 90s painted with a local hardware store brand of Federal Safety Yellow.