Some thoughts:

1. I think it is scientifically impossible to get more "first world problem" than this.

2. If the man feels the need to retreat into a space that is exclusively his where he can't be bothered, the real issue is the marriage itself (ie both parties should give one another space when needed).

3. I loathe the term "man cave" (I'd never heard the term "she shed" but that's equally ghastly). Instead, I suggest (assuming we're all frightfully wealthy and can afford them - see point 1) the couple invest in a library, where both parties can go and read in silence.