Although I adore The Remains of the Day (one of my all-time favourites), this novel didn't work for me. Brilliantly written, yes, with a great premise, but dystopian narratives of this kind need an element of rebellion (even if it is doomed - see The Handmaid's Tale or Nineteen Eighty-Four, for instance) and I simply couldn't swallow the characters passively submitting to their fate.
I'm also going to brazenly recommend my own novel Peaceful Quiet Lives, as it seems to be getting scarily relevant to what is happening in America. When I wrote it a few years ago, I had no idea it would get this relevant this quickly. You can check it out here. Alternatively, the first six chapters are available on Medium (chapter 1 is here). Sorry for the shameless plug, but I genuinely think you might like this novel. :)