As a cinema purist, I'm a strictly all-in-one-go kind of viewer. I can't watch films in chunks. Also, funnily enough, the version you watched isn't quite the fullest version of this film, though it is close to definitive, and obviously a vast improvement on the bastardised two hour US cut of 1984 which made no sense whatsoever. Here in the UK (and in Europe) we've always had the longer versions, thank goodness!
One thing I absolutely agree with you on: This film is a masterpiece - probably my second favourite Leone film after Once Upon a Time in the West. It is a staggering, dreamlike, beautiful piece of work, which as you correctly point out, definitely has some uncomfortable edges as well (I've never been happy about that rape scene, even though there is narrative justification for it). But the performances are so great, with De Niro and Woods on top form. It's haunting, enigmatic, moving, and quite, quite brilliant. Phenomenal score by Ennio Morricone as well.
I also agree that there is definitely a case to be made for the latter segment being an opium fuelled fantasy. Such ambiguity is what makes this film so remarkable.
All things considered, you've just made me want to watch this again. I shall dig out my copy.