Pretty much my approach. I despise snobbery against genre fiction, because genre fiction is just as capable of being "art" at its best. If someone tells me with a straight face that The Lord of the Rings isn't a work of art, I pity them.
At the same time, yes, I completely agree that genre fiction wirters should aspire for literary excellence. Personally, I enjoy good stories regardless of genre or lack thereof. I've always thought the term "literary fiction" to be absurd in any case, because all written fiction is "literary".