@56ilywm1tIt's at least arguably the case that some of the relationships in my Variation on a Theme series count.
Spoilers below.
While there is a poly-ish sort of relationship forming between various characters, each of those has a fairly distinct reason ranging from lust to romance to intellectual attraction to other factors.
But there are also close female characters who are in the inner circle but not in the 'harem' (or, at least, I could claim they're not). If you 'fool around' to a limited extent, at very irregular intervals, because there's a question, is that being part of the harem? What if it's full sex, but still very seldom?
Part of it is the difference between 'love' and 'lust.' One of those 'non-harem' characters is a lesbian. She unquestionably 'loves' the male MC, but that love is nearly entirely platonic. A limited amount of fooling around is within her limits, but that's it. There are many ties between them, though, including discussions of his potentially providing the DNA for her future children. Is that 'harem' or not? It's certainly not a lust-based reason.
In any case, it feels like it might work for what you're mentioning. I certainly hope no one feels like it was just written that the various inner-circle characters fell 'crazy in love' with the MC simply to serve the plot.
Well, one might have. Sort of. And, heck, two might have. But neither are currently involved with him in the story, so ... does that count, or not? :)