Tis the season to drape thousands of lights upon my tree and listen with half an ear to my Christmas CDs. I enjoy it all; but sometimes at this time of year I miss the companionship of a spouse or partner. Perhaps it's no surprise that, singing along with "O Christmas Tree", I found myself mouthing the words, "O Christmas Tree, you are my only lover." What a sad and lonely Christmas the songwriter must have been having, thought I. Then, sanity reasserting itself, I listened more carefully to hear what they were really singing. By Jove, it certainly sounded like "you are my only lover," no matter how carefully I listened. (The Time-Life Treasury of Christmas, 1987, Disc A, "Medley," Arthur Fiedler and the Boston Pops.)
This carol is interesting because it wasn't written originally in English, but in German ("O Tannenbaum"). As a result, there are no "correct" English lyrics; what you read, hear, or sing depends on the translation used. Each book of carols in which I looked it up and on each lyrics web site, the words are somewhat different. So I suppose it's possible that, in this one translation, someone slipped in "you are my only lover," although I haven't been able to verify that.
If so, I suspect it was the Lady Mondegreen's doing. For more Mondegreen Christmas lyrics, see Snopes.com. And read additional information about Mondegreen lyrics by the author of the entertaining book, Deck the Halls with Buddy Holly and Other Misheard Christmas Lyrics.