Curious why you might discount this factoid as untrue? Not saying you necessarily believe all Diana says, but curious why you are not buying this one bit?
Admitting that she was Mannakee's lover pretty much places Diana in the position of having been the first in the marriage to have strayed, not so? Why would she set herself up for that unless there was some truth to it?
Somewhere in all the literature there is the story that either Charles himself or someone else walked in on Diana sitting on Manakee's lap. Clearly boundaries were being crossed. That Diana attests to them having been lovers seems pretty persuasive to me since Diana spun a lot but she always spun on what could be considered facts.
Bottom line, Charles knew about the flirtation and/or affair, and once he knew action was taken. (My bias is that Charles was not cheating on Diana when he started to become aware of Diana's wandering eye: imagine the effect on him with this knowledge, with the added twist that he was now aware that Diana was not to be 'controlled' in any way).