I have many friends who are divorced, some amicably, some acrimonious, (because he/she found other loves) and not one expressed a desire to visit their ex wife/husbands grave.
Apart from the ones who wanted to dance on or at least weigh it down!
I don't think most Brits have that outlook. You may visit the grave of a loved one regularly to start with, but times move on. I can't remember the last time I visited my first husbands grave, my son is a different matter, although that is not a yearly event.
I agree, Charles and Diana had moved on with their lives at the time of her death, she hadn't been a part of him living his life long before the divorce, nor he hers.