TRH The Sussexes cannot control what the media or its consumers focus on. We've seen in both articles from People and the Daily Mail that TRH The Sussexes were supporting military charities involving bereaved children and military fitness. If people want to talk about the appropriateness of wearing poppies, then that's not a failure of the Sussex brand- those people just don't care about what they're advocating for, and never will regardless of what TRH The Sussexes do.
It is true that these events bring mutually good press to both the military bases and TRH The Sussexes, but it would be the same for any celebrity doing the same thing (and many have). TRH The Sussexes, however, get criticized harder than many celebrities with the same philanthropic interests.
The annoyance, in my humble opinion, is due to the fact that in spite of publicly and bitterly leaving the BRF nearly four years ago, TRH The Duke and Duchess of Sussex are still treated like they're still working members: they still interact with high-ranking political and military figures, they still garner a higher prestige than other celebrities of similar caliber, and many fear that their "rival court" might cause trouble for the BRF in the long run.
Speaking purely from a public relations perspective, you are correct that the Sussex's cannot control the narrative and are criticised more than other celebrities. Unfortunately for them, it is the very negative narrative that they themselves built when they left the UK that works against them now and that is most certainly a Sussex failure. Most of the things that they did during that period brought them only notoriety, something that other "philanthropic" celebrities know is a bad thing for burnishing one's image. This is why their (or is it just Meghan who was signed on?) PR team are having to work so hard to try and change the narrative. In my opinion they still have a lot of work ahead of them before they will be seen in the way they wish to be seen by most of the public. Time and possibly their future celebrity children will help in the long run.