It seems quite absurd that Prince Andrew is still in the line of succession and also Prince Harry considering his public display against the royal family. These two are quite the antithesis of Queen Elizabeth and Prince Charles.
https://www.royal.uk/counsellors-state
Removing someone from the line of succession is actually quite complicated. The line of succession is defined by law and the law cannot be changed unilaterally by the King or the Queen in the United Kingdom. Changing the law requires an act of the British Parliament.
To make things even more complicated, the line of succession applies not only to the UK properly, but also to 14 other countries (including Australia, Canada and New Zealand) where the Queen is also Head of State. Accordingy, an act of the UK Parliament called the Statute of Westminster, 1931, which is still in force in the UK, says in its preamble that:
And whereas it is meet and proper to set out by way of preamble to this Act that, inasmuch as the Crown is the symbol of the free association of the members of the British Commonwealth of Nations, and as they are united by a common allegiance to the Crown, it would be in accord with the established constitutional position of all the members of the Commonwealth in relation to one another that any alteration in the law touching the Succession to the Throne or the Royal Style and Titles shall hereafter require the assent as well of the Parliaments of all the Dominions as of the Parliament of the United Kingdom
Therefore, in the best case scenario, to enact any change in the line of succession, the Parliament of the United Kingdom would also need the assent of the "Parliaments of all the Dominions", which, in modern language, would mean all the other Commonwealth realms.
Furthermore, some of the realms, for example Australia (including all Australian states individually in addition to the Federation properly) and New Zealand (but not, in this case, Canada) additionally understand, as seen in the debate over the recent Succession to the Crown Act 2013, that the succession law is also part of their domestic law, so, more than just "assenting" to any new British legislation changing the order of succession, they believe they have to change their own law (by separate acts of their own Parliaments) to keep the symmetry between the lines of succession in the UK and in their own countries. All that is a very lengthy process, which is why, for example, it took almost two years for the Succession to the Crown Act 2013 to come into force in the UK.
Removing someone from the position of Counsellor of State is, however, much simpler. Legislation would also be normally required, unless the person in question had already disqualified himself (or herself) on the ground of domicile requirements for example, but it would be a purely British domestic matter. As I explained before the role of Counsellor of State does not extend to the Commonwealth realms, which already have a Governor General to perform the Queen's role, and, in the event that the Governor General was also unavailable, could call on a so-called "Administrator" of the goverment to replace him or her. In Canada, for example, that would automatically be the Chief Justice of Canada as seen recently when GG Julie Payette resigned and the Administrator took over for six months or so until HM The Queen appointed Her Excellency The Rt Hon Mary May Simon CC CMM COM OQ CD FRGCS to replace Mme. Payette.