Let’s not confuse anger with oneself with self-blame and harsh self-criticism. Anger is a wonderful emotion to have provided we express it in a healthy, constructive manner. So, the next time you are angry with yourself, go ahead and be angry, but ensure you express that anger in a healthy, constructive manner. That means: no self-blame, no insults, no blurting, ‘Oh, I’m so stupid, what an idiot I am!’
  More specifically:

1. Vent, but not at yourself. That’s the important bit. Treat yourself like you would like to think you would treat someone else: with latitude. Express your anger, but don’t be nasty to yourself! No self-insults!

‘Self-criticism is like a dog barking: it’s trying to help, but doesn’t know when to shut up.’
Unknown.

2. If you do blurt out a self-insult, retract it.
    ‘No, I take that back. I’m not stupid.’
  Always retract those insults, whether or  not you think they’re true. After a while, you won’t bother insulting yourself because you will want to avoid the tedium of having to retract it.
  Besides, the retraction will help you move on.

3. Describe the situation. ‘I made a mistake. That happens.’

4. If you can, nurture yourself.
– ‘It’s not the end of the world. Mistakes can be rectified. Time will pass.’
– Take into account all the times you have done things right!
After all, in life you have done many things correctly. Just having come this far in life is an achievment in itself. So, treat yourself like you would treat someone else: fairly.

In short, if you want to be angry with yourself, go ahead and be angry, but ensure you express that anger in a healthy, constructive manner. When we do that, we fulfil our urge to be self-critical without actually being self-critical.
  What are the benefits?
(1) We can learn from the incident without beating ourselves up.
(2) By not insulting ourselves we come to accept ourselves more. We become easygoing and relaxed.
(3) Being less harsh with ourselves we become less fragile. We add to our resilience.

I also suggest you retract the judgments you make of other people. Instead of, ‘He is so stupid’, retract it, and then rephrase it to be accurate: ‘No, he isn’t stupid. What he said was wrong.’ Can you see the difference? Less harsh and less disconnecting.

Q. ‘We need to be self-critical. The harsher we are on ourselves, the more likely we are to not make the same mistake again.’
It doesn’t work that way. When you insult yourself you’re undermining your self-confidence, and that won’t help you avoid mistakes.

Q. ‘What if I dispute my self-criticism and say, “I’m intelligent!”’
If that works for you, do it. My concern is that if you try to dispute the insult you might hear an inner voice argue with you, and that’s an argument you may end up losing! I suggest you avoid buying into the argument and simply retract your insult. ‘I take that back. I’m not stupid’.
  Anyway, a simple retraction is neutral, so you are more likely to develop the habit of making that retraction.

Q. Why do we insult ourselves when we make a mistake? Why are we so harsh with ourselves?
Two reasons I can think of:
(1) We may have received harsh criticism in earlier years, and have come to think it’s approopriate.(2) It’s Shame visiting from the Dark Forest. When we make a mistake and say something like, ‘I’m such an idiot’, that’s Shame trying to help us not make that mistake again. But when we simply retract each insult, without trying to prove our point, Shame goes away and we get to move on.
  ‘Shame might persist.’
  Shame is a loyal friend, ever vigilant. Just keep retracting its insults. It will eventually stop visiting.