In all honesty, this question has never really bothered me too much, but I figure I should cover it for completeness.
The argument goes that there is too much evil in the world for us posit a "good" God. If there is a good, all-powerful God, then why is there so much in existence that by its very definition goes against his character? Moreover, how did evil even start? If it was introduced by God, then God is not good, and if it was introduced by something other than God (or is a mistake of God's), then God is not God. Thus, the paradigm of a good God is self-refuting in a world with evil.
As I said, I am not overly troubled by this. It seems to me that if a good God only allows for good, then he necessarily bans free will from existence. I don't think I have the faculties necessary to make a value judgement on which is better, free will or a good universe, but if I had to pick one, I think I'd pick free will. At the very least, we can say that free will presents a powerful counter argument to the problem of evil, and a reasonable explanation as to how a good God and evil can coexist.
But I think there's something deeper here that is often missed. Not only is evil not a defeater of God (in an epistemological sense), I would argue that it actually qualifies as evidence FOR the supernatural. Making any sort of claim about evil carries with it an implicit assumption that good and evil are real things, and that good is definitionally superior to evil. This is ultimately just a rewording of the Moral Law argument. So in arguing against the existence of a good God, we are forced to invoke (or rather admit) a real, objective standard of good and evil.
My own view is that this whole good and evil business doesn't make sense in a purely natural world, and we therefore only have justification for believing in it if we are prepared to admit something beyond the natural world (here by "believing in it", I mean believing that good is superior to evil in an objective way, and is not an arbitrary value judgement conjured up as a relic of our evolutionary past. This is distinct from believing in the fact that we experience the pull of what we would call "good" and "evil", which I don't think any reasonable person would deny). It should be noted that there are a great many very intelligent atheists who disagree with me on this point, and think that an objective moral standard can coexist with a purely natural world.