1. Could'a Been A
James did an awesome breakdown of Marvel villains vs heroes in his post yesterday; calling out the motivations of the villains and comparing them to the motivations of heroes. I'm going to take his premise one step further and propose that the best villains could be the heroes if only the story were told from a different POV. The means, no matter how horrible, can be justified if viewed through the right perspective. Imagine if Renfield had told the story of Dracula. Or if Watson had bonded with Moriarty instead of Sherlock.
I love, love, love SFF stories that provide two (or three) protagonists, who are direct conflict with each other while pursing the same goal. Their motivations and means prevent them from collaborating. When the Grand Climax comes, I'm not sure who I want to win because I'm emotionally invested in all of them.
2. On The Cusp of Redemption
Some of the most intriguing villains out there are one Grand Gesture away from redemption. We wonder will they? Won't they? And if they do The Good Deed once, will they do it again? Will they truly be redeemed or is this a ploy to lull the hero in false security? This makes the antagonist the ideal contrast to a hero who is one Bad Decision from villainy. Whenever the villain and the hero are that close in method and motivation...whenever the villain and the hero are one choice--one trait--away from becoming their nemesis, the stakes are higher and the characters compelling.
I'll give a shout out to Papa Midnight in the Hellblazer series as an example.
3. The Noble Duty Diamond Maker
Ah, the rebel protagonist, sure to warm a woman's...heart. But what about the antagonist? The guy so rigid in his performance of duty that he drops diamonds when he walks. Clearly, we know his fatal flaw and the basis of conflict with the protagonist. But why is a guy who represents lawfulness and order a villain? Corruption? Then he's not a Noble villain, he's a different category of needs and desires. If you want a real gut-twisting villain of the Imperious Right, you have to know (and show) why he chooses the methods he does.
Captain Bligh in Mutiny on the Bounty, Javert in Les Miserables, Chauvelin in The Scarlet Pimpernel, all men of duty, all men following orders, all men of the law, all men whose methods were suspect but their motivations were driven by duty to god and country. Noble men. Villains none the less.
There you have it, dear reader, three frameworks to help you dig into the minds of your villains.