What prevents dragons from destroying or ruling Middle-earth?

Question

There appears to be little defense against them. Is it lack of will to do so? Do dragons not cooperate with each other? Or is there some force that prevents dragons from laying waste to the cities of men and elves?

Answer 1

First, dragons are not all powerful, and the dragons still alive in Middle-earth at the time of LotR are weak compared with the ancient dragons. In "The Shadow of the Past", Gandalf says

It has been said that dragon-fire could melt and consume the Rings of Power, but there is not now any dragon left on earth in which the old fire is hot enough; nor was there ever any dragon, not even Ancalagon the Black, who could have harmed the One Ring, the Ruling Ring, for that was made by Sauron himself.

I note that the only 'contemporary' dragon we know of -- Smaug -- was killed by a mortal man who was not even a Numenorean. Dragons are a terrible foes, but do not completely overmatch Men.

Further, all dragons have a soft spot, a vulnerability:

But Shelob was not as dragons are, no softer spot had she save only her eyes.

Evidently this vulnerability is a characteristic of dragons.

Even the first dragons, while much more powerful than Smaug, have their limits:

There came wolves, and wolfriders, and there came Balrogs, and dragons, and Glaurung father of dragons. The strength and terror of the Great Worm were now great indeed, and Elves and Men withered before him

But the dragon was not the game decider:

Yet neither by wolf, nor by Balrog, nor by Dragon, would Morgoth have achieved his end, but for the treachery of Men.

Even the greatest of dragons can be slain by one of the greater elves:

Before the rising of the sun Eärendil slew Ancalagon the Black, the mightiest of the dragon-host, and cast him from the sky; and he fell upon the towers of Thangorodrim, and they were broken in his ruin.

Second, all we know of dragons says that they are loners. Morgoth can assemble a host of dragons, and Sauron might have been able to use Smaug to good (well, bad) effect, but there's no hint in Tolkien or in fairy tales that dragons work well together. While Smaug was not the last dragon alive, he was the last in the West of Middle-earth and Sauron was not able to procure any others to use in the War of the Ring.

Even if Smaug was Ancalagon the Black, he is only one creature and doesn't seem to be able to organize an empire like Sauron could. To dominate Middle-earth, you need to have organized agents everywhere to exert your will. Sauron had an organization and had people (well, things, anyway) ready to step in and rule newly conquered lands. A single individual, even a powerful one, can't dominate a continent all alone.

Dragons don't seem to do the organization thing. (They do the swoop in and burn and loot thing.)

Finally, Smaug in particular and Western dragons generally seem to have the acquisition of wealth as their main goal in life. Power seems to mean nothing; wealth, especially ill-gotten wealth, is what motivates them. I can see a more powerful dragon than Smaug raiding Gondor and carrying off gold, but I can't see it gaining control and setting up a system of taxation to bring in the cash.

So:

1. They're not powerful enough
2. Even if they were, they don't do organization well enough
3. Power? Yawn. I want gold.

Answer 2

Tolkien never wrote on this as far as I know, so one can only speculate. The vast majority of dragons were killed in the War of Wrath at the end of the First Age. Indeed we do not know of any dragon alive in the late Third Age other than Smaug, who was killed by a single arrow shot by Bard, a normal human, albeit a hero and a descendant of a King.

But leaving aside the question of what dragons there are to attempt to rule Middle Earth, ruling does not seem to be what dragons do. We are shown exactly two dragons operating not under the immediate command of Morgoth. Those are Glaurung (father of dragons) and Smaug. Each attacked and destroyed a single underground fortress, amassed a bed of treasure there, and largely stayed put thereafter. True, Glaurung did eventually pursue Turin, but that seems to have been by the command of Morgoth.

Nor is it at all the case that dragons are invincible. Ancalagon the Black was slain by Eärendil at the end of the First age, Glaurung by Turin, and Smaug by Bard. Attacking or defending against a dragon is no doubt hard and risky, but not impossible. The only dragons seen working together are those under the direct and immediate command of Morgoth. Traditional western dragons are solitary, and Tolkien seems to have followed this tradition. Indeed there is no evidence that dragons are able to breed on their own—it may be that all of them were directly created by Morgoth.

There is no specific force that prevents dragons from "laying waste to the cities of men and elves" but the only times we know it happened were to cities that provided underground halls (created or inspired by Dwarves in both cases) suitable for conversion into a dragon's lair, and the revenge attack on Lake-town.

ADDITION: It is worth remembering that in much of Western tradition Dragons are personifications of greed. This is followed in The Hobbit with the comments on "Dragon sickness" and the fate of the Mayor of Lake-town. This is also true of the dragon in the poem "The Hoard", which the fictional editor of The Adventures of Tom Bombadil comments may be a distorted echo of the story of Turin and Mîm (and so the dragon a version of Glaurung). It is surely true of the legendary dragon Fáfnir, which JRRT no doubt had in mind.