When does Gandalf learn the nature of Durin’s Bane?

Question

Durin's Bane, the Balrog of Moria, was unearthed at some time in the Third Age.

Shortly afterwards, the city was abandoned, and no one with any sense came that way again for a long time.

As stated in this answer, it is clear in the LOTR films that Gandalf knew the Balrog was there. I am not sure if it is so clear in the books what he knows of its nature.

If he does know of the Balrog in the books, is there any indication when and how he finds out?

The nature of Durin's Bane seems to have been a bit of a mystery for most of the Third Age; the elves do not know. Obviously, the dwarves think that it has gone; otherwise they would not try to retake Moria.

I've checked the LOTR wiki and skimmed The Silmarillion but have not seen anything obvious about this.

Answer 1

Not until it's already upon them

Emphasis mine:

'What happened away up there at the door?' [Gimli] asked. 'Did you meet the beater of the drums?'

I do not know,' answered Gandalf. 'But I found myself suddenly faced by something that I have not met before. I could think of nothing to do but to try and put a shutting-spell on the door. I know many; but to do things of that kind rightly requires time, and even then the door can be broken by strength.

'As I stood there I could hear orc-voices on the other side: at any moment I thought they would burst it open. I could not hear what was said; they seemed to be talking in their own hideous language. All I caught was ghâsh; that is "fire". Then something came into the chamber - I felt it through the door, and the orcs themselves were afraid and fell silent. It laid hold of the iron ring, and then it perceived me and my spell.

What it was I cannot guess, but I have never felt such a challenge.

[...]

'Ai! ai!' wailed Legolas. 'A Balrog! A Balrog is come!'

Gimli stared with wide eyes. `Durin's Bane!' he cried, and letting his axe fall he covered his face.

'A Balrog,' muttered Gandalf. 'Now I understand.' He faltered and leaned heavily on his staff. 'What an evil fortune! And I am already weary.'

^{Fellowship of the Ring Book II Chapter 5: "The Bridge of Khazad-dûm"}

For what it's worth, in the text Gandalf himself proposes Moria as an alternative to Caradhras:

'There is a way that we may attempt,' said Gandalf. 'I thought from the beginning, when first I considered this journey, that we should try it. But it is not a pleasant way, and I have not spoken of it to the Company before. Aragorn was against it, until the pass over the mountains had at least been tried.'

[...]

'The road that I speak of leads to the Mines of Moria,' said Gandalf. Only Gimli lifted up his head; a smouldering fire was in his eyes. On all the others a dread fell at the mention of that name. Even to the hobbits it was a legend of vague fear.

^{Fellowship of the Ring Book II Chapter 4: "A Journey in the Dark"}

Answer 2

It's unclear but probably not until he saw it

The balrog of Moria was awakened in TA 1980, that is, more than 1,000 years before the Fellowship entered Moria in TA 3019. It killed King Durin VI and earned its nickname: Durin's Bane. It also killed King Nain I the following year and many other dwarves before they fled.

The existence of Durin's Bane prevented the dwarves from reoccupying Moria after the Battle of Azanulbizar at the end of the War of the Dwarves and Orcs in TA 2799. After the Pyrrhic victory Thraín, son of Thrór and father of Thorin Oakenshield, tried to re-enter Moria. But Dáin said:

You are the father of our Folk, and we have bled for you, and will again. But we will not enter Khazad-dûm. You will not enter Khazad-dûm. Only I have looked through the shadow of the Gate. Beyond the shadow it waits for you still: Durin’s Bane. The world must change and some other power than ours must come before Durin’s Folk walk again in Moria. (RotK, Appendix A)

At this point, some (at least) of the dwarves knew of the existence of Durin's Bane. This makes Balin's attempted re-colonisation in TA 2989 seem recklessly foolhardy, however, they had just vanquished a dragon and an army of wolves and orcs so maybe they were just overconfident.

Given all this, why did Gandalf not expect to meet a balrog in Moria?

The only reasonable explanation is that no one had worked out:

Durin's Bane = Balrog

Gimli recognised it as the former, Legolas as the latter.

When he referred to it at the Council of Elrond, Gimli called it "the nameless fear". Later in Lórien, Celeborn said "We long have feared that under Caradhras a terror slept" but did not seem to know the nature of the "terror". Describing his confrontation with it through the door of the Chamber of Mazarbul, Gandalf says, "what it was I cannot guess".

It seems that the dwarves did not know what Durin's Bane was — Durin's folk had not been involved in the wars of the First Age and even if there were refugee dwarfs who had been, that was many generations ago. Presumably, first hand accounts by dwarves who actually saw Durin's Bane would be rare because seeing it and living to talk about it is a neat trick. So the dwarves did not make the connection and neither did anyone else.

Answer 3

First of all: no one has the slightest idea. More than five thousand years have passed between the War of Wrath and the awakening of the balrog and so it's reasonable to think it was not part of everyday lore. Also, TA 1980 is the year when the Nazgûl return to Mordor and lay siege to Minas Ithil, so it's not unreasonable at the time to think Durin's Bane is another kind of Sauron's terrible servants. No one but the dwarves saw it and most of those who saw it didn't survive to tell the tale. Also, the appearance of Durin's Bane — whatever it is — have rekindled the beyond-ancient animosity between the Elves and the Dwarves (ancient: it is rooted in the very creation of the dwarves and Ilúvatar adopting them) so if there were any survivors they certainly did not sit down with Galadriel or Elrond to give them a nice description of Durin's Bane.

So while there's a common knowledge about Durin's Bane as a being far stronger than a dwarf or even an army of dwarves, as for how it looks and even what it is, no one has the slightest idea.

In the movie Saruman had a book showing the balrog as Durin's Bane, well, that just doesn't fit the narrative of the book.

Edit: that Legolas recognized the balrog as such is less surprising, we know his grandfather came from Doriath and saw the War of Wrath and surely have told tales of it many a times to his son and possibly his grandson although there's no data when Legolas was born — for all we know he might have been born in the First Age himself. While many thousand years have passed for the elves it's only three generations.