I am wondering if a Balrog would have been able to see someone who was wearing the One Ring.
Smaug was unable to see Bilbo when he was wearing the One Ring, so it makes me think that if Frodo had put on the One Ring at the time when Gandalf was facing off against the Balrog in Moria, Frodo would have become invisible to the Balrog.
If Frodo had put on the One Ring in Moria, would he have been invisible to the Balrog?
This is hard to answer since Tolkien was not very explicit about the metaphysics of the unseen realm, even in his private writings. The presentation in-text is a blend of the original conception of invisibility in The Hobbit - where that is the one and only power of Bilbo's ring - and a later idea that the Ring pushes its mortal bearer into a realm of spirit. In general, with "magic" in Tolkien, we shouldn't imagine a fixed set of rules, but something with a moral and aesthetic dimension. The unseen is the same, and I will argue that how it works is not a matter of objective mechanics, but has to do with the nature and purpose of the beings who act in it.
In The Hobbit, the ring's invisibility is effective against orcs, dwarves, wood-elves, men, hobbits, giant spiders, Smaug, Gandalf, and Gollum (not yet counted as a hobbit). I'd count two times for Gandalf - Bilbo finding the group again after escaping the goblin tunnels, and when Gandalf and others are searching for the unconscious invisible Bilbo after the Battle of Five Armies.
In The Lord of the Rings, the Ring makes it easier for Ringwraiths to see you, and Sauron himself can perceive the Ring being worn. The Ringwraiths are in the same unseen realm as Frodo when they see each other on Weathertop. There are no examples, other than the exceptional case of Tom Bombadil, of a ring-user being seen by a non-ring-user. When the Nazgul are fleeing "unhorsed and unmasked", none of the pursuing elves are able to track them - a group which must include some who like Glorfindel had a presence in the unseen. They are described in The Silmarillion as "unseen by all eyes in this world beneath the sun", a simplification, but one which lands differently from something like "unseen by all eyes save those who had beheld the light of the Trees", which Tolkien did not write.
Even Gandalf, who has an Elven-ring, does not seem to see Bilbo when he dons the Ring at the birthday party. Afterwards he says "I am glad to find you visible" and while he might be dissembling, the most obvious reading is that the Ring worked on him just as well as it did in The Hobbit. Gandalf is, I think, able to see Glorfindel's shining aura: his conversation with Frodo makes it sound like he recognizes the description from his own experience. It could be that he's recalling seeing him as Olorin in Valinor, or heard it from another of the Calaquendi, but we see enough other examples of his heightened perception that I'm inclined to take it as first-hand. (Gandalf senses the Balrog through a door, speaks telepathically with others, sees Frodo's hand as translucent after the stabbing, etc.) He can perceive the unseen, at certain times, but cannot see a Ring-user.
My sense is that the unnatural magic of the Sauron-made rings, especially the One Ring, means that they are doing something different from what happens to an elf who dwells in the Blessed Realm. The latter is a gift from the Valar, and the former is a transgressive act of sorcery. It is to do with Sauron's epithet of "the Necromancer" and his specific affinity with death (including, in The Silmarillion, vampires and werewolves and death-cults). The wraith experience is consistently presented as horrifying, which is not true of the Unseen in general. It is cold and dark and isolating. Even orcs fear it, as we learn from conversation overheard in Mordor. The same is true for the Ring's extension of life, leaving you "thin and stretched", as opposed to the true immortality enjoyed by the elves. I am inclined to think that the wraith-world is only one aspect of the Unseen - and a particularly evil one at that - which is not automatically accessible even to those who ordinarily see the unseen. There is some wiggle-room here: the Barrow-wights, the Army of the Dead, and the bodies in the Dead Marshes can be seen even by mortals if the circumstances are right. Those who are in the wraith-world by virtue of Sauron's rings seem to be there more completely and consistently.
Now for the Balrog. A Balrog is another kind of evil spirit, a fire-demon of the ancient world. The Balrog of Moria is the only one known to have survived the destruction of Thangorodrim at the end of the First Age. They have nothing in particular to do with wraiths or the dead, although they can work magic (as in Moria). I do not think that merely being a Maia, even an evil one, is enough to be able to see wraiths. As the Balrog is bound into a monstrous form, it is perhaps less likely than the average Maia to be in touch with the unseen. It is also of lesser stature than Sauron, whose power is in the Ring. I think that it could not see a Ring-user.
In the narrative, the Balrog does not pursue or interact with Frodo specifically - it's more interested in Gandalf. While early drafts, described in The Return of the Shadow, had the Balrog being sent from Mordor, that's not part of the published version. Other early ideas were that its place would be taken by a Black Rider, or by Saruman, which would push the story in slightly different directions. As written, the function is to kill off Gandalf for a while, but not to heighten Sauron's pursuit of the Ring: the unfolding of the story seems to require him to not be quite sure where the Ring might be. The introduction of the Balrog serves to provide a threat that is not from Sauron himself, but is still sufficient to dispatch Gandalf. So there is also some story purpose to having the Balrog be unrelated to the Ring.
