In Book II: The Ring Goes South, chapter 7, Galadriel states that Sauron is only starting to suspect that she is holding one of the three great elf rings, Nenya. How about the other two rings?
At first Sauron did not know, but he could guess:
Concerning the Three Rings Sauron could learn nothing from Celebrimbor; and he had him put to death. But he guessed the truth, that the Three had been committed to Elvish guardians: and that must mean to Galadriel and Gil-galad.
Unfinished Tales Part 2: The Second Age Chapter IV: "The History of Galadriel and Celeborn"
At this time, of course, Galadriel was holding Nenya and Gil-galad had Vilya and (possibly) Narya1.
Vilya passes to Elrond about 4 years after this2, after Sauron is routed by the Númenóreans:
[B]roken and humiliated [Sauron] returned to Mordor, and vowed vengeance upon Númenor. The army that was besieging Imladris was caught between Elrond and Gil-galad, and utterly destroyed. Eriador was cleared of the enemy, but lay largely in ruins.
At this time the first Council was held, and it was there determined that an Elvish stronghold in the east of Eriador should be maintained at Imladris rather than in Eregion. At that time also Gil-galad gave Vilya, the Blue Ring, to Elrond, and appointed him to be his vice-regent in Eriador;
Unfinished Tales Part 2: The Second Age Chapter IV: "The History of Galadriel and Celeborn"
I can't find any further account of what Sauron knows about the Ringbearers in the Second Age. Galadriel's comment in Fellowship suggests that he still only suspects, but doesn't know. It seems likely that he still believed Gil-galad possessed one (or more), Gil-galad being the High King and all, but I can find no reference.
And of course Sauron had bigger problems at the conclusion of the Second Age.
Galadriel suggests that Sauron never became definitively aware that she bore Nenya:
Verily it is in the land of Lórien upon the finger of Galadriel that one of the Three remains. This is Nenya, the Ring of Adamant, and I am its keeper.
'[Sauron] suspects, but he does not know - not yet.
Fellowship of the Ring Book II Chapter 7: "The Mirror of Galadriel"
Although Galadriel is not infallible (being a character in the book and all), she suggests that she has a degree of access to Sauron's mind, so we can probably take her at her word:
I say to you, Frodo, that even as I speak to you, I perceive the Dark Lord and know his mind, or all of his mind that concerns the Elves. And he gropes ever to see me and my thought. But still the door is closed!
Fellowship of the Ring Book II Chapter 7: "The Mirror of Galadriel"
As for Vilya and Narya, its unknown. Considering Sauron had killed Gil-galad in the Battle of the Dagorlad, he probably knew Gil-galad didn't have any of them. However, it's unclear to what extent he is aware of the others.
It seems reasonable to suspect Elrond and Círdan, as the other two leaders of the last High Elven regions in Middle-earth. But if he merely suspected Galadriel, the most obvious choice in the Third Age, then I don't think we can say anything more conclusive than "he suspected" regarding the other two either.
Gandalf is an interesting case, since Saruman knew that he possessed Narya:
And the Grey Messenger took the Ring [of Fire], and kept it ever secret; yet the White Messenger (who was skilled to uncover all secrets) after a time became aware of this gift, and begrudged it, and it was the beginning of the hidden ill-will that he bore to the Grey, which afterwards became manifest.
Unfinished Tales Part 4 Chapter 2 "The Istari"
Considering how badly Saruman was being manipulated by Sauron through the palantir, I can't imagine Sauron not knowing, but I have no evidence to suggest he did.
1 It's not entirely clear when Gil-galad entrusted Narya to Círdan; according to Unfinished Tales:
Celebrimbor followed [Galadriel's] counsel that the Ring of Air and the Ring of Fire should be sent out of Eregion; and he entrusted them to Gil-galad in Lindon. (It is said here that at this time Gil-galad gave Narya, the Red Ring, to Círdan Lord of the Havens, but later in the narrative there is a marginal note that he kept it himself until he set out for the War of the Last Alliance.)
Unfinished Tales Part 2: The Second Age Chapter IV: "The History of Galadriel and Celeborn"
2 According to Appendix B:
1697 Eregion laid waste. Death of Celebrimbor. The gates of Moria are shut. Elrond retreats with remnant of the Noldor and founds the refuge of Imladris.
[...]
1701 Sauron is driven out of Eriador. The Westlands have peace for a long while.
Return of the King Appendix B: "The Tale of Years" (i) The Second Age
As far as I know, this point is never covered in the LOTR, The Hobbit, or The Silmarillion. It's only stated that the three Elven Rings are hidden from him. My guesses would be:
Sauron is pretty sure one ring is in Lothlorien, although he doesn't know for certain who wields it.
He probably guesses that one is at the Grey Havens, but thinks it it will stay there to give the Elves an way to escape Middle Earth. Tactically he believed it could be disregarded until the end-game. The idea of the Elves giving up a Ring voluntarily and cutting off their own escape may not even have occurred to him.
When Glorfindel reveals himself near Rivendell, Sauron probably assumes he's holding the last Elven ring.
But this is all just speculation on my part.
