The elves left Middle-earth for the Undying Lands after Frodo destroyed the One Ring.
Who was the last elf in Middle-earth?
Was it Arwen, who remained behind for love of Aragorn, until she died of grief after his passing?
Was it Legolas, who stayed for a while before crossing the western sea with Gimli?
Was it somebody else?
Did the elves of Mirkwood leave for the Undying Lands along with the other elves?
There is no last Elf in Middle-earth - there's still Elves here!
Elves cannot live in Middle-earth indefinitely, embodied. Though the Three Rings helped keep them around and embodied, all Elves in Middle-earth were subject to fading, the loss of their corporeal bodies.
On earth the Quendi suffered no sickness, and the health of their bodies was supported by the might of the longeval fear. But their bodies, being of the stuff of Arda, were nonetheless not so enduring as their spirits; for the longevity of the Quendi was derived primarily from their fear, whose nature or ‘doom’ was to abide in Arda until its end. Therefore, after the vitality of the hröa was expended in achieving full growth, it began to weaken or grow weary. Very slowly indeed, but to all the Quendi perceptibly. For a while it would be fortified and maintained by its indwelling fëa, and then its vitality would begin to ebb, and its desire for physical life and joy in it would pass ever more swiftly away. Then an Elf would begin (as they say now, for these things did not fully appear in the Elder Days) to ‘fade’, until the fëa as it were consumed the hröa until it remained only in the love and memory of the spirit that had inhabited it.
But in Aman, since its blessing descended upon the hröar of the Eldar, as upon all other bodies, the hröar aged only apace with the fear, and the Eldar that remained in the Blessed Realm endured in full maturity and in undimmed power of body and spirit conjoined for ages beyond our mortal comprehension. (History of Middle-earth, Volume X, Myths Transformed)
When an Elf faded, they had the choice of going West. But not all heeded the call.
But it would seem that in these after-days more and more of the Elves, be they of the Eldalië in origin or be they of other kinds, who linger in Middle-earth now refuse the summons of Mandos, and wander houseless in the world, unwilling to leave it and unable to inhabit it, haunting trees or springs or hidden places that once they knew. Not all of these are kindly or unstained by the Shadow. Indeed the refusal of the summons is in itself a sign of taint.
It is therefore a foolish and perilous thing, besides being a wrong deed forbidden justly by the appointed Rulers of Arda, if the Living seek to commune with the Unbodied, though the houseless may desire it, especially the most unworthy among them. For the Unbodied, wandering in the world, are those who at the least have refused the door of life and remain in regret and self-pity. Some are filled with bitterness, grievance, and envy. Some were enslaved by the Dark Lord and do his work still, though he himself is gone. (HoME, Volume X, Laws and Customs of the Eldar)
So, the spirits of lingering and rebellious Elves are still haunting Middle-earth, indefinitely.
As to characters we know, probably the last Elf we know to have left (not the last elf to have been there), is either Círdan or Celeborn. It seems to have been Círdan's task to man the Last Ship, to ensure that every Elf that wanted to physically leave could do so, but at the same time in the Lord of the Rings prologue, we have this:
It is probable that Meriadoc obtained assistance and information from Rivendell, which he visited more than once. There, though Elrond had departed, his sons long remained, together with some of the High-elven folk. It is said that Celeborn went to dwell there after the departure of Galadriel; but there is no record of the day when at last he sought the Grey Havens, and with him went the last living memory of the Elder Days in Middle-earth.
Since Círdan was also around throughout the Elder Days, the implication is that either he and Celeborn left together, or Celeborn left after Círdan. It is also conceivable that, since the departure of Círdan and Celeborn is unrecorded, that they left after Merry and Pippin's death (sometime after 1484) but before Legolas left in 1541 - if we don't consider Legolas to have memories of the Elder Days. Notably, Legolas did build his own ship.
Cirdan certainly intended to be the last of the elves who did eventually sail into the west. From Appendix B of LOTR:
“Take this ring, Master… for your labours will be heavy; but it will support you in the weariness that you have taken upon yourself. For this is the Ring of Fire, and with it you may rekindle hearts in a world that grows chill. But as for me, my heart is with the Sea, and I will dwell by the grey shores until the last ship sails. I will await you.”
Technically, one could argue that you could count Maglor the son of Feanor as the last elf in Middle-earth since we don't know for sure what happened to him. There are multiple ways of interpreting his fate.
"And it is told of Maglor that he could not endure the pain of which the Silmaril tormented him, and he cast it at last into the sea, and thereafter he wandered ever upon the shores, singing in pain and regret beside the waves. For Maglor was mighty among the singers of old, named only after Daeron of Doriath; but he came never back among the people of the Elves."
The Silmarillion, "Quenta Silmarillion: Of the Voyage of Eärendil and the War of Wrath"
I'm basing this on known Elves who remained behind in Middle-earth, not the ones who eventually left for Valinor (which has already been answered)
