tl;dr: It's not clear exactly what happened in the Hog's Head, but Aberforth was probably the one who threw Snape out while Albus was focused on hiring and protecting Trelawney. They also probably didn't have the evidence or the authority to bar Snape from leaving, and besides, Dumbledore's not the type to simply kidnap a suspected Death Eater who couldn't be proved to have done anything wrong.
There are a few discrepancies between Dumbledore's account in Book 5 of what happened in the Hog's Head that day, and Trelawney's account in Book 6. In particular, Dumbledore says that
"My — our — one stroke of good fortune was that the eavesdropper was detected only a short way into the prophecy and thrown from the building."
(Book 5, Chapter 37: The Lost Prophecy)
Meanwhile, Trelawney recounts that
“Yes, there was a commotion outside the door and it flew open,
and there was that rather uncouth barman standing with Snape,
who was waffling about having come the wrong way up the stairs..." (Book 6, Chapter 25: The Seer Overheard)
This seems to contradict the fact that Snape was "thrown from the building" halfway through Trelawney's prophecy. So it's not entirely clear exactly what went on that day, nor is it clear whose recollection is more accurate. On the one hand, I would take Dumbledore's memory over Trelawney's any day, not in the least because Trelawney was unconscious during the key moment; on the other hand, we know that Dumbledore was deliberately withholding information from Harry, specifically the identity of the eavesdropper.
Still, in the absence of any evidence pointing otherwise, let us assume as Himarm did in his answer to the linked question that neither Dumbledore nor Trelawney are truly misrepresenting the course of events. To be fair, when Dumbledore says that Snape was "detected only a short way into the prophecy and thrown from the building," it doesn't necessarily mean that Snape was thrown out halfway through the prophecy — only that he was detected halfway through the prophecy. He could have been — and, indeed, if Trelawney is to be believed, must have been — "thrown from the building" after the prophecy's completion.
But, if Dumbledore is to be believed, Snape was thrown from the building. To finally answer your question, however, I don't think Dumbledore was directly responsible for throwing Snape out. The way he phrases things in that first quote sounds a lot more like something Aberforth did, while Albus was busy with other matters — presumably, with hiring Trelawney:
"Well, after that, you know, Dumbledore seemed much more disposed to give me a job..." (Book 6, Chapter 25: The Seer Overheard)
At this point, Dumbledore's top priority would have been to protect Sybill Trelawney; if he didn't, and Voldemort found out about the prophecy, Voldemort would undoubtedly go after her. That was why Dumbledore hired her, and that was what he was no doubt doing while Aberforth took care of Snape.
Perhaps throwing him out wasn't the wisest decision in hindsight, but what else could they do? They didn't know exactly how much he had heard, and they were no doubt still trying to figure out the implications of Trelawney's prediction (remember, Dumbledore never thought much of Divination in the first place; there was still the chance that Trelawney was just faking it to impress him).
More importantly, Snape hadn't done anything technically illegal, nor was there likely to be any hard evidence of any prior illegal activity which would be grounds to arrest him. Just being a suspected Death Eater probably wasn't enough to, say, land him in Azkaban (after all, plenty of Death Eaters wormed their way out of imprisonment). Dumbledore was influential, but likely not quite influential enough (he had no official government authority) to kidnap suspected Death Eaters willy-nilly.
It also just isn't really Dumbledore's style to kidnap suspected Death Eaters, regardless. Even if people are suspicious, if he doesn't have any hard evidence he tends to just give them free rein while keeping an eye on them (think Quirrell in Book 1 and Lucius Malfoy at the end of Book 2). So even if he was the one who let Snape go, it still fits with what he usually does.
