I've occasionally seen letters with incorrect graduate program names, and it's not something I'd worry about. It's never bothered me, and while I've seen other committee members point it out, I've never gotten the impression that they particularly cared. (They've pointed it out in the spirit of "Oops, someone messed up" rather than "Here's a negative factor we should keep in mind.")
There are two cases in which it could be harmful:
If you name a substantially lower-ranked program, then the reader might wonder whether this low ranking influenced your recommendation. (Maybe you have several variants of the letter, and send more enthusiastic variants to lower-ranked programs.)
If you include any comments about how suitable this specific program is for the applicant, then of course these comments will become worthless if you name the wrong school. However, most letters for applicants to graduate school don't include such comments, and even when they do it's rarely a crucial part of the letter. (It's particularly awkward if you send Harvard a letter talking about what a perfect fit Yale would be and how you have encouraged the applicant to go there if accepted.)
You mention the possibility of hurting the applicant by revealing where else they have applied. Theoretically, this could look weird or problematic if they've made eccentric choices about where to apply. (If a top program knows they're also applying to a much lower-ranked program, then the committee might take it as an admission of weakness or an indication that the applicant has a strong personal reason to go elsewhere.) However, I wouldn't worry about it in practice. It's expected that everyone will apply to a range of schools, and I don't recall ever having seen a case I thought would really worry admissions committees.