In the US, except for professional master's degrees like MSW, one can get a master's for free, in less time (and even get paid a stipend) by starting a PhD then dropping out.
(In my PhD, you could do this with nothing but one year of ordinary coursework and a single modern language test.)
And starting a PhD program gives you the option of continuing.
So, why does anyone pay for a master's degree in cases where they could instead pursue the above plan?
One possibility is that if you did want to do a PhD later on, you'll find it much harder to get back into a one if you dropped out of one already.
Also, I've found there are usually many more positions available for master's students than PhD students at a University. For a Masters's, space is usually limited by space in lecture halls, while PhD's are hard limited by the number of supervisors available. For master's projects, each supervisor can usually have several at the same time (in the same year), while for PhD's (depending on the field) supervisors may only take one (or a small number) students per year. Thus it would be easier to get into a (paid) master's as opposed to a PhD.
Finally its a bit of impolite move on the students behalf. Depending on how the funding is structured (for the supervisor) they may not be able to get a another student after you drop out, to "take over" the project. Thus they have wasted time on their project. Its also pretty impolite to the other students who could have taken your place and seen the PhD through to the end.
I suspect that there are a lot of factors.
One is that there are a lot of universities in the US that have masters programs, but no doctoral program. Some of their masters students just continue on from a bachelors at the same place.
A second factor is that there are more needs out there for education and "mad skilz" than you might expect. Programs in management, for example, are often taken by employed (night school masters) for advancement in their current job. Employers used to pay for this, but I think that is rare now. But an advance will come with better salary and more opportunities. I've seen the same to be true among software developers. They want to modernize their skills, but not do research. The masters may be tailored for these needs where doctoral study is not.
A third factor involves what a student's advisor will support. Some will recognize that a student isn't really suitable for a career in academia and will only write letters for masters level applications.
A fourth (probably minor) factor is that not all students understand that you don't need to get a masters first to join a doctoral program in the US and, perhaps surprisingly, no one tells them otherwise.
A fifth factor is that some people just have different goals than others and, while they want to learn more, don't really consider doctoral study and/or don't want to game the system in the way you suggest they can.
A sixth factor, of course, is that the standards and requirements for admission to doctoral study are pretty high and the competition is pretty fierce. Someone faking it will probably give of "tells" that can be caught during the process. If you don't express true commitment in you SoP then you are probably less likely get in.
I just retired and started an MS program. Why didn't I apply to the Ph.D. program?
Admission to the local Ph.D. program is highly competitive. The MS program, not so much.
The requirements for the MS degree are not onerous, no prelims, no qualifiers, I only have to find one other referee besides my final project advisor, i.e. no putting together a thesis committee that may end up having to sit for 4-6 years.
I have enough savings that I can take care of tuition and living expenses without a strain. I've been a TA before. No thank you. I have other things I want to do with my time.
Because I'm funding myself, I can study whatever I can convince some faculty member to advise me on.
The program can be part time, so I can adjust my MS work load to accommodate my other post-retirement activities.
I'm not seeking to start a new career. I'm just exercising my curiosity in a more rigorous way than a self directed reading program.
There are a few other retirees in the program. I accept that we're a bit of an edge case. Most of the other students are already working at jobs they like and that are quite remunerative, mostly in engineering or software development. Taking a position as a funded Ph.D. student would require them to give up their well-paid job for a not so well paid TA or RA. If they really wanted a Ph.D. they could try to convince their companies to given them time and funding to work on their research project at the Ph.D. level. Certainly possible, but a much higher bar.
Consider the example of Stanford. It is significantly easier to get into the MS program there than the Ph.D. program. So, if you can only get admitted to the MS program, and think it will help your career enough to be worth the money, then you might choose that route. I think companies will sometimes pay for all or part of a master's degree.
Other universities simply offer stipends and research positions based on qualifications, independently of whether the student intends to leave with a master's or stay for a Ph.D. And some universities will admit students to the Ph.D. program without promising funding. So it's not always a matter of "pay for a master's, get paid for a Ph.D."
In addition to the other answers:
Many masters programs exist only to generate revenue.
Just going to add another perspective here for people pursuing a masters degree while working. It may be more financially advantageous to pay for a Masters instead of quitting for a year to work on a PhD.
Just to give an example, let's say I'm offered a $33k/yr stipend to pursue a PhD. One option would be to quit my job to get a PhD, but drop out after a year and get a new job with a 21.4% pay increase. Then let's say the cost of getting a Masters is $66,340. I would need to make more than $124,175/yr to be profitable staying with my company and doing a 2-year Masters program compared to taking your approach. There may not be many people this applies to, but there probably are some, especially if they can get lower tuition and/or can't get a good stipend for their PhD.
I will say that as an alternative approach, you may get the opportunity to do a PhD while you work as well, possibly as a collaboration project between your company and the collaborating university. If you do this though, I don't see a good reason to drop out early with your Masters.
For smarter peoples, getting into a PhD program with the intention to leave with a Master's degree (MPhil maybe?) is, simple put, not profitable. We're not even considering the fact that PhD positions are scarce and require significant devotion and perseverance to.
PhD programs usually spans across 5 years or more, and it's rare that you can "quit as Master" in the first two years. More likely, the options are "quit with nothing" and "struggle for another year". On the contrary, one can normally finish their courses and receive a Master's degree in less than two years, and you can start a good-paying job right away. The one-year difference can be much larger than you'd imagine. You can earn more than 3 years' worth of PhD stipend plus your Master's tuition. It's even worse if you failed to "quit as Master" in year 3 and have to struggle for longer, which further amplifies the difference from job-after-Master.
Not to mention that it's easier to hunt down a decent job right before you graduate as Master than when you're quitting a PhD program, and all the efforts to enroll in PhD when a Master is much simpler.
why does anyone pay for it - because human behavior is rarely optimal. The question is posed as a fantasy about a global optimum that barely amounts to a local one. There are differences between programs, and what you see as "of course everyone should" is just an indication of limited experience, nothing more. There's no question here really. It's a ruse :)
One program I'm familiar with expects you to take the PhD qualifier in the first 2-3 semesters if you are on a funded PhD track. For an MSc student, it's basically unnecessary stress, unless they enjoy redundant examinations (how would that jive with the "let's optimize everything" worldview begged by the question?). 2nd failure and you're out, BTW, and you're then nominally barred from the same field of graduate study in that department - no matter whether it's masters, doctoral or post-doc.
Another program I'm familiar with has much more funding available for Master's students than for PhD students, and all good MSc students there are funded just fine.
In India, PhD programs are extremely scarce, while Masters are offered at almost all universities of repute. Given that about 1/5th of all graduates in the world are in India, that might be a substantial influence.
In many places you are not even allowed to start a PhD before you have finished a MSc.
There also are places where no one would actually pay to get either degree but where stipends or employment at the university would pay for the studies.
Well... unless you count "opportunity cost" as an actual cost.
