Firstly, it's worth noting that the word 'Private' comes from the phrase 'private soldier', first used in medieval Britain to signify an individual hired (or forcibly conscripted) to partake in a military campaign, and were usually allowed to go back to their old life, at least until the next campaign. Back in those days, these individuals were not regarded as fully in the army, but mercenaries at best, or serfs at worst. They were soldiers, but not official members of the Army.
Some basics of this idea has remained in the British Army, and the US as well, in the form of withholding rank insignia until promotion.
In the British Army, the lowest rank of Private (Pte) wears no rank insignia.
In the United States Navy, which Star Trek rank structure emulates, the lowest rank of seaman recruit (E-1) wears no rank insignia, although seaman apprentice (E-2) does.
The phrases 'stripped of rank' and 'stripped of all rank' originally comes from the actual act of stripping the current rank, or stripping all ranking insignia from the soldier's uniform, respectively.
Military prisoners are demoted to seaman recruit (E-1) as the most severe punishment, and thus "stripped of all ranking insignia," while in military prison or awaiting discharge. This is to signify that they no longer have a rank that has rank insignia, yet are still subjects of the service.
Thus the phrase 'stripped of all rank' signifies demotion to the lowest rank of Private, and removal of ranking insignia. The phrase does not, by itself, signify discharge from service, although it is common for discharge to follow.
In the case of Michael Burnham, she was tried and sentenced by a Starfleet court-martial rather than a civil one because mutiny is not a civil offense, and thus her sentence must be served before discharge.
As to the question of why the other prisoners wore grey jumpsuits without the Starfleet badge, as pointed out in the answer by Nathan Griffiths, they were civilian prisoners not Starfleet prisoners. Of course, this opens up the issue of why civilian and Starfleet prisoners were travelling together to work the same Starfleet dilithium mine, and the answer is also (partly) to be found in that same piece of dialogue:
Prisoner 1: You hear why we're getting transferred to Tellun? Dilithium pocket went piezoelectric. Ripped apart the bottom of the mine. Bam, 50 cons... vaporized. So, lucky us. We get to be miners for the war effort now.
Prisoner 2: I hear half the screws on Tellun are scumbag Andorians. Andorians are cold in all the wrong places.
Prisoner 1: The three I killed were cold everywhere. That kind of talk got your shiny little panties in a bunch, Starfleet? What are you in for, anyway? Staying out past curfew?
There is a war going on and like during any war, keeping your front-line supplied is absolutely essential. Dilithium is the main source of fuel for Starfleet, and this dialogue infers those civilian prisoners have been sequestered to the dilithium mines to replace the miners killed in an accident. Of course, it still seems a bit of a stretch for Michael Burnham, the first and only convicted Starfleet mutineer, to be there with them, and to answer that we need to look forward to episode 12, Vaulting Ambition.
In this episode it is revealed that Lorca is actually Alternate Universe Lorca, and has been manipulating his way through everything we've seen so far for his own personal reasons. Part of this manipulation involved finding Prime Burnham and getting her onto his ship (due to his relationship with Alternate Universe Burnham).
It is not be a stretch for a Starfleet commander of Lorca's standing to arrange for her transfer to the mines, ensure the Discovery was in place to arrange for the shuttle to get infected with species GS54, and conveniently rescue them. This is all heavily implied when Burnham and Lorca first meet, but only makes sense when the later reveal is made.
Lorca: Michael Burnham. The Michael Burnham, on my ship. When I saw your name on the shuttle manifest, I.. re-read your file. Pulled up your court-martial transcripts, and you're something.
Burnham: Captain, what am I doing on this ship?
Lorca: I guess you might have to ask that storm out there.
Burnham: I received no warning that I'd be transferred to another prison facility, which is customary. Then my shuttle changed course - halfway through the journey.
Lorca: Maybe the universe hates waste.
