No. Analog telephone modems (a.k.a. POTS - "Plain Old Telephone Service" modems) worked because when the traditional telephone network (a.k.a the PSTN - "Public Switched Telephone Network") digitized the audio, it did so with 8-bit samples 8,000 times per second, for a total of 8 bits * 8 kHz = 64,000 bps digital audio. It wasn't possible to fully utilize that full 64kbps for data because of problems with the analog <-> digital conversion (ADC/DAC), but if you had a digital modem on a digital phone line (like an ISDN B-channel), you could send at 53-56kbps through a single DAC. But from the analog line, you could only send 33.8kbps through a single ADC.
But the problem with cell phones is that they've never provided a full 64kbps digital audio channel like the landline PSTN did. They use lossy codecs to compress your voice audio down to fit into a 9600 bps channel. So if you want to try to do audio-modulated signaling over that channel, I think you'd be lucky to get 4800 bps (4.8kbps = 0.6 KibiBytes/sec) out of it. And that's assuming you design custom modulation schemes to get the most out of what that audio channel gives you. I have no idea if any of the traditional modulation schemes used by POTS modems would work well over GSM voice channels. For all I know, an off the shelf POTS modem may just completely fall on its face and not even be able to sustain the ages-old 300 bps rate over a GSM voice channel.
Update: I found a reference online that said that doing analog audio modulation over GSM voice channels was typically limited to 2400 bps, so my estimate above was quite generous.
Also note that technologies existed to make dialup-like connections over digital cellular networks but without the severely rate-limiting analog complication of hooking a POTS modem to the headset jack of the cell phone.
One solution was called "Circuit-Switched Data" or CSD, and usually involved hooking your computer to your cell phone via a serial connection of some type (using traditional RS-232 or RS-422 style serial interfaces or IrDA in the old days, and USB or Bluetooth more recently), and having your cell phone give that serial interface direct access to the 9600 bps data connection to the wireless carrier's equipment. Inside the wireless carrier's network, your 9600 bps data stream would be fed into a digital modem hooked up to a digital phone line, so you could use this technology to make a 9600 bps modem call to any modem on the PSTN.
CSD was followed by High-Speed CSD (HSCSD), which was like CSD except your cell phone would use 4 GSM voice channels simultaneously, so you'd be able to make ~56kbps connections, just like (in the downstream channel at least) the last era of POTS modems (V.90).
Over digital cellular technologies like GPRS and EDGE, it was possible to use that serial connection to the phone to do PPP, but that wouldn't have worked to just any modem hooked up to any phone line; the other end of your PPP connection would need to be a PPP server that receives its PPP connections not from modems, but over IP, tunneled in L2TP. This was common among ISPs that outsourced their dial-up modem POPs to third parties like Level 3 and UUNet, and is still common today among DSL ISPs (that's why so many DSL services require PPPoA or PPPoE).
