Short story in which a scout on a colony ship learns there are no habitable worlds

Question

I'm trying to remember a short story I read ages ago.

In it, there is a colony ship that left Earth in search of a new home after the Earth was destroyed. There are scout ships that go out from the main colony ship to look for habitable worlds. The officers among the scout ships have a ritual where one of them is left alone in space while the ship leaves but comes back later to pick them up. During that time, the one left alone floating in space realizes that if there were other habitable worlds they would have found them. They realize that there are no habitable worlds and all of the high ranking officers already know it.

Answer 1

This is "Riding the Torch", written in 1974 by Norman Spinrad. I first read this around 1985 in a German translation titled "Der Weg der Flamme", which was later reissued as the more accurate, but less poetic "Flammenritt". The eponymous "torch" is the nuclear drive of the ships in which the remainder of mankind travels the stars.

The story is as written in the question: long before the events of the story, what's left of mankind boards a few primitive starships. Eventually they manage to turn their Raft of the Medusa into a galactic carnival cruise with all sorts of luxury and entertainment. However they do not give up the hope to find a new habitable planet to live in, and a fleet of scout ships, with crews that are somewhat detached from the rest of human civilization, is flying ahead to find a new world.

Jofe D'mahl, a producer of "senseos" (virtual shared realities) is provoked into joining the scouts after they ruin the premiere of his new production by spreading news of another promising candidate planet; the scouts suggest that joining them would give him material for a real masterpiece.

After spending several boring weeks removed from all communications onboard the scout ship, D'mahl agrees to follow one of their rituals and leaves the ship in a space suit to expose himself to the void. He returns seemingly broken, since it seems to him that mankind's exploits amount to nothing. The scouts explain to him that their sensors are much better than they let anybody know, and that they know for a fact that Earth was the only habitable planet to have ever existed. When D'mahl confronts them and asks why they hide the truth, they explain to him that this is in fact a recruitment mission, because they want him, as the greatest artist of his time, to reveal the real facts to his peers.

Challenged from the task, D'mahl goes into a writers block, until he realizes that there is no point in crying over a lost home, when mankind has actually build themselves a much better and more fulfilling home with their own hands. In his masterpiece, a parable of a triumphant Job who defeats a mischievous god, he manages to turn a devastating existential truth into a story of mankind's eternal perseverance.

I loved that story then, and love it now, and would recommend it to anybody.