Provide great answers
If a user doesn't get an answer at all or there's no activity whatsoever on their question, they wouldn't be too motivated to come back to ask a new question.
If they get a decent answer addressing what they asked, that's good, but it's not really a notable experience and one could probably get a similar result on a number of other sites.
If they get an answer which:
- Explains the underlying concepts they may not understand
- Provides links to useful resources
- Addresses other closely related problems or pitfalls they may also run into
- Helps them avoid similar problems in future
- Addresses other problems in their question
This is a much more memorable and appreciable experience which is likely to make them remember the site fondly and return when they have another question or feel the need to contribute with answers themselves.
If the user posted an answer instead of a question, providing constructive replies / comments adding additional information may help in similar ways.
Show that we care about (good) answers
Someone posting their first answer probably does it to help people. If this answer is posted to an older question (which, I imagine, is the most common use case for a new user), it may get ignored completely or shadowed by older answers. This wouldn't be rewarding for the user.
Some sort of acknowledgement or praise might help a lot. A few upvotes or a comment thanking the user could go a long way, even if the answer isn't perfect. If the answer doesn't add anything whatsoever beyond already posted answers, one could potentially combine thanks with (constructive) criticism.
Provide a great experience
This is similar to the above, but goes far beyond that.
Even if the answers themselves are amazing, they may not want to stay or come back if the site itself isn't enjoyable to use.
A lot of this has to do with the UI:
- Are there popups or obnoxious ads (e.g. overly large or animated ones or ones with irritating placement)?
- Does the site "look pretty" / clean?
- Does using the site feel smooth (e.g. smooth scrolling, no noticeable lag)?
- Is navigating easy and intuitive?
- How much "noise" (things that aren't directly relevant to the user) is there?
- Do buttons and keys do what you expect them to do (based on how things work on other sites)?
How their question is received, especially when it's unwanted, is also important. If their question is downvoted, closed or deleted, they may not want to come back, but this is a balancing act between enforcing rules and being welcoming.
There are also other elements at play, such as which comments people leave, what the UI tells them and how their experience corresponds to their view of the site. As an example: I find the help center of Stack Exchange creates the impression we're quite easy-going about what's allowed here, which does not correspond that well to reality. Preventing new users from leaving comments (and then demolishing their "answers" which are actually comments) also isn't the best experience.
How friendly people are also comes into play here. Avoiding rudeness should be the primary goal. Anything beyond that is quite subjective: some people may want others to be "warm" while others may feel such content is "fake".
Show interesting related content
If you show them something else they want to click on, this makes them stay on the site longer, maybe post more things or get more familiar and comfortable with the site.
If there's little to no exposure to other content on the site while they're here, it might give the sense that there isn't much else going on here, which wouldn't motivate someone to return.
External exposure / showing up on Google
You don't necessarily need them to choose to come back (at least not initially).
If the site keeps showing up on Google when they're looking for answers to their questions, they'll naturally keep ending up here (especially on threads they may be able to contribute to) and thinking about the site.
Gamification
As mentioned by Glorfindel, imaginary internet points can help with retention.
I'm not convinced this would mean much for a first time user though, more for users who've contributed a few things already. And the overall experience also needs to be good for gamification to help.
