In October we announced the beginning of our third Community-a-thon event. The event took place between October 19 and November 8, 2022.
The goals for the event are similar to those from the previous two years:
Improve Empathy between all Stack Overflow employees and our sites and communities.
Achieve a high degree of participation during the event, that can hopefully carry on afterwards as well.
Increase our collective familiarity with our core product and elicit useful feedback on actionable ways to improve our functionality and user experience.
I’d like to report to the community on some of the stats from the event, as well as general findings and reactions from participants.
Participation & Content:
Out of 500 eligible participants (full time staff as of the beginning of the event who were not on leave) we had a total participation of 428 or 85.6%.
This included 246 staff who signed up to participate as well as 182 staff members who did not sign up but were active on the network during the course of the event and scored points in the contest. It is worth noting that after a year of tremendous growth at the company, we had more signups for the event this year than we had eligible staff for the event in 2021.
Participants were drawn from every department in the company.
Participants who signed up were asked at the beginning of this event to rate their level of experience on a 1-5 scale. 80% of users rated themselves between 1-3 (less experienced) and 20% rated themselves as 4-5 (more experienced).
To help encourage participation and make the event more fun, we had a contest throughout, where participants received points based on activity and engagement on different sites in the network, as well as for posting well-received questions and answers. During the course of the event:
112 different staff members posted 829 questions and answers on 82 unique main and meta sites.
29 users earned a “repeat content bonus” (4 different well-received posts on an individual site), the top finisher on the leaderboard earned it on 9 distinct sites, and the second-place finished earned it on 7 sites.
128 participants earned t-shirts (or opted in for charitable donations in lieu of receiving t-shirts) by scoring over 30 points in the contest
Feedback
We ran a survey of participants after the event, in which participants were asked about their impressions of the network as well as general feedback. We got 56 responses to this survey.
When asked about their plans for future participation on the network compared to their previous levels of participation, 62.5% of respondents stated that they intended to participate more frequently, 34% intended to participate on the same level, and 3.5% were planning on decreasing their levels of participation. (In the chart below, green indicates users who intended to use the network more after the event than before, and red indicates users who intended to use the network less after the network than before).
When asked to rate the priority to which we should give attention to improving user experience on the network for users, the following were all rated as being important (from most important to less important):
- New user onboarding
- Asking experience
- Content discovery
- Answering experience
- Subjective content
- Commenting/chat
During the course of the event, participants submitted 82 pieces of feedback on all aspects of the product and user experience (we used a question on our internal Stack Overflow for Teams instance to collect this feedback). We also received varied feedback from staff relating to many aspects of the network.
Some selections:
Event:
This really helped put into perspective what it's like to be a new user on a Stack site, which is invaluable to me in my role here! And I learned a thing or two!
I was happy that I could actually answer questions and be helpful to others in certain communities!
Onboarding challenges:
I found the attitude, particularly on StackOverflow (in the tags I felt most comfortable operating in) to be somewhat harsh. New users brought their questions to the site and were very quickly barraged with comments, which weren't particularly gentle, telling them they had asked their question poorly with little explanation on how to do better. Sometimes a link was provided, but no actual encouragement. Often these questions were subsequently closed by votes within the first few hours of being asked. The users never seemed to come back to try and improve their questions.
As a new user, it's very difficult to gain enough rep to participate, especially if you're more interested in/able to do curation than in answering questions. The constantly full edit queue was a major roadblock.
There's a really steep curve from joining to being able to participate, and felt like a lot of trying to click things to learn that I didn't have the rep to do them (like upvoting when I didn't have the rep to do so).
Advice for new users:
I thought the new user experience was great. For someone else just starting out, I would highly encourage them to view the help section for whatever network site they are about to post on so they can better understand the moderation rules and have a much lower chance of getting their question closed.
Draft your questions THOROUGHLY before posting
Don't expect to earn significant rep from edits or upvotes at the start. The climb is long and slow.
Barriers to participation:
Most of the questions I can think to ask are already on the site. As a new user, I can't even upvote these questions or answers which means I have no way of interacting. The questions I can answer usually already have a pretty good response. As a new user, I also can't comment on answers that I think could use improvement. Without enough novel information, it feels crappy to create a new answer with the same information, just slightly tweaked. In the end, most of my points come from visiting sites daily, checking for questions and then answering fast enough to beat out the more active users.
Some users with decent rep just like to complain. Especially if your answer is now incorrect due to additional information provided by the OP in an edit. Something to identify that a response occurred prior to an edit and might be stale would be helpful.
I have to admit that I was surprised a bit negatively by the community as members sometimes downvote good questions (either from me or others) without giving any explanation as to why they did downvote. I would suggest linking downvoting up to a certain threshold with a multiple-choice field at least so the op can learn and adjust.
General reactions to the network:
The depth and scope of the sites was eye-opening, even though I went into it knowing that we had expansive topics in the Stack Exchange sites
I would recommend a heavier focus on meta participation. That seems to be where a lot of feedback occurs and it would be good for our staff to see that direct feedback.
My questions got answered very quickly. I found it very helpful to get an answer and not have to wait a while for the community to answer my question.
I really appreciate how fast answers are given and how detailed many of them are. This especially amazes me since people do this for free. It really shows how dedicated the community is to knowledge transfer.
I liked seeing the mods commenting on questions trying to help new users improve their questions. It wasn't necessarily clear to me when I should be commenting vs supplying an answer.
I found that each exchange was really different in standards. Some sites like Physical Fitness SE were pretty lax on questions they allowed that many times were opinions. While other ones felt very pedantic such as Chemistry SE. I think it's important to probably just view so many questions ahead of time to even get a gauge of what's allowed and what's not. The help sections could help, but we probably want people to jump in and ask good questions right away, but that means that reading some long detailed help system is in tension with that.
In summation
This is just a selection of the hundreds of pieces of feedback that we received from different channels. I am deliberately sharing both positive and negative feedback here to make it clear that we are getting a wide selection of this in all areas. Feedback is being shared and dispersed appropriately throughout the product organization as well, where it will hopefully be able to impact future roadmap and product decisions.
Looking ahead, there was a tremendous amount of positive feedback to the event itself, and we are evaluating plans for future iterations.
The event organizers are happy to respond to questions or comments left as answers below.
