MSE Feedback post: Feedback on Q1 2021 Community and Public Platform Roadmap

There are two main themes in this post, mirroring the themes that have emerged over the past year. I'll comment on them separately...

Feature work

It’s been years since we’ve invested deeply in the major pain points that Q&A faces.

That sums it up nicely, thank you. All of the rest on this topic serves only to elaborate or obscure this point.

Aspirational language is par for the course, but... Most of us don't come to Stack Overflow to "connect with the broader community to get the help we need with our programming challenges" - we come here because we're trying to get something done and SO has the information we need. When I close that browser tab, hopefully I'm thinking, "great! Now I can get git to do what I need" - not, "well, git's still borked, but what a great connection!"
And this is where SO's dev teams have consistently dropped the ball for years: the current recipe for exposing good information was pretty much nailed down by 2010 - everything since then has been either incremental improvement, or... A frantic and somewhat-futile struggle to mitigate the problems of scaling a knowledge base by thousands of new questions a day. 2009, 2010, 2011 and 2012 each saw the introduction of major, game-changing alterations to the system to respond to issues that were holding back the site; since then, while many fine people have poured effort into it, the net effect has been negative, with issues that were minor in 2013 now called out as critical problems in this very blog post. The foundations have been undermined through years of neglect, and now the repairs will be expensive.

Fortunately, the last year has seen what could be considered a turning point in that trend: key efforts such as the ability to follow posts and the migration to CommonMark hint at a renewed interest in those boring foundational needs. The roadmap for this year hints that this may continue:

• Review queue rework is long overdue: the current system was designed in 2012, for a much smaller volume of work and with much lower expectations. Most of the people who worked on it are long gone, and nothing engenders apathy quite like an inherited legacy system. Regardless of the success of specific ideas finding their way into the new designs, the exercise of designing it and trying to make it work stand to teach a new generation of people an old set of lessons.

• Outdated answers were something we used to talk about as, "wouldn't THAT be a good problem to have?" - well, SO is approaching 13 years old now, and like any teenager it is itching to throw off the trappings of its childhood and start getting serious. And like trying to parent a teenager, any effort made to mitigate this problem is likely to result in lots of yelling and slammed doors - but if y'all persevere, society will be better for your struggles.

• New Editor - given a popular fork of the current editor ran circles around its parent years ago, it's nice to see work being done to finally bring some modernity to Markdown editing on SO. I'm very happy to see the syntax highlighting land - that's been my preferred UX for Markdown for a long while now. The current implementation is pretty rough - I'm using it to type this, and have spent almost as much time fighting with it as I have typing - but, if its creators are diligent (and don't neglect pull requests), it offers to remove quite a few long-standing pain points for both new folks and regulars.

Oh... And then there's election automation. Ehhhhh... this:

This project got delayed a couple quarters back.

...is not accurate. It's been delayed for over 8 years now. It is a cursed project, because the folks who stand to benefit from it aren't users; they're Community Managers. Which brings me to the other theme of this blog post:

Community Management

This... Can also be summed up with one key sentence:

So, this quarter, as we make our next few hires, we’re working on breaking up the work into three sub teams.

Yeeeeaaaah. You're doing a re-org on the CM team, yet again. The CM team turns 11 in 2021, and I'm pretty sure there've been at least 11 re-orgs. When no one at a company knows what to do with the teams they have, the lazy solution is always a re-org: you get to look busy while having a ready excuse for not getting anything done.

I don't want to be so negative here, but... This really should have been a red flag to whoever typed this up; when you lead off the section by talking about how your team is so over-worked that users are feeling pity for them, maybe "...so we're gonna beat them into submission with yet another re-org" isn't... exactly... what you wanna follow that up with.

I'm happy to hear you've finally hired another CM. That'll finally restore the team to the size it was when... Grace Note was hired. In 2011. When there were 65 sites on the network and Stack Overflow averaged around 3-4K questions per day.

But yeah, sure. A re-org is just what the doctor ordered. Whip them lazy CMs into shape! 🙄

Dear reader: does it seem that I'm being too harsh? I assure you that I am not - I was a part of the team when this strategy was tried in 2019, and it did not end well - I'm pulling a lot of punches here, but suffice to say the last half of this blog post scares me, because it indicates that the company still has not learned the lessons from that experience. That said... There is much to celebrate, and it is important to do so properly. So if you want a dose of positivity, head on over to my celebration thread and get your fill of joy!

It's been just over a year since I left the company and I'm interested what happens because I spent over a decade to participating on the sites. I also acquired a few stock options and many friends along the way. Finally, I occasionally find a result on Stack Overflow while doing coding tasks at work. So I'm invested in the future of the company. Like it or not, Stack Exchange Inc. has a business that is directly tied to the health of its community. Teams, Ads, Jobs and even Enterprise depend on the reputation the Stack Overflow community has built over the years.

In the wake of a company foolishly antagonizing its most dedicated contributors, one might expect a sense of urgency in solving the most pressing concerns. Review queues, fixing outdated answers, a new editor and user onboarding are worthy goals. But I want to look at an item that has sorta languished for years because it symbolizes the company's lack of focus:

Automating site elections

Facilitating moderator site elections is one of the many important tasks our Community Management team is responsible for. However, running and managing elections is a very manual and time-intensive process. This project got delayed a couple quarters back. This quarter we’re rolling out improvements over the course of the quarter that automate this process for Community Managers and makes the election experience more streamlined for users as well.

Back in 2016 I praised the network's election system. It's a distinctive feature that practically demonstrates community agency. One of the things I'm proud about my tenure as a CM was expanding the franchise. It was the right thing to do. But there was also a very practical reason to give more sites elected moderators. Elections scaled faster than manual appointments.

In order to have autonomous communities, we need to pick moderators who are already invested in the communities they serve. But moderators have tremendous authority. So you don't want to install petty tyrants to run sites. Instead, the CMs carefully selected three capable people to moderate. Then we had to select one or two more because often the best candidates aren't interested. When there are 20 sites, that means we needed to keep track of 60+ moderators. With 160+ sites, well, you need to either hire more CMs or create better systems to monitor moderators.

Instead, Stack Exchange Inc. stopped hiring CMs and invested next to nothing in CM tooling. (Though I do appreciate the developers who made tooling side project on weekends and whatnot.) CMs identified two projects we thought were critical:

1. a monitoring system built on Bosun and
2. automated elections.

Without my notes, I can't say when we asked for these things, but the network was smaller and the CM team was larger. In retrospect, Bosun was probably the wrong tool for the job. Or maybe I did a poor job explaining the idea. In either case, that never got off the ground and, since it was my idea, I wasted a lot of dev time on it.

But elections is an easy problem. We'd hammered out the steps over dozens of elections. The CMs wrote and rewrote a functional spec with every detail explained. Shog and I automated major chunks of the process (scheduling and tabulating the results). At one of our meetups, we talked with our managers about the myriad of tasks a community team needs to do. And several of us explained that automating elections was critical to managing so many sites.

I'll never forget the response: "Community managers shouldn't be doing double-duty as programmers." At the time, I thought "Yes. Someone finally understands the problem. Now we have executive buy-in to dedicate developer resources to community tools." And I even thought it would fit into the Stack Overflow for Teams strategy because all those teams need community management support. Alas, I misunderstood. The community team is amazing, but it has a lot fewer ex-programmers now. And just a lot fewer compared to 2017.

So looking at the March timeline for "Automating Site Elections" I'm afraid I don't have much confidence it'll happen then or anytime in 2021. In meteorology, this is called the persistence forecast. My hope for progress on election automation has been disappointed so many times in the past, I don't see why this year will be any different. We'll see which is correct in two months, I suppose.

I honestly feel discouraged.

This is a negative take, so read into it what you will, but it's been about a year and change since these changes and communications have come to grace our attention, but I haven't felt like the community has benefited or moved forward in any quantifiable way in that time frame.

What I've observed from the sides is that the improvements the community has been asking for have been moving too slowly compared to the gains that the corporate product has been. I get it, the lights need to stay on, but it does feel like those lights shine just a bit brighter.

With the addition of a curator team, it seems like the company is going to put in place some policies that could inhibit how curators actually operate. I get that there are some improvements that could be made, but the chief source of my discouragement is that the improvements being suggested or implemented come from a source of stakeholder that does not actively use the service. So that terrifies me.

Lack of clarity on the sorts of curation that are valued

Editor's note: I compiled the following three sections from Makoto's clarifying comments in response to a question on the section above.

Can you share more about the policies you're concerned we'll enact through the curator team?

Let's start with the not that thinly veiled sentiment against curators that exists at a level that most curators choose not to engage at - on social media. You don't have to go very far to find a post railing about how someone's question was closed, or people sharing experiences about how their experience on Stack Overflow was amplified because they got their question closed or downvoted (or both). I get that both of those actions can lead a person to feel bad; I sure felt some sting when I got my first downvote.

The issue with this is that for better or for worse, that's the crowd that gets attention. (ref. Twitter-driven Development). They say to you guys that their experience is bad because of such and such reason, and their question was closed and/or downvoted, and they don't like the community for that. Sour grapes all around.

To complicate the emotion, Stack Exchange has been obtuse about what their intentions are around downvotes and other curator-like activities. The overwhelming public signal exists that this activity is unpopular and/or unfavorable, and the position we have taken (which was one that was left some eons ago) is that we should strive to have high quality questions and answers on the site, and while we kinda sorta feel for the person that has to get work done, them trying to outsource their work on us for free just doesn't work.

The company hasn't been direct with us on what it is they actually expect curators to do or what role they are expected to fill. To further add to this, the not-public support for it hasn't been what I wanted. To put it another way, I think over this last year the "top cover" for curators has essentially evaporated. That's why I said what I did about the downvote research post.

Curator team, curator isolation

So then there's talk of a "curator team". Cue PTSD. The team would consist of people who haven't been around as many of us, who don't share the same insights or wisdoms that we do, nor have as deep an understanding of the nuances of what curation actually is. But this belies the bigger issue - curation is happening pretty much for free, and the people who have been inspired to do it are doing so mostly because they choose to, not because they feel any sort of obligation to.

I was one of those people who wanted to curate the site because I felt like it was the right thing to do. I do that much, much, much less nowadays. I couldn't just stop because I still have that muscle memory to deal with duplicates or poor questions. But I don't feel like there's much point to it because it doesn't feel valued within the company.

There's another angle to this. I would be shocked if this curator team both consisted of less than 20 people and was wildly successful. Content curation is a full-time effort and needs to be supported as such. Anything less than that is going to doom this effort. Shog made some points that only Shog could make about re-orgs, and while I don't know any further details, I can only hope that you can expand your team massively to both fill the need and as a show of taking this kind of thing seriously.

What I genuinely fear is going to happen is that you're going to nerf the very tools that you've given the community due to some signals you're getting from those who dislike getting downvoted. I haven't seen any indication that you're not going to, nor any indication that you're committed to strengthening them, either. The people who knew how to strengthen them aren't with the company, and realistically speaking, the suggestions and recommendations to do so are worth more than volunteer time, so I don't know if you'd get them for free. So, I just feel like we're stuck at an impasse.

People will want to improve the site and they'll have their own motivations to do so. However, I really don't think that this will work out unless you actually have proper veterans who understand the network and how things work actually taking input from the community at large.

Conclusion: confession of bias

I acknowledge and own that this is a pessimistic position. The problem is that I don't think I can be talked off the ceiling. You're going to have to show me that the ground isn't lava, for reals this time.

There’s probably a lot of different opinions on this in the mod community, but to me the reason to be careful about when to bug a CM about something wasn’t really about their workload. The ability to just ping a CM is essentially a shortcut, I skip the queue when I do that. A regular user has to post on meta or send an email and wait until a CM reacts. By engaging a CM directly in chat I make this a synchronous conversation, as they generally feel obligated to respond and not ignore me entirely.

Using this ability makes sense as a tool to surface more important issues or time-sensitive ones. If every mod would just dump all their pet issues on CMs unfiltered, this tools wouldn’t be useful. So I don’t think moderators being careful about when to bug a CM is a bad thing, it only becomes a bad thing if even CMs aren’t pinged about important issues because they are so overworked.

To me there was also a difference between issues particular to my site and general SE issues. I felt much more comfortable in bugging CMs about site issues as I was the appointed or elected representative for the site and this was pretty much in the job description.

The CM team has been reduced by SE to a fraction of its size in better times, and growing it again is critical to ensure they can do their job. I don’t see how these reorganizations help, right now it’s simply far too few people.

I’d also be very, very careful about isolating the responsibilities of the “Trust and Safety Team” too much. You really need to know the communities to do this job well, otherwise you’re likely to just set some metas on fire.

I mainly have a few questions around the makeup and planned re-organisation of the CM team.

• The current round of hires has been focused on mainly folks with professional Community management experience. I’m given to understand y’all have at least one hire. When will we be seeing the new hire around the network, and considering the focus on specific requirements, which team do you foresee these hires being in initially?

• We currently have a community operations team of 2 people (that I know of) that occasionally backstops the CM team. I also believe that to an extent

  Additionally, this team will work with the product team to identify tools that will enable better automation and improved workflows for Community Management.“ .

  Sounds a little like what Jon Ericson (the former CM) was doing at the tail end of his time here. What would be the changes that are planned from what we have now, and in the past?

• Trust and Safety Team

  Having a little experience with some of the more … interesting events on SE, an essential part of ensuring a community is safe is the ability to pick up on and handle issues with humanity and some extent of humility.

  This needs, as much as rules - people who enough of the community respects knowing where you stand with the community - and you . Considering past events - what does SE intend to do better - to actually improve things. Strangely enough - done right the curator support team will be the best eyes and ears for stuff like this, so I hope that y’all don’t silo off this team, and ensure the folks are properly equipped to handle… well SE folks.

When one of my users doesn't feel safe, I can't just handle the source of the problem...I also need to make sure my users feel safe and to some extent, that requires trust.

There's remarkably few people with that level of personal trust in the organisation.

How do you intend to build up those competencies in the team?

• Curator Support Team

  The team will communicate regularly with the moderators, the Moderator Council, and community members across the Metas, Teams instances, and chat rooms. 

  This sounds like ‘traditional’ community management. SE’s lost nearly 40 years of experience in a very short period, both due to firing, and folks moving on. The response to the current round of hiring from the mod community was underwhelming from what I can tell. And folks in the community, mods or otherwise are the naturals here.

  And… it's a hard sell now to get them to apply.

  The first two times I applied to be a CM was “It's open - I’m going to regret it if I don’t give it a shot”. People were excited and I knew more than a few mods applied. I had multiple folks convince me to give it a shot this time - and there was very little mod interest. Part of it was the requirements - but generally folks weren't even talking about it.

  As tempting as it is to figure restructuring will make things better - I’d urge y’all to also look back to see what sort of people were effective CMs, at least internally realize the mistakes made over time that brought us the current situation and do better. Consider the barriers that folks who might be a good fit face as well.

  And I guess critically - for the sake of the community - y'all have to get this right. There may be an SO, or even an SE - but a lot of what makes us unique has been how there wasn't a line between staff and community for so long.

We talked. We had open, honest lines of communication that were unfiltered, and that helped get us where we are now. Rebuilding that trust - having a sustainable model for the community management team where they aren't one of the piggy-banks broken whenever SE's in trouble, and where they're as much the voices of the community in the company - trusted , valued and listened to, as much as they're the voices of the company to the community is important. Critically - figure out how big a team you need and keep it that size.

While it could/has been said these were decisions made in the past , they should also inform the long term plans of the organisation. While what's broken is broken, there's always the option of fixing it stronger and better, rather than rebuilding it.

What caught my attention:

2. Building a place for learners

Sure, why not, the nth attempt~ vision to square the circle should finally just work out, right?!

Okay, snark aside, I am actually glad that this is on your "radar".

Especially on stackoverflow, I am looking at questions that really boil down to "can someone sit down with me for a while and work with me through my learning assignment?" And every time I close vote such requests, I feel kinda bad. Sure, when I notice such requests from newbies that really invested time and energy, I try to help with comments as much as possible, but in the end, sending them away "please consider talking to your peers or tutors" is probably not what they expected to hear.

I doubt that just creating a completely new community would solve this, as a learning place is so much apart from a "quality focused Q&A" community.

But if you want to achieve the goal stated just below that paragraph

improving the likelihood a question remains open and gets answered

then this needs to be addressed in meaningful ways that scale. Otherwise, you always end up with x % of requests coming from "learners" who need "teaching", not specific answers. And x being large enough to disrupt the "specific Q&A" business of the site.

But as said, this is a "huge" discussion that would open a whole can of worms on its own. If you go down that route, please put a lot of thought (and research about previous ideas and attempts) into it.

status-completed

This is fixed now

Can we please get a table that is readable on mobile?

It won't scroll left or right, or zoom either. So half the content is lost.

I am using Chrome on Android.

The term “curators” is used in quite a few places. For example, in the first paragraph, it says “… directly supporting our moderators, curators, and general user base”. Then section “3. Improving tooling and automation for growth” starts with “Our moderators, curators, and community managers play such an important role in maintaining our sites”. There’s even a section “Curator Support Team”. However, I’m not clear on what exactly that term means there.

At Definition of Curator by Merriam-Webster, it says

one who has the care and superintendence of something

From that, and the context in the blog, my impression is that “curators” means the members who are providing various “moderator” (i.e., not diamond moderator) type support, e.g., voting, flagging, editing, etc. However, does this also include members who just ask questions or answer them, as they are helping to “curate” the site in general by providing useful content to it? To help ensure there’s no misunderstanding, I would appreciate having a definition of what this term specifically means in the context of it being used in the blog post.

In addition to Luuklag's feedback the second table is missing a footer bar.

Using Firefox on Android.