thirty-two

Update June 3rd, 2024

Thank you to everyone who took the time to share their thoughts. We will review your suggestions and share an update once we have decided on what action to take with working group deliverables.

Over the last few years, a few different projects have required working groups to be formed by willing community users. Some of those were:

  • Moderator Election Revamp
  • OverflowAI
  • Code of Conduct Update
  • GenAI Stack Exchange site
  • Moderator tools

They have been extremely helpful in getting projects off the ground and collecting the necessary feedback to get projects over the finish line. We look forward to continuing to utilize working groups to assist with various projects over the next year and beyond. We have found that working groups are valuable in bringing community members directly into the development process. The conversations that take place in these groups help highlight concerns and allow us to launch projects or policy changes that are much closer to community wishes than we can deliver on our own. We are here to have a conversation about the kind of deliverables community members would like to see from these groups. M ore specifically, we want these groups to be more transparent than they have been in the past.

A brief explainer about working groups

Working groups are valuable because they allow us to bring together a small to medium assembly of users to focus and work on a specific issue. We love broad community feedback and will still be seeking that out as we always have. But, it’s much easier and more effective to have a small but diverse group work towards a common and focused goal in some circumstances. This helps us avoid the “ Too Many Cooks ” problem.

User selection for a working group is pretty straightforward. When looking for participants, we generally have a Google Form that includes questions about working in groups and why a candidate is interested in joining a working group. The relevant community managers and sometimes product managers will review all interested individuals and invite as many individuals as possible who look like they are a good fit. Some working groups are kep t small, and some don’t have a size limit. It all depends on the particular needs of the project. Particular opinion is paid to diversity of opinions and other factors.

Our working groups have typically operated using a private but sometimes public chat and a private Stack Overflow for Teams instance. These groups usually have a schedule they try to adhere to and have specific subjects they are asked to work through. The working groups have a clear mandate for what they should emerge with, in terms of deliverables. Sometimes, there isn’t a schedule and they have just an end goal of completing what they are working through. Oftentimes, they collaborate as a group and focus on one of three things:

  • Updating or crafting a new policy
  • Feedback on a new feature/product
  • Brainstorming and planning out what a new product/feature/process could look like

Once the working group has completed its mandate in collaboration with a community or product manager, some type of deliverable has typically been created. However, in the case of product/feature feedback, these groups might work as a feedback mechanism, and their feedback is acted on as a part of continuous development. When the need for feedback is completed, it will be closed without public acknowledgment other than to those specifically involved in the group. In addition, some working groups might be working on things that include sensitive information. This could mean something like moderator tools and outcomes for projects like those, which more than likely wouldn’t be shared with anyone other than the groups they are specific to.

What kind of deliverables would you like to see from a working group?

As I mentioned, we anticipate there will be more working groups in the future, and we would like to make the outcomes of those groups less mysterious, whether that means a finalized report on the end product or the opportunity for participating members to share their experience working on it publicly like some sort of retrospective post . Or anything else you could think of. Ultimately, we want to ensure that these groups have more transparency in their work and that when they are concluded, we consistently share outcomes with the community.

To be concise, what do you want to see after a working group has completed its work? This could mean something like the following:

  • Reports about what a working group discussed, decided not to utilize, and ultimately decided on.
  • Timelines on how work progressed
  • Community retrospectives on what went right or wrong on a particular project.
  • Data dumps of discussions in public and private places, with individual community member identities anonymized. Some projects centered on moderation tooling or conversations that would help individuals abuse network elements would have the option of remaining private.

Please leave any ideas you have as answers to this post. We will be accepting feedback on this post until May 29th, 2024.

Improve this question
fifteen
  • five
    I'd be in favor of anonymised dumps, preferably with attribution to community_memberNN and SO_staffNN instead of speakerNN if possible. I.e: that we can still link the dots if there's things like "As XXX said, .... " in the talk and have a feeling of from which point of view the message come.
    –  Tensibai
    May 15 at 15:25
  • six
    +1, good question. To be honest, I am getting whiplash here... one day this howler of miscommunication and plain "we don't care about your views, community" , and a week later this?
    –  Stephan Kolassa
    May 15 at 16:33
  • thirteen
    @StephanKolassa Companies are not people. They are groups, with all that entails.
    –  wizzwizz4
    May 15 at 17:13
  • two
    @Tensibai I don't know that dumping everything is a good plan, that can interfere with frank discussion. Openness is one of the things that sounds great in theory and fails miserably in reality.
    –  Bryan Krause
    May 15 at 18:07
  • three
    Could this post be updated with links that give example outcomes from working groups for the 5 examples? OverflowAI wasn't well received , I can't remember seeing anything more positive and theres a track record currently of community feedback being trampled over by ignorant overlords
    –  Sayse
    May 15 at 18:46
  • sixteen
    @StephanKolassa: In addition to what wizzwizz4 said, with the OpenAI partnership in particular, I imagine the decision came down from a much higher level. In general, the CM team tends to lean towards being as honest and communicative as possible.
    –  V2Blast
    May 15 at 19:24
  • @BryanKrause Well, as long as it's properly redacted for the names, that should not be a problem, but I agree this can have a negative effect anyway.
    –  Tensibai
    May 16 at 6:12
  • four
    I still find it quite mentally jarring that the product (SO for Teams) which is sold to "break down knowledge silos" is being used on the public platform to effectively create silos of communication and knowledge. I've participated in three: Staging Ground Beta, GenAI site, and AI Heuristics Development. SG and AI Heuristics makes sense to me being on SOfT. AI Heuristics particularly should be private info. But a lot of stuff on this list of past examples, I just don't get, and feel like they should be discussed on meta.
    –  starball
    May 16 at 6:42
  • twelve
    I can’t help but feel this is bikeshedding. The inner workings such as timelines and data dumps on what has already been decided are completely secondary to what has been set in stone by these decisions . Without any mechanism to influence things after the fact all of these things are just different kinds of window dressing.
    –  MisterMiyagi
    May 16 at 8:25
  • twelve
    I think this simply moves the problem to something that is "almost, but not quite entirely unlike tea". By itself, the proposal looks good, but I can't help but see two fallacies: first, you have an history of "NDA" everyone who would know "private details" of upcoming potential issues. How many times did I see users privy to your secrets commenting that another storm is coming soon but unable to say anything else? Second, considering that the company always ignores questions that would cause them issues to answers, how can I hope that any "deliverable" would not be pre-censored at the source?
    –  SPArcheon
    May 16 at 9:11
  • ten
    The most important deliverable would not come from the working group -- it should be a report from the company saying how they're interpreting the output of the working group, what actions need to be done with respect to the output of the working group, and how it will be prioritized.
    –  Scott Seidman
    May 16 at 14:43
  • four
    A nice report with a basic summary of what the group did, its conclusion, and what's happening next is enough for me. Bonus points for including discussion of dissent: things the group advocated for but aren't going to happen, etc.
    –  Steve Bennett
    May 17 at 1:58
  • two
    I don't know about deliverables, but it may help to strongly recommend/require processes which are known to make such working groups more effective. For example, in pretty much any working group it helps to have meetings with a clear agenda (the agenda item of the first meeting may be how to set the agenda) and to have some minutes with summary of topics, decisions, action points (NOT a word-by-word transcription).
    –  Nemo
    May 28 at 10:37
  • two
    " What deliverables would you like to see out of a working group? " I want to see bugs fixed (review queues broke ~8 months ago and there's been zero dev response to acknowledge it, let alone fix it), parent tags/tag hierarchies implemented, I want to see more heuristics auto-blocking users who type blatantly off-topic questions in the ask question box, and I want to see chat updated for the first time in ~10 years.
    –  TylerH
    May 28 at 19:56
  • two
    Inline tag editing has been partially broken (and on your bug tracker) for close to two years now as well.
    –  TylerH
    May 31 at 15:02

4 Answers four

Reset to default
eighty-six

This is going to sound negative, but I want to be honest:

As a general principle, transparency is good. But I find it hard to care much either way. Personally my desire to see deliverables from a working group is greatly diminished. I want to see improvements to the platform, more than documents from a working group. Documents from a working group don't do me much good. I've lost trust in working groups to lead to improvements to the platform, and so seeing their deliverables is not a high priority to me. I'd prefer you show me the improvements to the platform rather than working group documents.

After the moderator Tools/Policy working group, I've lost trust in working groups. That group put in a bunch of effort and came up with a ton of suggestions for how to improve things that I thought were great. I am hard-pressed to identify any of those improvements that were implemented. The company on its own designed and implemented something different. What the company implemented is cer tainly interesting and was a very reasonable thing to try, though my impression is that unfortunately its utility has turned out to be rather limited on many sites.

My impression is that company just doesn't have a lot of software developer resources to implement the ideas that people come up with. That's perfectly understandable. It's easy to come up with ideas, but a lot more work to implement them. And, there are many additional considerations that a working group is not well-suited to address, which anyone who implements something has to think about. But, given that, what's the point in having a working group put in so much effort if, realistically, the suggestions probably aren't going to be implemented? And what's the point in reviewing deliverables from a working group if, realistically, they probably aren't going to be implemented?

For the Code of Conduct revisions, I am not familiar with anything that required a working group. Soliciting feedback from all moderators seemed fine. Perhaps there is some aspect of the process that I've been unaware of. But again, I wouldn't care about deliverables from such a working group. Rather, I'd want to see and have the chance to provide input on a proposed revision.

My perspective is that the GenAI policy working group has been a failure. The original vision was that some users could agree on some heuristics that mods could use to delete posts that are, with high confidence, generated by AI. But I think that has not been a success. The process has been too cumbersome and bureaucratic, and the heuristics that have been accepted are too complicated to understand, and as a result, I find it irrelevant to my moderation. I find it more realistic to ignore everything that working group came up with and simply treat the policy as saying "don't delete posts because you think they were AI-generated, sorry". So for that working group, transparency and deliverables are irrelevant to me.

I think there is an important distinction between different types of working groups:

  • working groups whose mission is to propose a policy; vs
  • working groups whose mission is to propose changes to the software that runs the platform; vs
  • working groups whose mission is to give the company feedback on such changes.

My thoughts on deliverables for each of these types of working groups:

  • For policy-proposal working groups, we don't need deliverables. We just need to see the proposed policy, and a rationale, just like any other proposal on Meta. And we need a company representative who will engage with the community and accept input.

  • For platform-change-proposal working groups, perhaps all we need to see is the proposed change and rationale.

  • For software-feedback working groups, I'm not sure the company needs a working group, as opposed to a focus group, and I'm not sure any transparency is needed; the company just needs to have a representative communicate with the entire community about the software change and defend it and listen to input.

More broadly, I see working groups as part of a broader trend towards decision-making that is more top-down and less democratic and does less to engage the community and build buy-in. That's arguably a natural evolution, as a community grows. But still a little sad to see.

It's entirely possible I am being too pessimistic or negative. I apologize in advance if this is not helpful.

Improve this answer
one
  • three
    I agree and feel the same about the election revamp working group.
    –  user1271772
    May 18 at 0:02
twenty-three

I don't think there is a universal set of deliverables that would be appropriate for each working group. Instead, once the need for a working group has been noted, one of the first things the working group should do is establish a charter. This charter should outline how the working group intends to validate their work and progress with the broader community.

I suppose if you needed to define a universal set of deliverables, then that set would probably only be the charter. The community can evaluate what problem the working group is working on and the charter and see if the deliverables make sense.

Perhaps, over time with enough similar working groups, you can start with some recommended deliverables. You may find that groups working on policies or groups working on feature ideation or groups working on processes may identify some common deliverables for their type of problem. You may also find that some deliverables may be common across multiple types of problems.

Instead of trying to do too much up-front planning, defer decisions until as late as possible, which is the creation of the working group.

Improve this answer
eleven
  • six
    Eh, I'm recalling a case where a working group of moderators was created and they got absolutely bogged down in exactly this sort of "charter" discussion and everyone lost interest before anything was accomplished. These more focused working groups are the replacement for that. I worry this would be a step back.
    –  Bryan Krause
    May 15 at 18:10
  • twelve
    @BryanKrause If you're referring to the Moderator Council, that isn't what happened. I was there and elected to the Mod Council. The issue was that there was no well-defined problem to solve. There was a group of people ready and willing to work on hard problems facing the community, but no problem was queued up for these people to work on. The proposed working groups solve that problem by forming a group around a specific problem, rather than a group that problems come to.
    –  Thomas Owens
    May 15 at 18:15
  • five
    The task you were given was to figure out what the moderator council would do and how it would operate, and no decisions were ever made on those tasks. Basically to create a Charter.
    –  Bryan Krause
    May 15 at 18:17
  • three
    @BryanKrause I don't think that's a fair characterization at all. The Mod Council was a group looking for a problem. Trying to define a charter for something so nebulous and vague is almost impossible. These working groups are going to avoid that because they will have a specific problem to solve - a specific policy to update, a specific feature to ideate upon, a specific feature to improve. These working groups shouldn't be compared to the Mod Council. The failure of the Mod Council is why this working group approach is happening.
    –  Thomas Owens
    May 15 at 18:38
  • one
    "The failure of the Mod Council is why this working group approach is happening" - which is why I am recommending not re-introducing the problem of starting each working group out with a meta-problem rather than the actual problem they're meant to address.
    –  Bryan Krause
    May 15 at 18:41
  • eight
    @BryanKrause It is quite normal to launch a group with defining a charter. The key difference is that the problem space for a working group is well-defined. The problem space for the Mod Council wasn't. Since every group is solving a unique problem with a unique set of people, it makes sense for each group to define how they want to work in a way that makes sense. This is also a fundamental principle of Lean - defer decisions to the last responsible moment. It makes no sense to mandate, up-front with no knowledge of the problem or people, how those people will work to solve the problem.
    –  Thomas Owens
    May 15 at 18:43
  • four
    Okay, as I think more about it, I do think this can make some sense, but I also think it should be bounded in some way to facilitate a quicker decision. Something like a menu of options, either whole categories or a la carte list of what to release so that this can be done in something like one "session".
    –  Bryan Krause
    May 15 at 20:44
  • one
    From a community point of view the expectations on a working group would probably at least include a final report within a certain time frame including a clear discussion of the problem and the recommended solution and a discussion of advantages and disadvantaged of it. Whatever the working group will be working on.
    –  NoDataDumpNoContribution
    May 15 at 20:52
  • one
    @NoDataDumpNoContribution I'd agree with that. However, I don't know how to make it universally true. For a working group on policy, that would be draft policies and then a final draft proposal. But for a working group on a new feature, that would be things like more detailed mockups showing functionality and specifications of behavior.
    –  Thomas Owens
    May 15 at 23:16
  • two
    I have to agree with @BryanKrause here. "A group looking for a problem" is a group that shouldn't have been formed. If the group is formed, and can't agree on their purpose, disband it.
    –  Scott Seidman
    May 16 at 14:40
  • two
    @ScottSeidman And that's exactly what happened. The Mod Council no longer exists. These new working groups, which are formed around specific problems, replace it. There are also plans for longer-standing groups but with specific focus areas proposed . These groups are also more expansive since they can include non-moderator users.
    –  Thomas Owens
    May 16 at 14:49
seventeen

I'd keep it simple

On the conclusion of a working group

  • A clear timeline for implementation of what the working group was working on

  • decisions based on the working groups conclusion.

  • While I'm going to disagree with Kevin on 'full' disclosure, have a involved Community manager or involved staff member from an appropriate team write a summary of the findings, disseminate it on the group for feedback, and publish it on the appropriate meta for comment and finalisations of plans, with the staff member involved being in charge of dealing with the commentary

  • a commitment to follow through on the actual project taking into account meta commentary for polishing and error correction.

  • if a datadump is provided of posts, it really should be

It is also worth consider the subtext here

SO inc has had a habit of requesting feedback, getting excellent feedback, then losing focus, or pushing through 'features' no one really wanted for marketing purposes. Community feedback is pointless unless it acts as a countercheck for bad ideas, encouraged good ones and is generally heeded.

Practically, we're a chatty bunch and a datadump isn't exactly designed to be readable. My asking for a summary has 3-4 implicit goals. It gives us a summary and forces everyone involved to distil down the findings of the working group's conclusions.

If we needed specific details a meta post lets people ask and has a staff member to follow through. The rest of the community members involved will chime in anyway, and it seems like a good opportunity to exercise those meta writing skills and build connections with the community. Clarity and open-ness can be "Here's everything, go wild" or working with the community to clarify, assuage concerns and generally share if needed.

I disagree that too many cooks is the problem - practically there's been many projects that have had good clear feedback, and forgotten, cause management lost interest. Its rather hard to run a restaurant if you're famous for classical french cooking, but management suddenly decides one week you need to serve Dosai, Mee Siam the next month, and suddenly decides blindfolded, upside down dining is the next big thing. This is how the company has been for the last decade or so.

As such the most important deliverable is the desire for the company to follow through, and see a product that comes out that meets the community's needs, or to not waste effort on something that's clearly going to get broken and mocked, or worse, abandoning something that's useful and well liked.

Improve this answer
five
  • three
    After all these years, you still want to believe in the company's good faith. You still want to spend hours working for them (your community) without any recompense, free of charge? What will be your reward? I don't believe these sub-committees will ever reach a conclusion, they will never be any consensus between working groups and management.
    –  Mari-Lou A Слава Україні
    May 16 at 7:15
  • six
    Honestly? At this point, probably writing meta posts and maybe every so often getting that rare win. Maybe even the chance to be a little smug when I'm right. I personally am not a fan of the idea of working groups without a solid understanding of the community's needs but that's not what its asked about. I do want to give constructive advice as long I can be bothered to. Honestly, If I did it cause I expected something back , I'd be burnt out long ago. I pick matters that're important to be, give subtle nudges in that direction.
    –  Journeyman Geek
    May 16 at 7:27
  • And, maybe this in itself is the value I get from being on meta. I don't think there's really any other explanation or outlet for what I do here, and its not exactly its a transferrable skill :D
    –  Journeyman Geek
    May 16 at 7:29
  • @JourneymanGeek The skill transfers quite well to bureaumancy, I find. (I've been told it helps with political activism, too, but I don't have the stomach for that.)
    –  wizzwizz4
    May 27 at 14:03
  • If I'd made some different choices I might have ended up in politics :D. And in a sense I think a lot of what I do on meta and mod spaces is activism. I'd like the company to be less adversarial to the community, live up to their promises and generally do better. Its politics of a sort :D
    –  Journeyman Geek
    May 27 at 14:43
-5

Everything.

What posts were opened, their contents, the discussion that occurred surrounding each, the answers/comments, results of in person or other survey research, all in a format anyone can directly access without knowledge of how to process a datadump.

I certainly understand keeping things related to moderation private, but for everything else... if your opinion can't stand up to scrutiny publicly it shouldn't be part of the private discussion where that can't occur.

Improve this answer
thirteen
  • fifteen
    Nah, I think people work better together when they can be direct with each other and aren't worried about outside constituencies. Look at any legislative body, the work gets done in private, the talk on the floor is all for an outside audience. For some of these working groups, it may also be necessary for company employees to talk about things that are business secret and may or may not actually happen. You put a gag on them if you make everything public because then everything they say needs to be approved by Marketing, Legal, etc.
    –  Bryan Krause
    May 15 at 19:21
  • one
    @BryanKrause that's exactly why it should be public. Hiding all this behind closed doors puts us, the public, at a position that we cannot act on. I certainly understand it'd be beneficial for the company's bottom line to be able to keep their plans for further abusing the public platform behind closed doors. If there's nothing to hide, don't hide it.
    –  Kevin B
    May 15 at 19:24
  • seven
    People may be less willing to ask for clarification that might make them come across as uninformed, less willing to bring up an idea that is unpopular or they perceive to be unpopular, less willing to change their mind, etc.
    –  Bryan Krause
    May 15 at 19:24
  • It should all be public so that people aren't willing to speak honestly and the company can't solicit feedback on things that haven't been made public yet? Because you think that will mean more things are public rather than even less request for community feedback?
    –  Bryan Krause
    May 15 at 19:26
  • one
    I'd rather requests for community feedback be requests for community feedback, not feedback from a small group of common users that the avg user can't be a part of.
    –  Kevin B
    May 15 at 19:26
  • eleven
    Your post is exactly why the company is going to working groups rather than the type of feedback you imagine, because when they ask the public "What do you want working groups to deliver? " they have to wade through posts that ultimately boil down to "NO WORKING GROUPS! " How is that helpful?
    –  Bryan Krause
    May 15 at 19:29
  • How is being picky about what is released to the public transparent? I most certainly am in favor of “no working groups” if they lead to smoke and mirrors and trust us bro development.
    –  Kevin B
    May 15 at 20:19
  • seven
    This post is a) Recognizing that a problem exists that some of the current working group discussions have been too opaque, and b) Asking the broader community for public comment/suggestion/advice on how to resolve that. I do not see how that corresponds to smoke and mirrors and trust us bro development, it looks like an honest effort against those things. If you had a problem with working group transparency before, this is an initiative worth supporting.
    –  Bryan Krause
    May 15 at 20:28
  • one
    I don't see a reason to support it if it isn't going to result in the transparency necessary.
    –  Kevin B
    May 15 at 20:38
  • Lets assume there's a complete datadump, even if redactions are not needed, it's still be a lot more information at once, potentially in a not easy to read format. If the final product dosen't reflect what was discussed by the working group, or there's dissenting voices, I feel like reflection in the summary of the working group's conclusions or responses to that might be sufficient
    –  Journeyman Geek
    May 16 at 0:56
  • two
    That's to say its less a matter of 'open-ness' vs secrecy than having actually actionable, concise information vs 'here's too much data to catch up, have fun'
    –  Journeyman Geek
    May 16 at 0:57
  • two
    If the medium was transparent to begin with, we wouldn't need to worry with dumps. It certainly wouldn't be the first, or only, case of working groups existing, functioning, and being entirely open to the public.
    –  Kevin B
    May 16 at 2:08
  • six
    As unpopular as this answer is... Having been in two working groups now, I do heavily agree that the work done in such groups could've (should've) been done publicly. With the exception already mentioned in this answer (moderator tools), nothing that was done needed to be hidden in private chatrooms or on private teams. I can understand the 'wish' for working groups as a way to focus and perhaps speed things up a bit, but the 'hidden' parts of it irk me too.
    –  Tinkeringbell Mod
    May 21 at 8:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .