OpenAI will also surface validated technical knowledge from Stack Overflow directly into ChatGPT, giving users easy access to trusted, attributed, accurate, and highly technical knowledge and code backed by the millions of developers that have contributed to the Stack Overflow platform for 15 years.

One of the major selling points of "OverflowAI" is (was?) attribution; most LLMs don't provide any sort of attribution for where they're getting their information from. Directly feeding Stack Exchange content into ChatGPT or other OpenAI products sounds, to me, like that attribution is not going to be retained. It says "attributed", but doesn't give any details: Will it cite Stack Overflow in general? The specific post? The specific user? Will this abide by the CC by-SA license that posts are under?

Will the users who actually spent time writing and contributing their content be attributed in any way when their content is fed into OpenAI's products?

I'm altering the deal.

Pray I don't alter it any further.

― Darth Vader on Bespin https://www.youtube.com/watch?v=jsW9MlYu31g

Most replies focus on certain process and legal aspects. What I want to bring up is the emotional aspects. Irrespective of whether the license agreement allows this to be done with our content or not, the whole thing is triggering an emotional response with me, and it's not a good one.

I feel violated, cheated upon, betrayed, and exploited.

I believe a lot of others feel the same. If you feel the same, upvote this.

Attribution is one aspect of it, but not the only aspect. It is the entire way how the community works and how we, the humans, created content on this platform to make the world a little bit better.

This whole thing feels ethically wrong, and emotionally damaging. Humans are meant to exploit machines, not the other way round. Exploiting us, who helped make the world a little bit better, in this way, is a turning point. It makes the world for us worse instead of better.

People already start contemplating actions against this, like purging their questions, answers, comments, and even deleting their accounts. I was contemplating deleting my account over this. I still am.

Congratulations, Stack Exchange.

Update: I know of a user whose account has been suspended for 7 days because they started deleting their stuff.

I know that deleting my account won't delete my content. But it will at least prevent me from giving more content in the future.

Consider this, where do you think Stack Overflow is in its journey? Has most of its content already been created? Or will most of its content still be created in the future?

Update 2: I know that there is an argument to be made that we've basically agreed to this right from the start. No, we haven't. We have agreed to whatever type of processing a reasonable person could expect to happen from the license conditions at the time of "signing" the agreement with the technology available at the time. Training LLMs and such was neither available nor reasonably predictable at the time of "signing" the agreement.

As someone rightly pointed out in the comments, this is like Darth Vader on Bespin saying "I'm altering the deal. Pray I don't alter it any further."

I know - viscerally - that y'all are shorthanded. I also know that things sometimes get dropped in your laps unexpectedly with little time to prepare, so I'm hoping that this was merely an oversight... but I'd have to echo Mithical's point, particularly as link-only answers are a core concern on the platform.

Y'all have, unfortunately, posted a link-only question, which relies on external links for all of the important content. This means you're expecting people to visit two different links and read long-ish posts that aren't written for this audience to get all of the information they need to understand and respond to this post.

I think we're all well aware that AI is an extremely contentious subject on the network and the core community have very strong concerns about it. I appreciate that you link to the February blog post about socially responsible AI but, again, summarizing the core ideas from that post in this context rather than linking to it alone would have given people much more community-centric context about the content in those two sources. Particularly since the MSE announcement of that post was similarly poorly received and few questions have been answered.

Right now I'm seeing a lot of concern about whether this means that y'all are more likely to override the rules about posting AI-generated content on sites or changing your thoughts about attribution. While these points are addressed in the Commitment blog post, I'm going to be honest - that post is written in a very indirect and unclear way, so I'd really appreciate a synopsis in simpler words. For example:

In addition, community answers should be derived from quality, accurate, sourced data.

On the surface, this is a very nice sentiment - one that we can all get behind. Unfortunately, it doesn't actually say that AI-generated answers can't be posted by community members, reviewed or not - if people assume AI generated content is "quality, accurate, sourced data" (it's not), then they will think it's fine. It's also unclear what "community answers" means - two very different interpretations are:

• Answers written by community members
• Answers on a Stack Exchange community site

The former leaves huge openings for AI-generated answers, since they're not created by community members and the prior (much clearer) statement about questions uses "community" to mean "community member" as opposed to "assisted and curated by AI", further leading to the former interpretation.

The full statement about questions is:

questions on Stack Overflow (whether by a community member or assisted and curated by AI) are posted only after human review.

Is possible to say something similarly definite about answers to indicate that the company commits to ensure all answers must at minimum have human review to ensure accuracy while ensuring that sites' communities have oversight to further restrict using these tools to answer questions?

It'd be really great if, now that the post is out and you are getting feedback of confusion and questions from community, if you could edit a community-focused explanation/overview of the two posts into the question so that a reader doesn't have to view the full press releases to understand what's happening.

To be clear, this isn't about specifics - it's about the most general big-picture things. You're building on a foundation that's not clear to the users, even if it's obvious to you. Perhaps you find what's been written in these posts to be crystal clear. You know what the company intends and that the interest in the community connection is genuine. The community doesn't know that.

You can dismiss my statements as me looking for the negative interpretation in everything. I've been there. I've struggled with people trying too hard to read into and over-analyze every word I've written and finding cracks when I'm trying to be open and honest. I know it's difficult. But that's why I spent time clarifying things rather than just brushing those concerns aside.

If you can't clearly and succinctly say things like, "We will not create or incorporate tools to automatically post answers using AI to the sites." and "We will discourage users from manually posting or converting AI-generated answers without human verification and review to the sites." - over and over until people actually start believing it, people will continue to look for indications that you're stating things in ways that allow you to point at your prior statements and say that you're not "technically" violating them.

No, please don't do this.

ChatGPT and SO are two mutually exclusive things.

If we bring ChatGPT to SO then what will be the difference between SO and any other GenAI site answering questions?

SO has maintained a trust of "quality content" of over 15 years with its dedicated users, moderators, contributors, etc.

Please do not break this trust with ChatGPT.

Some questions that come to mind:

• Is it going to be possible for individual users to opt out of having answers touched by OverflowAPI?

  • Follow up: can sites opt out of being included in OverflowAPI?
  • Related: What sites are included/excluded from OverflowAPI?

• What applications do y'all plan to use OpenAI's models for?

• Is SE receiving financial compensation for this partnership or is it purely software exchanging? If yes, how much?

  • Follow up: if yes, what'll the money be used for? Funding more AI things or giving resources to fixing/improving the SE platform without AI?

• How does OverflowAPI actually work?

• Is there a plan for if users start deleting answers en masse, editing/writing-in terminology to try and throw off any LLMs, or otherwise try to disrupt the quality of answers in OverflowAPI?

How does this impact the previously-announced partnership with Google? It seems like OpenAI and Google are both getting similar things out of their respective relationships, and that is access to the use of Network data for training. OpenAI also recently announced an entry into search. Can we expect both OpenAI's services as well as Google Cloud AI technologies to be integrated into the platform, since both announcements appear to indicate this?

There was a past blog post on "socially responsible AI". One of the key elements was "attribution is non-negotiable". OpenAI, historically, has done a poor job of attributing parts of a response to the content that the response was based on. Google's Gemini has some built-in capabilities for citing content with related web content, but there are still no guarantees that it relates the statement to the correct source material or that attribution meets the CC BY-SA requirements for attribution. Given the company's stance on the need for attribution and OpenAI's past problems in that space, why this partnership? When can we expect to see more details about how OpenAI is solving the attribution problem?

Are we going to continue to see AI services blocked if they do not sign a partnership, especially with no public announcement? This also includes the mentioned-but-not-detailed "guardrails" on the API, SEDE, and data dumps.

I've been talking to a few people/reading a few posts offsite and I think there's a great deal of confusion over it.

There are a couple of key issues here with messaging:

• the moderator teams were not given any prior notice of any of this, and we're likely the ones dealing with the fallout of people getting mad about it

• a lot of people think this is "getting genAI on the main sites 2.0", the press release links to teams but lacks clarity on its own merits.

• the press releases don't actually say anything about how OpenAI, Google and others intend to, or could use the API.

• to get any clarity over what the StackOverflowAI API is I'd need to waste someone's time registering as a potential end user. While the end points might be restricted, there might be value in having the API documentation open, and the actual API restricted.

• there's a bit of offsite soapboxing and people wanting to delete all their content which could/should have been avoided.

Essentially, we've got a ton of fluff and no clarity over what this actually means for us - nor do we have the tools to communicate with our communities over concerns they have. Considering the rocky road that we've had on the topic, I'm surprised no one thought about how to message this to the community.

As excited as y'all are about new partnerships, I feel like old ones got missed out - and an opportunity for open communication.

As a moderator, I don't know what to tell my community about this. Not in terms of "Oh god, its so bad", more of "I've got nothing at all." This isn't a good state of things to have.

Most people are focused on attribution (and rightfully so), but it seems that not much attention is being paid to the share alike part of the CC license. In AI contexts, copyright law is still being tested in court and many things are uncertain. There is a very real risk that training an AI on this site's data will not necessarily be considered "fair use" (it fails the "serves as a substitute for the original" test, among other things), which means there's a risk that the trained model will be considered a derivative work and thus required to carry a license similar to CC-BY-SA 4.0. When you make agreements with these AI companies, are you including provisions and processes to ensure their AI models will be properly open-sourced should these sorts of court decisions happen?

In which case, why should any of us bother to contribute our time, energy and expertise to enrich you any further?

For that matter, why honestly did we bother in the first place, to provide unpaid labour for you, without which this website has no value whatsoever? It's made worse when you're going to just feed that effort into the digital equivalent of a compost heap, for it to be decomposed into excrement, but really why bother with this place at all instead of, say, a not-for-profit?

On an unrelated note, I'm not contributing anything further to this place ever again, and putting stuff on https://software.codidact.com/ instead. Which just so happens to be basically the same as this site only run by a not-for-profit!

Licensing....

When a user contributes content (ie, Questions & Answers; ie, data that would be shared with LLMs), the terms of service grant two licenses.

1. User content "is perpetually and irrevocably licensed to Stack Overflow on a worldwide, royalty-free, non-exclusive basis pursuant to Creative Commons licensing terms (CC BY-SA 4.0)"
2. "you grant Stack Overflow the perpetual and irrevocable right and license to access, use, process, copy, distribute, export, display and to commercially exploit such Subscriber Content, even if such Subscriber Content has been contributed and subsequently removed by you"

I often simplify the lawyer version in the TOC by saying that by posting on the site, users grant licenses:

1. To anyone in the world: under CC-BY-SA 4.0.
2. To Stack Overflow (the company): to use however they want, unrestricted by the CC-BY-SA license.

Stack Exchange Inc can do whatever they want with Q&A data.

I know many users are asking about attribution, and this post maintains that attribution is a non-negotiable requirement of the partnership.

However, I want to point out that there is no requirement that Stack Overflow provide the data to OpenAI with the CC-BY-SA license intact, because Stack Exchange, Inc is not bound to the CC-BY-SA license. Quoting (again) from the Terms of service:

Stack Overflow [has] the perpetual and irrevocable right and license to ... distribute, export, display and to commercially exploit [Q&A] Content"

How will the company be licensing the data to OpenAI and other future customers who wish to use it for LLM training data?

Stack Overflow can ensure that data provided to OpenAI and other LLM creators is licensed under the CC-BY-SA license, in order to ensure those companies are required to attribute it accordingly.

Attaching the CC-BY-SA license to data provided to OpenAI would additionally bind OpenAI to the "Share alike" clause of that license--effectively requiring that in addition to attributing specific ChatGPT responses to Stack Overflow, it also requires those ChatGPT responses to carry the CC-BY-SA license.

It seems highly unlikely to me that OpenAI would be willing to have certain GenAI output (whether ChatGPT, or coding suggestions or whatever) to be covered under CC-BY-SA. Therefore, I am assuming that Stack Overflow will be providing LLM training data under a product that uses a license different than CC-BY-SA. Stack Overflow users have provided the company with "perpetual and irrevocable right and license to...distribute..and commercially exploit" Q&A data, so it would be well within the company's rights to make the new product (LLM training data) use a completely different license.

This joint press release presumably means that the ink on the contract is dry, and thus the terms under which the data will be shared has been determined. I think many of the questions on this meta post would be answered simply by letting us, the community, know how Stack Overflow is licensing the data to OpenAI.

When did you consult the users and moderators of this website, who have created all of its value? Why do we learn of this decision via a press release? After the whole episode about the new AI policy and the moderator's strike, haven't you learned anything? I don't think your attitude to us, the creators of this site, has changed at all. I don't think it ever will.

I'm just going to delete/deface my answers one by one. I don't care if this is against your silly policies, because as this announcement shows, your policies can change at a whim without prior consultation. You don't care about us, I don't care about you.

How is the socially responsible aspect taking into consideration the CO₂ emissions from LLMs?

I saw no mention of it in your linked article "Defining socially responsible AI: How we select partners."

I'm just frustrated with the continued poor communication.

GenAI is arguably the most heated discussion in tech there is right now; it's polarizing, confusing, and scary to a lot of people, especially surrounding attribution and apparent content theft (whether legal or simply perceived).

I find it unbelievably frustrating that apparently nothing has been learned by Stack Exchange on the comms front since the first round of this with the Google AI partnership, which was also poorly received (scored +35/-455 at time of writing).

Just like that one, all we've been presented here is a link to the blog with effectively zero meaningful tailored messaging; this post could have been a completely automated notification of the blog post and it would have had exactly the same amount of actual content.

It just feels like the company keeps shooting itself in the foot here, and then is shocked at how much it hurts when it happens again in exactly the same way the next time.

We, your community, care about this platform. We want to stay updated with partnerships. We even want you to get paid and keep the lights on!

But that also comes with wanting to know more than what the public press release full of buzzwords provides, to have some insight into how the partnership might practically affect us and the platform, and maybe most importantly, wanting assurances about issues important to us (like attribution), which should have been stressed in the original post from the beginning.

I strongly disagree with their approach, but the fact that there are folks already going nuclear from just this announcement illustrates just how badly the company has botched the messaging on this one. The first shot at explaining this in terms the community is willing to hear has been missed by a mile. Again.

For reference, the correct thing to do was to defend us from AI, to work to block and sue OpenAI and companies like them and cause them as much pain and frustration and headache and misery as legally possible. You would have been heroes even if you failed.

Instead, you sold out everyone for a very short-term dollar.

It makes me sad to say that Stack Exchange is now in free-fall, being gutted and exploited and sold for every last buck you can scrape together before its dying gasp.

So when is the class action starting?

And let's appreciate the ironic juxtaposition:

My answers have been read by over 151,000 people on the physics Stack Exchange site (PSE). That makes me the top 3% of users. I wrote those answers to help the community. I am not okay with my answers being used for the benefit and profit of OpenAI, a company I do not like.

I was banned yesterday from PSE for deleting my answers, which were edited back to my former writing.

For some context, I am of the belief that the goal to advance AI to an intelligence greater than humans doesn't lead us to a brighter future. We can agree that that is OpenAI's stated goal, even if we disagree how realistic it is or what the timeline might be. So, I don't want to contribute to their effort at all. That is the practical purpose behind editing away my answers. I believe this partnership may be a result of the added profit motive after purchase of SE by investment firm Prosus.

Please, I love using SE because SE is not a forum, and I’m using Mathematics Stack Exchange and English Language Learners Stack Exchange as I can get more accurate answers at those sites.

But why do I and other users want to contribute anything to this site if ChatGPT (OpenAI) has been used in this site?

The community needs to know that SE is not the only place for Q&A, there are other places for the Q&A sections.

Take an example, if I don’t use Mathematics Stack Exchange, I can go to AOPS to ask my mathematics questions too.

So SE definitely is not the only choice for Q&A, and more sites/forums will be developed in the future.

I really hope that the community thinks wisely and does not share our content with OpenAI’s products.

Note: Quora actually also introduced ChatGpt, see here. That’s the reason why I don’t like using Quora, as I’m afraid of getting AI-generated answers or bad-quality answers.

Reply @Some Guy,

I know this is just a strawman argument and in my answer, I don’t mean that ChatGpt has been used in creating posts. But there are many things we are not understand in this announcement.

@Rosie just answered the attribution update in this OverflowAPI, but there are so many things like license and the usage of OverflowAPI are just questions.

As we know, ChatGPT and SE are two mutually exclusive things. So, I don’t believe it is a good idea such that OpenAI is partnership with SE.

Like what I write in @Iyxal’s answer,

The most important part is the last question, if OverflowAPI can disrupt the answer quality, I definitely will left this rubbish site.

So I will wait for the update for OpenAI first.

Wow, you guys made me log in again.

(And that's not a good thing)

I, personally, am not anti-AI but this partnership was not discussed with us, your users. You previously banned ChatGPT's webcrawler but have since removed it and have now granted access to the Overflow API directly.

I totally understand people's objection to this change in policy and their attempts to remove their questions and answers in protest, even if I won't be doing so. Afterall, that lack of transparency is why I logged out in early 2020 and haven't been back since. Well ok, I did have to return and log in so I could download my Jobs profile when that shut down.

So long and thanks for all the fish.

How can you be so sure that our content will be properly attributed on the ChatGPT end? Even if they intend to, intentions can fail. ChatGPT is intended to reject certain queries, but as we know we can 'jailbreak' ChatGPT and receive answers to queries intended to be blocked. So no 'guarantees' can be made in terms of ChatGPT responses as far as I can tell.

You assert that attribution will be given... but is there any concrete evidence to back that up, given ChatGPT has a poor track record of giving correct attribution, let alone attribution at all?

I would like to allude to How not to be a spammer of all places:

Don't tell – show! The best way to avoid being seen as a snake-oil salesman is to demonstrate a solution, rather than simply asserting that the problem can be solved.

So can you show/prove to us that attribution will be given, rather than just asserting so?

And what recourse do we have if our content is being reproduced by ChatGPT, and the attribution we were promised was not given?

So let's say I ask OpenAI a question that I know the answer to and have actually answered on this site. OpenAI spits out a response, and it clearly starts off with my correct answer, but then hallucinates up some random details and gets it all horribly wrong.

Can I flag that up with OpenAI? Will Stack Exchange see the flag? Should I flag it up on the relevant Meta here instead as a nudge for SE to likewise nudge OpenAI to fix the bug?

Since a number of users seem to be under the impression that they can prevent their posts from being scraped by OpenAI by deleting or vandalising them, it feels pertinent to ask: how much of Stack Overflow has OpenAI already scraped? Is it possible that my posts have already been fed into ChatGPT, and therefore, deleting them (hypothetically speaking) would be a waste of time?

For that matter, what happens if a Stack Overflow post is fed into ChatGPT and later deleted from SO (because, say, it's spam, or plagiarised, or both)? Would it be removed from the LLM as well? If not, I can easily see a vicious cycle where users post ChatGPT answers on SO, those answers get fed back into ChatGPT, and the model slowly degrades instead of improving.

As a final point, I'm not sure whether this has already been addressed, but is this only going to affect Stack Overflow, or are OpenAI going to scrape the entire network? As mentioned above, I've seen several users deleting or vandalising posts across the entire network, not just SO, in an attempt to prevent them from being scraped.

There are so few details that I can't tell how well this has been thought through, but my immediate instinct is "not very well".

Just comparing how the community reacted to banning GPT vs Partnership with OpenAI shows how they feel about this relationship and these two different products!

I’m not going to talk any further about attribution but the future of this community really concerns me! It seems to me that generating anything (even a simple reworded message), for any purpose (from a simple search aid to a complete answer), kills TRUST in SO which took years to achieve and people tend not to contribute when there is lack of trust.

_{Fun fact: there is little banner above this text that reminds me of something :)}

"Do the people who actually contributed the answers get anything out of this deal (attribution? access to the trained model? part of the profit?) – Aykhan Hagverdili"

Saw this comment; and the one thing many would like to know

This is incredibly sad. I'm not a contributor here but I've learned SO much from this resource and value it highly. I'm putting a lot of effort right now into trying to protect original content from being scavenged and I thought Stack Exchange would be against this for some reason. Apart from the obvious issue of people's original thoughts and expertise being used to 'train' LLMs, has anyone noticed how bland and soulless A.I. content is? Even when the information is technically correct and not mangled, the tone is so offputting - 'it's important to understand...blah blah blah' - maybe we should have two internets, one where all the bots talk to each other and everything is an algorithm, one for humans?

Regarding attribution, the Stack Overflow team used to be very clear about the value of it:

https://stackoverflow.blog/2010/08/11/defending-attribution-required/

Please help us defend your right to have your name and source attached to the content you've so generously contributed to our sites. We will absolutely do our part, but many hands make light work

Does Stack Overflow still care about attribution? Will my many contributions to Stack Overflow over the years be correctly attributed, when used by AI products trained on those contributions?

1. This violates the user agreement as well as the content license, and if this sees the light of day, you will be on the receiving end of a very large class action from a very informed group of people.

2. If this goes forward, giving out content to be consumed by GenAI is to admit that you do not own the content, and thus there is no enforceability for protecting SO Network content from simply being copy pasted to a separate platform without attribution.

Proper attribution is defined in the license agreement, and that is what must be provided. There needs to be direct citation for works used; that is the license. Without direct citation, it is plagiarism unless the entirety of the work is remixed solely from one source and the source is referenced (which isn't the case here). AI remixes from multiple sources, and therefore it must explicitly cite those sources discretely in order to abide by the license.

Curious about the terms that apply to your content? Download it! Find the date your post was created, look up the ToS from Stack Overflow itself here: https://web.archive.org/web/20150701000000*/https://stackexchange.com/legal/terms-of-service .

Update after @Rosie answered:

I still don’t understand why Stack Exchange are in a partnership with OpenAI. I felt like I’m been cheated. But hopefully, I took more time in editing posts than asking or answering questions. But what can we do with those users who were contributing (asking or answering) very much in Stack Exchange?

I’m actually not familiar with the Term of Service in Stack Exchange, but I felt hopeless when I saw this in the comments and a question. It looks like the users who contributed much can’t do anything, delete their accounts or posts are useless!!!

the Terms of service give Stack Overflow license to use your post content, even if they are deleted. -AMtwo

note that deleting your account does not result in deletion of your content. -super-starball-ultra

You can also see this question for some information: How can a user unhappy with the LLM partnerships protest constructively (and ideally- effectively)?

I feel like now Stack Exchange has solely started disrespecting its users’ values and efforts.

I hope the management can be better and go into partnerships with those companies to get some benefits. And these benefits don’t last long. Who will believe Stack Exchange, from now on?

For me, I won’t believe Stack Exchange in its management.

I’m not understanding what this stupid announcement is for, but the thing that I confirmed is that Stack Exchange is very bad currently.

I despair ... SO has been on strike over ChatGPT use and the company just go for more!!!