Are comments obsolete in favor of Generative AI?

Question

With generative AI, you can just ask it to explain the code to you. In my experience, it is reliable and accurate - which cannot always be said about manual comments. I am specifically referring to comments on signatures - not necessarily inline comments, TODOs, or things of that nature.

It would save time (I think) by not writing and maintaining comments. A good generative AI for code isn't always free, but I suspect it will become built into IDEs sooner rather than later. Without comments on signatures, there is no IntelliSense/preview when you hover a function, but I suspect that will be added to IDEs soon.

My question: I make an effort to always comment the signature of anything public or unintuitive. But I am now considering stopping comments (except for exceptional cases). Is this a bad idea?

An example:

public bool IsEven(int number)
{
    return number % 2 == 0;
}

My comments:

/// <summary>
/// Checks whether the number is even
/// </summary>
/// <param name="number">The number to check</param>
/// <returns>Whether the number is even</returns>

Generative AI (JetBrains) comments:

Answer 1

"With generative AI, you can just ask it to explain the code to you"

And it will explain to me what the code does. Which I already know, because I can see the code myself with my own eyes. What I need comments to do is tell me things I need to know, that cannot be ascertained simply by reading the code; generative AI cannot do this any more than I can.

Answer 2

Three things:

1. If AI could sufficiently document code, it should be done once at the time the code is authored, not many times every time someone forgets what the code does. This is primarily an efficiency concern. But, it also allows the author an opportunity to validate or edit the comment to align with their own understanding.
2. The explanation generated in your example is harder to read than the actual code. But, this is the nature of LLM's: They do with many words what you can often do better with few (or none).
3. The best comments don't usually need to explain "what" code does, but "why" it is as it is. E.g., look at this:

constructor() {
  this._graphqlParse = someLib.parse;
}

It looks benign enough. We're just using the parse method from "someLib". What if we want to swap that out later? Is it OK to do that? If I don't have some in depth knowledge of "someLib", I have no reason to think I shouldn't swap it out with another library.

But, what if it had this comment on it?

constructor() {
  // someLib's parse method safely escapes template literal params for us
  // so we don't have to worry about injection attacks.
  this._graphqlParse = someLib.parse;
}

How I think about that tiny little assignment and library dependency is now completely different. I now know that if I go hunting for a faster library or a library that emits metrics or whatever, I need to find one that also sanitizes strings.

In the original, undocumented code, "what" the code does is obvious. "Why" I've done it the way I've done it was not. The "why" matters. And an LLM can only "guess" (same as you¹) at the original author's intentions.

---

^{1. An LLM doesn't really guess like a human does. It's basically a giant statistical auto-completion engine. But, I think you know what I mean here. The LLM doesn't "know" what the author's intentions are.}

Answer 3

That function is a great example of why not all functions need comments. Comments (and function docs) are to explain things that aren't immediately obvious. For example this (pretend) hand written documentation is useful:

/**
* Determines if invoices for this product requires the 
* <a href="http://mycompany.confluence.com/wingbat">wingbat</a> 
* payment pre processor. If not it can be sent RAW to accounts
*/
public boolean needsWingbatPreProcessor(ProductIdentifier productIdentifier) {
     // hats come from ACME hat and their payment provider can't handle raw invoices
     if(productIdentifier.hasHatIncluded()){
         return true; 
     }
     // turns out that BOT products have an undocumented bug that requires the preprocessor, See JIRA-89
     if(productIdentifier.getIdentifier().startsWith("BOT-"){
         return true; 
     }
     return false.
} 

For reference I gave an AI my example method and asked it to add javadocs and comments. This is what I got back:

/**
 * Determines if a wingbat pre-processor is needed based on the product identifier.
 * A wingbat pre-processor is needed if:
 * - The product identifier indicates a hat is included.
 * - The product identifier starts with "BOT-".
 *
 * @param productIdentifier The product identifier to check.
 * @return {@code true} if a wingbat pre-processor is needed, {@code false} otherwise.
 */
public boolean needsWingbatPreProcessor(ProductIdentifier productIdentifier) {
    // Check if the product identifier indicates a hat is included
    if (productIdentifier.hasHatIncluded()) {
        return true;
    }

    // Check if the product identifier starts with "BOT-"
    if (productIdentifier.getIdentifier().startsWith("BOT-")) {
        return true;
    }

    // If neither condition is met, return false
    return false;
}

The hand written docs are clearly more useful, giving a link to an article on what wingbat is (Which I'm pretending is an internal tool). And explaining the weird cases in this function. On the other hand the AI one just describes what it does

Answer 4

In many cases, the most useful comments are those which don't describe how something works, but those that describe what alternatives were considered, and whether they were found to be inferior or might be worth further exploration. Such information won't be present in code itself, and examination of the code won't be able to create it.

Generative AI is a nice parlor trick, and for some quick graphics tasks like thumbnail generation it may actually be useful, but the results shouldn't be trusted unless vetted. If one uses results without vetting, generative AI may yield some slight benefits if one doesn't mind the more significant losses when it turns out to be just plain wrong. If one vets the results sufficiently to avoid consequences for mistakes, the effort spent vetting could have been spent as well simply doing the original task.

Answer 5

I think the fundamental problem here is how many managers think of source code as an instruction manual on both the art of development and on how they do business.

So if they have a piece of code, however pathetically trivial, they follow the logic that there is, within the code, both the information necessary to produce a new developer capable of writing that code or similar code, and an explanation of how their business works (in a rich, meaningful way) and why the code was designed that way in the first place.

Their view on AI is also typically informed by the hype that it is somehow "intelligent" in the way that developers are reckoned to be, and that AI is as intelligent as the most intelligent developers out there.

It is from these ludicrous opinions that they proceed to think that AI can successfully explain a piece of code and answer arbitrary questions about it, in a way that an average developer cannot quickly do for themselves or in a way that eliminates the need for a skilled and familiar developer on their staff.

Now code commentary has only one function. It is to introduce design information, readable by the developer, which would not otherwise be recorded in the code at all (because it is not necessary for computer execution, it is only relevant to the developer engaged in design or re-design).

Sometimes commentary pulls together information that seems to be present latently in the code once you know to look for it, but the problem is that it is spread diffusely and with no visible sign of the connection - the purpose of the commentary is still to introduce new information that is not in the code itself.

Perhaps occasionally, it is to draw the attention of the developer to a specific pitfall which would otherwise get hidden amongst the thickets of code.

So understanding that the purpose of comments is to record things that the code otherwise doesn't record, there is absolutely no reason to think AI will make code commentary redundant, unless your commentary was already unnecessary.

Because an AI, in analysing the code, certainly cannot extract more from it than a skilled developer who is concentrating on reading it. And by waffling about the code, it certainly does not help to highlight pitfalls or important features either, but simply buries them in a different haystack.

Those who study software history will find that there is hype every few years about techniques which are supposedly "intelligent" and will eliminate the need for skilled developer labour. Yet there are more developers than ever before.

The curse of studying history, of course, is to be doomed to stand by and see the same mistakes made over.

Answer 6

Yes comments are obsolete. But not because of generative AI. Because you your function is just IsEven and that's enough.

On the question of AI, we should note its not explaining the function. It's searching millions of lines of code for similar functions and their comments and the documentation for the operations. finding the similarities and the generating similar looking text with your variable and function names substituted in.

try

public bool CheckEven(int number)
{
    return (number ++) % 2 != 0;
}

...In other words, the method returns true if number is odd and false if number is even....

https://deepai.org/chat

Answer 7

Comments on method signatures have been "obsolete" for decades provided the method signature conveys enough information for the human to understand what the method does at a high level. This isn't a new development because of AI or LLMs.

In September 2022, I would have stated that there is no point to add comments to a method signature if the method name, return type, parameter names, and parameter types are already meaningful. Fast-forward to today, and AI chat bots are a household name. My recommendation still stands. Comments in this narrow use case are unnecessary now, and they were unnecessary then. The method signature comments state the obvious. So obvious, in fact, that a machine learning algorithm can predict the most likely string of words that a human would use to describe it, because many humans have written similar code and comments that were used to train the model.

This is not a criticism of large language models. Instead, it is good to be mindful of the limitations of this technology. LLMs are a tool like any other. You need to understand how to use them for it to be effective. Rather than say "Boo AI! Yay humans!", here are some guidelines for determining when to offload this to an LLM:

• If the method signature is obvious at first glance to a human, it doesn't need comments, nor does the human need AI to explain anything. To be honest, this is old advice that predates LLMs by many, many years. I believe your code example falls into this category.

• If an LLM gives an uncommented method a good summary, then don't invest time in adding comments. This assumes other developers have access to this kind of tool. Beware that different LLMs can give different output. Do each of the developers working on this codebase have access to the same LLM? If not, consider commenting the method anyways.

• When a LLM cannot accurately describe the method, and neither can a human, then the human better write some comments to explain this to everyone. None of us know what the &$#@ this code is doing — carbon-based or silicon-based intelligence.

  • Once the human has written good comments, it would be nice to incorporate this information into the corpus of data used to train the LLM so it has a better chance of summarizing similar-looking code in the future.

• And finally, remember that the limitations of LLMs are evolving quickly. Better training data or algorithms can change the decision to offload some of this cognitive work to artificial intelligence.

Code comments are a tool, just like IDEs, and large language models. The onus is on the human to understand the use cases and limitations of each tool before using it. I think declaring code comments obsolete is too broad to properly capture the nuance of communication between engineers. More likely, current LLMs will augment our existing tools, rather than replacing them. Comments are not obsolete, but the use cases for comments might change because of the introduction of a new tool.

Answer 8

Comments should not explain what the code does. They should explain why the code exists. A.I. will never explain what the original coders intent was. We don’t need A.I. for this. We need coders who know what comments are for. Tell me why you wrote this. Tell me why I should ever call it. How it works is really not the business of comments.

I should be able to refactor this code so that it uses a bit mask to do the same thing without touching a comment. If refactoring breaks the comments then those are bad comments.

Answer 9

There are already plenty answers that address the core issue of whether an AI can generate a comment well enough without further supervision and whether the "reader-side" generation strategy rather than an "author-side" supervised variant to use AI's to generate comments is a good one.

However, there are also a few additional indirect limitations to where this approach can work based on outside requirements and economic factors.

For many projects applying an AI to the source code is simply not an option. Because:

• It is or will be expensive (using the most powerful AI model); it currently is expensive for the AI model providers, they just hope for later return of investment and it is questionable whether AI tailored to software generation will be able to finance itself with advertisement like search engines
• It cannot easily be self hosted (simple models yes, the really powerful ones are proprietary along with some of the tooling)
• Code might need to be editable offline: Not every place has internet connection, not every system is connected to the internet and AI systems are typically web based; even if just a temporary issue, it slows down the development flow if a comment/functionality description is not available when you need it
• If the model is not hosted by the company, they might - e.g. for security or privacy reasons - not want to apply the AI to all their data including their issue tracking systems as both the actual code as well as the issues might be very important strategic assets, the company might want to protect as best as it can; external service providers, potentially even hosted in foreign countries are not exactly trustworthy enough for many companies/projects; Tickets could also have customer data that should not leave the company premises.
• Many accessible AI models do not learn in general from your data, they just include it in a particular query evaluation. The ones that keep learning are more prone to being misled, more expensive and would either mix your data with external data (if an external service) or be even more expensive to run ^^

Perhaps AI will become massively cheaper to run and everyone has it running locally on their machine. That would alleviate most of the external factors mentioned here, but currently that is not in sight. For example, this article from Business insiders stipulates that 1) Tech companies plan on spending over $1 trillion on artificial intelligence. 2) The return on investment may take a long time and be disappointing 3) some experts said AI might not perform well enough to justify its exorbitant cost

And on the practical side, AI surely might change the way we program, but either it gets so good it can program on its own or it needs human supervision - which then also holds for writing comments - and will remain a tool in writing code and comments by humans for humans.

Answer 10

The documentation of a function/method/class/interface should not be what it does, but its contract: what it commits to do, and what it doesn't commit to do, not how it does it. You have the source code for that!

AI can tell you what the function does and how it does it. Only the people who designed it can tell you what they promise it will do.

Remember that:

• The implementation may change. What works today in cases not in the contract may not work in the future.
• Most functions are quite a bit more complex than checking whether an integer is even, and will most often call other functions, both in the same package and in other ones, as well as system functions. Some of those may in turn vary in what they do (in time, or on different platforms). Do you want a description that tells you "this functions calls this if X and that if Y"? Or do you want it to tell you what the overall result is?

Answer 11

Maybe, soon an AI will write your comments. And yet another AI will read the generated comments to digest it for someone else. I bet that if the generated comments are as fluffy as in your example, the latter will become increasingly popular.

Meanwhile, we all know since clean code that comments are not supposed to waste our time for obvious explanations (here, 11 lines of blabla to explain 1 line of code). Focus should be on what's not so easy to grasp in the code, and in that case, comments are an investment for the future with little overhead. At least you're sure that the vital minimum is there.

Answer 12

It is not a bad idea, if your code is self-explanatory.

Comments are a nice tool to express concepts that have meanings in a higher level of abstraction respect the one of source code. Pointless if they just rephrase what code already "say".

But... AI support is welcomed when the coder still has to be trained and those inferences that are obvious for the seasoned developer, would require efforts for her (the apprentice) to be done (at the cost of disturbing her flow).

Instead: even the apprentice can benefit from reading about things (in the comments) that ... well: she does not fully understand, but she will do, one day.

A common behavior is limiting the context (meaning, subject) of what can/should be said in comments. Usually we talk about the software itself, not about the design, not about the method with which we choose one design instead another one.

This behavior is common when/where the same source code must be shared among different developers and other roles. In this case, the source is not a good place in which to talk about design methods.

When the source is handled by just one developer things change and comments become also a reasoning (even thought informal) tool.

It's not said that comments have to express just informal expressions. They can be also formal.