11
\$\begingroup\$

Assuming questions with less popular languages take longer to be answered, I'd assume a decent wait time on reviews for them. What surprised me though, was to see that two up-voted questions I posted in JavaScript weren't answered, one over a week ago and the other 36 hours ago.

8 months ago, Stack Exchange Management posted a question, How is Code Review Doing Right Now? The answer at that time was: Not Great. And one of the primary issues listed was that it takes a long time here for questions to be answered.

This site is an outstanding learning tool for both reviewers and askers, so I'd really like to keep it around for the future. Perhaps we should take a look at where our strengths and weaknesses lie with this issue.


So, I'm curious to know:

In 2014, how quickly, on average, are questions of popular languages answered on Code Review?

  • Java
  • C#
  • C++
  • JavaScript
  • PHP
  • Ruby
  • Python

By answered, I mean having an accepted up-voted answer.


I've seen people query this sort of thing in Stack Exchange, but I wouldn't have the slightest idea how to do it.

\$\endgroup\$
7
  • 1
    \$\begingroup\$ If you ask a question in meta about questions on the main site, the main questions get answered faster ... ;-) \$\endgroup\$
    – rolfl
    Commented Jul 20, 2014 at 17:19
  • 2
    \$\begingroup\$ I still wouldn't use acceptance here. Code Review isn't like other sites in which the correct answer will be accepted. On here, only the "most helpful" answer (deemed by the OP) may get accepted, but it may not always happen. \$\endgroup\$
    – Jamal
    Commented Jul 20, 2014 at 17:20
  • \$\begingroup\$ @Jamal good point. Should the criteria be when an answer is given at all? Up-voted? I'm unsure how advanced the queries can be \$\endgroup\$
    – user73428
    Commented Jul 20, 2014 at 17:22
  • 1
    \$\begingroup\$ @jt0dd: Upvoted should be good (you shouldn't count a non-answer, which may come first). \$\endgroup\$
    – Jamal
    Commented Jul 20, 2014 at 17:23
  • 1
    \$\begingroup\$ Javascript is one of those Huge Question Fields, by that I mean that it's one of the biggest tags on SO and CR people post JavaScript questions like there are grains of sand on the beach. There is one user that I know of that strives to keep the Javascript Zombies at bay and I think he is taking a well deserved break, and I don't know enough of the more advanced JavaScript stuff to help him out most times. this isn't a Job (for most people) so be patient, someone will review your code. \$\endgroup\$
    – Malachi Mod
    Commented Jul 21, 2014 at 16:39
  • \$\begingroup\$ Maybe its time I started to broaden my review skills beyond C++ (starting to get OK at Javascript (not expert but OK)). But can't we slap together one of those fancy data graphs for this question. \$\endgroup\$ Commented Jul 30, 2014 at 19:08
  • \$\begingroup\$ I know that this is an old question, but I can tell from experience that questions in PHP almost never have an answer. I've made 3 questions. I've even given 100 reputation to one of the questions and no one answered. The only thing I get is a few upvotes. The only answer I got was pure luck, and I had handed 50 of reputation! Questions about PHP aren't worth it here. This is only my opinion and based on my (small) experience. \$\endgroup\$ Commented Jan 16, 2015 at 17:30

5 Answers 5

7
\$\begingroup\$

There is a fair amount that can be done with SEDE.

Here's a query I put together a while ago. It will hint at what's happening in Code Review when it comes to 'zombie' questions.

For the purpose of this query: Answer Delay

Let's call a question that is unanswered after 48 hours, a zombie. A question that is answered in less than 48 hours is a 'quick' answer. The assumption here is that zombies are 'hard' and will have to wait for the right person to come along....

So, how fast are quick answers given?

Wel, for the past number of weeks, of all the questions answered with in 48 hours, the average answer came at about 5 hours. This indicates a significant skew.

It would be interesting to calculate it as a histogram.....

Anyway, this does not answer your question directly.... but, if you combine it with this graph: Tagged Zombies you will get an idea of which tags are actively having their zombies culled.

\$\endgroup\$
10
\$\begingroup\$

This started as a comment to rolfl's answer in response to jt0dd's comment:

Your question from 'over a week ago' (June 30'th) was answered. Why you say it was not, is a mystery. That answer begins with: This answer is not a full review, but just a minor tweak.

But the comment was quickly growing way too long, so I'm going to post a full answer instead. (And this is kind of a strong reiteration of Simon's comment.


You cannot simply recommend that the site would benefit from shorter answers and simultaneously complain that a question you asked and received an answer to went unanswered because the short answer was also incomplete.

On StackOverflow, in the regard to answer length, there are a handful of scenarios that most questions fall into.

  • The question is a small contained problem with a short easy answer.
  • The question is a huge code dump, but by review of someone who isn't an idiot, the problem is tiny, so again, a short easy answer.

Those two scenarios are sort of the same. The only difference is in the asker's ability to narrow down a short, self-contained example.

The third scenario is when the asker has no clue how to do something and is asking how to do it. Any good answer would be unnecessarily long... and the question gets closed as too broad!


Sometimes questions might have available quick answers, and maybe we can do a better job of offering the quick answer in some cases here on CodeReview.

But a quick answer on CodeReview is still going to be as long or longer than an average length StackOverflow answer.

You are right. If the user has a problem with, for example, the way they name variables, then we can simply point that out, give an example or two, and move on. We don't need to run through every spot in their code where an they've got this same problem. But a good answer (whether here, StackOverflow, or anywhere else) ought to come with a WHY to defend the answer.


Realistically, the argument you actually seem to be making is that we should probably be more aggressively closing questions as being Too Broad. We should probably be encouraging askers to narrow the scope of their questions down to a single class as a starting point. Realistically, we probably shouldn't be handling entire projects, because that's a question about the architecture of an entire project, and perhaps is better suited for Programmers, where they'll ignore any of the specifics we'd get into here, but focus instead on the big picture and how the project works as a whole.

BUT EVEN AT THAT, even if we force askers to narrow their question down to a single class and close anything else as being too broad, it still doesn't guarantee that all questions can be answered in 2-3 paragraphs and be a complete review.

For starters, sometimes the problem is simply that the class is too broad and tries to do too much, and that's a fine review, but if you don't address any of the specific problems within the individual methods, or method naming, etc., then your review remains incomplete. Perhaps leave it at that "This class is to broad and tries to do too much. I'd recommend splitting it into Class A, Class B, and Class C." and hope the user returns with three new questions, one for each of the new classes he split the original class into.

But even still, we'll still be stuck with situations where there's just way too much to comment on to provide both a quick and complete answer to every question.

Consider this question.

My first answer, which takes up almost my entire screen top-to-bottom and it only addresses 4 lines from the asker's question. Now, assuming those 4 lines were the only 4 lines he'd posted, my answer would be a complete answer. Or if the 4 lines were the only problem in his code, my answer would be a complete answer. But neither is the case.

As such... I posted a second answer. This answer is about three times as long as my first answer. It does address significantly more lines of code, but it's still contained to a single method within the code posted in the answer.

In this case however... I don't even feel like my answer even completely address the method in question. There are readability issues to address and variable naming issues, etc. There are other things just within this method I could've commented on.

And even with these two, quite large answers, I still feel like I only addressed a small portion of the problems with the code in the question. Neither of my answers is complete.


I feel like we probably could do better to be slightly stricter on the scope of the questions posted. For example, this question is definitely way too broad and its scope should've definitely been narrowed. It really doesn't serve much purpose.

But here is an example of a question that started out with a very broad scope but got narrowed down to just a single class. My answer was posted within 6 hours of the question being re-opened (it had been closed as too broad), but still, it's a very large answer.

Although in this case, I feel my answer is a pretty complete to the question.


This isn't StackOverflow. Good answers are generally going to be quite large. And the worse the original code is, the larger a good/complete answer will be. Although I think it's probably best if the question instead has several good answers that each address a specific topic. The result will be that no single answer is complete, but every problem is addressed between all the answers.

But the other problem is, if you ask a question on StackOverflow, it's either answerable or off-topic. The same can't definitely be said about CodeReview. I can post code that I feel may have room for improvement, but "room for improvement" is kind of subjective.

\$\endgroup\$
6
  • 4
    \$\begingroup\$ For an answer that started as a comment... whoa this is big... \$\endgroup\$
    – nhgrif
    Commented Jul 21, 2014 at 0:33
  • \$\begingroup\$ I posted a long answer just now (my first); My meaning was not that long answers were not often necessary, but rather that if you have little to say about the code, a thumbs up and few sentences can be the difference between the asker believing that his question is ignored and being happy that he did a pretty good job. Notice in my answer, although it was long, I didn't rewrite much of his code at all. \$\endgroup\$
    – user73428
    Commented Jul 21, 2014 at 0:38
  • 1
    \$\begingroup\$ And how about we learn to review in stages? "This has structural issues. Fix them as follows. [...] There are more formatting and optimization issue that I'd like to touch on, but this is enough for now." And leave it to the OP to post his code again when the structure is right. \$\endgroup\$
    – user73428
    Commented Jul 21, 2014 at 0:50
  • \$\begingroup\$ "I think it's probably best if the question instead has several good answers that each address a specific topic." - We need more of that, rather than trying to write absolutely complete reviews that turn into walls of text that very few people read. \$\endgroup\$
    – RubberDuck
    Commented Jul 21, 2014 at 2:01
  • \$\begingroup\$ @jt0dd The problem is that a) that's still an "incomplete" review, and b) what one person sees as a structural issue another person may not. I might write a review about the massive structural issue and decide to wait until the repost for the other issues, but that doesn't prevent other answerers from posting the monster answers that address all of the individual issues. \$\endgroup\$
    – nhgrif
    Commented Jul 21, 2014 at 11:18
  • \$\begingroup\$ You can also take into account that it's likely that if you do touch one one major part, with a fair (not monster) size review, it will be up-voted equally, and other will add their own answers, simply noting the points you already made. The result? - 1) You ensure that the asker feels happy that he got a quick response - 2) The question ultimately gets a more complete answer in total, because everyone was able to focus on specific aspects of the code. \$\endgroup\$
    – user73428
    Commented Jul 21, 2014 at 15:53
7
\$\begingroup\$

Different answer to your specific questions that you asked on main site.

Your question from 'over a week ago' (June 30'th) was answered. Why you say it was not, is a mystery. The answer is a partial answer, for sure, but, it is there, and it arrived within 3 days. If you are not happy with the answer feel free to throw a bounty on to the question to get more reviews....

The second question you asked yesterday (on a Saturday) has not been answered yet, but, it is also a huge question. You also have 3 comments on that question with what I consider to have a 'nasty' feel.

So, you ask a question that would take about 30 minutes to read, and understand, and another hour or so to review, from a volunteer, on a weekend, (at night for many folk - north america and europe), and then you start adding 'sarky' comments like: 24 hours since posting, still nothing. Jeez...

That does not make me want to 'waste' an hour of my weekend for your benefit when you obviously do not appreciate the time people put in to doing a half-decent review, let alone really good reviews.

Then you post a meta question challenging the site's commitment?

Am I misinterpreting you here? Is there a misunderstanding about what you expect?

You suggest:

This site is an outstanding learning tool for both reviewers and askers, so I'd really like to keep it around for the future. Perhaps we should take a look at where our strengths and weaknesses lie with this issue.

What this site really needs is people who vote, vote for questions, and vote for answers. We also need people to ask, and answer questions.

You have voted exactly three times .... and I presume two are for the answers on your first questions. You need to get yourself and your friends to vote more, and answer more.

Both of those are things that will help the site, and those are our weaknesses.

Site Activity and vote graph

You will notice that since last November the site is doing much better.

\$\endgroup\$
3
  • \$\begingroup\$ Your question from 'over a week ago' (June 30'th) was answered. Why you say it was not, is a mystery. That answer begins with: This answer is not a full review, but just a minor tweak. \$\endgroup\$
    – user73428
    Commented Jul 20, 2014 at 18:28
  • \$\begingroup\$ I don't mean any offence, rolfl. I'd like to see the site improve. I do intend to contribute here. I look forward to posting high quality questions, and eventually, reviews, here. If you think I should show less code in that question, suggest that! I'm still learning how to post the right way here. Like you said, I've got 3 votes. I'm new. \$\endgroup\$
    – user73428
    Commented Jul 20, 2014 at 18:31
  • 3
    \$\begingroup\$ @jt0dd I understand, and mostly agree with, what rolfl is saying here. I am a bit known for my excessively long questions, and long questions normally take a lot more time to get an answer than shorter ones (unless you are lucky). So my recommendation is that if you want quick answers, ask shorter questions - or have more patience. There is nothing wrong with long questions in itself though, as long as the question is written in a well-structured way. \$\endgroup\$ Commented Jul 20, 2014 at 21:56
6
\$\begingroup\$

one of the primary issues listed was that it takes a long time here for questions to be answered.

That could be due to a number of reasons. I think the main one is that we don't have a large number of "season" reviewers around. Other than the top users who review consistently, we have many lower-rep users who review every now and then. This is not necessarily a bad thing, but we need to retain more reviewers so that new questions will be much less likely to become forgotten for a while.

Another significant one is voting activity. If a user posts one or more great answers and hardly gets any upvotes on them, they may become less inclined to continue posting them. In turn, they remain with fewer rep and fewer privileges, especially if they want to help moderate the site.

What can you do? Above all else, continue to vote on good content. It doesn't just have to be for your own questions or other answers on questions you've answered. If you like someone's post, give them their upvote to show your appreciation. If you can answer, then please do so. We should roughly have more reviewers than askers, and it is understood that reviewing code is not always so simply to do, especially if you're unsure of how to best help someone with their code.

\$\endgroup\$
2
  • 2
    \$\begingroup\$ All of that noted and agreed with, I think there's one really powerful thing that reviewers seem to be forgetting here. People aren't necessarily asking you to dissect every ounce of their code. I think it would improve the site's morale, especially in the area of answer-speed, if all reviewers would make a habit of saying that the code looks good, make a general note why, and let that be an answer - askers have no problem with that, and they'll feel much better about coming back to the site. \$\endgroup\$
    – user73428
    Commented Jul 20, 2014 at 17:58
  • \$\begingroup\$ @jt0dd: That's also why I sometimes refrain from going all-out on a review. I usually leave some things out so that there's still room for further answers. As for "looks good" answers, they must explain the reasoning behind this (though of course they can still be wrong). Answers that simply say "looks good" are subject to deletion as they really don't say much. \$\endgroup\$
    – Jamal
    Commented Jul 20, 2014 at 18:00
4
\$\begingroup\$

A Suggestion: Focus on "quick answers"

It's true, I am a new user here, mostly asking questions rather than answering them, but I think I see a place for very steep improvement with the issue of answer speed, and I'm going to post it as a suggestion. I use "we're" because I intend to review here too.

1. We're looking at questions

Questions are getting plenty of views, very quickly. It's not that people around here aren't viewing the questions. The issue is that when people don't know how to improve the code in any specific way, they leave, and from the view-to-answer ratio that I see on every question here, I'd say that this happens a ton.

2. We're passing them up when we don't know what to say

If a small fraction of the people who pass the question by, not knowing any improvements to suggest, would just say so, and give an up-vote, askers like myself would be much more inclined to come back and participate in the site, likely to be active members in the future.

For example:

This looks good for the most part, and here's a general reason why I think so. Blah Blah Blah, 3-4 lines of summary and suggestion.

3. We're trying to re-write people's code for them rather than offer simplified examples

You don't need to re-write the code to review it. If you don't see anything in need of complex or dramatic changes, just say so, and if it applies, offer some basic code examples and pointers.

4. And it's causing Code Review to have slow answer rates which contributes to low retention.

Leaving a quick, general answer, rather than passing it up because you don't know how to write an advanced review can be the difference between the asker staying or leaving the site.


This is simply the way that I see things, after asking questions, reading meta, and not getting feedback on questions that I put lots of effort into.

\$\endgroup\$
5
  • 4
    \$\begingroup\$ Good to have you around, welcome aboard! \$\endgroup\$ Commented Jul 20, 2014 at 18:25
  • \$\begingroup\$ +1 I like this. I do admit that I take #3 to heart, mainly because my (more-experienced) peers are more capable of doing it. I sometimes provide examples, but I can try to do it more often. \$\endgroup\$
    – Jamal
    Commented Jul 20, 2014 at 18:27
  • 1
    \$\begingroup\$ I'm a huge fan of seeing multiple short answers from different users on a question. Yes, great reviews tend to be walls of text and that tends to be most of what you see around here, but I think shorter answers do more for the OP and the overall rep earned here on CR. \$\endgroup\$
    – RubberDuck
    Commented Jul 20, 2014 at 21:15
  • 2
    \$\begingroup\$ The thing is, I'll often start an answer with "just a few comments about xyz"... and end up with a massive answer... before I even realize it. Part of me also feels that small-ish answers tend to favor FGITW and affect the overall thoroughness of the advice being given on this site... but I agree with answer quality not being a function of answer length. \$\endgroup\$ Commented Jul 21, 2014 at 0:22
  • 2
    \$\begingroup\$ on Point #2, I have seen questions where someone answered like this, saying that the code looked good, people saw there was an answer on the question and passed it by until I came along and downvoted the answer and posted a GOOD answer/review. I too at one time thought the same thing, and I still believe that something can be said about everyone's code, but not everyone can be the person that has enough knowledge to be able to say something about the code. Patience is a virtue, a very annoying one. \$\endgroup\$
    – Malachi Mod
    Commented Jul 21, 2014 at 16:57

You must log in to answer this question.