Research before surveying the literature - is it ever a good idea?

Question

The dominant narrative on how research "should be" done is to begin with a literature survey and then make your own contribution.

But I get the impression that it often happens the other way around - at least in fields such as CS where the cost of knocking some lines of code together is low (compared to, say, a 3 year medical testing programme). Creativity knows no rules.

• The obvious risk of this approach is that the research is subsequently found to be old hat, or better versions of the same thing exist already, or it commits an error already widely known in the field.

• The reward, on the other hand, is that such creation might never have happened if the requirement of surveying the literature first was always met. (Indeed some grad students have been known to quit before finishing the lit review their supervisor asked for - maybe they'd have done better if allowed to exercise their creative nature first?).

So I would conclude that it's an acceptable strategy when the individual is happy with that risk/reward balance (provided they already know enough about what they're doing to have some inspiration in the first place, that is).

Are there other arguments/perspectives I've missed here?

UPDATE

There are a lot of very good answers here and there is no one right answer. I'm not quite sure how to handle that. In my experience this is the kind of thing that tends to lead to questions being closed as "not constructive" on SE sites. I'm glad nobody has done so yet because I think the full range of opinions expressed below is a very worthwhile thing to read through. Thanks all for your contributions.

Answer 1

• One point I don't see so far is that a quick experiment (in lab or in silico) can result in showing that the idea wasn't that great after all. And in that case, it is rather unlikely that literature says anything about it at all.

• Another point (rather in line with @Andy W and @badroit's answers) is that spending some effort of your own on the problem may be needed in order to find the relevant literature.
  (I've had problems that I eventually solved myself, and where only the solution I obtained allowed me afterwards to find some relevant literature in a different field - simply because only the solution yielded successful search keywords)

• Of course, students solving old and known problems is often called "education" and considered necessary, not a waste of time...

• Last but not least, a few hours in the library often still make months in the lab necessary...

Answer 2

I don't entirely agree with the answer "No". This may be because my field is mathematics, though. Sure, not looking at literature beforehand means you are at risk of re-doing something that was already known and it's never nice to discover after spending lots of time working on something that it was already out there.

On the other hand, when you come across what seems to be a good question, I don't think it's such a bad idea to start by asking yourself "Can I answer this myself?" and not rush immediately to see what other people have already done in relation to this question. I see a possibility of positive outcome to not being aware of previous results for a significant time (of course not to the point of being on the verge of publishing and discovering the day before submission a very similar paper...).

As ff524 pointed out in a comment, "if you are a researcher, [...] you already have some idea what's been done before". If you find an interesting question and don't know about an answer, it may be because it does not exist, but it may also happen that it does and has been overlooked before. Maybe it came out at a time when it didn't meet the interests of the community, or it came from another community, or the communication on the result was not so good, or it didn't look interesting because the strategy was inappropriate. If you find this result early on, you will likely drop the question, not advertise it and the result (although of interest to the community) will remain unknown. If you start searching the answer without knowing about the previous result, it's likely that you will not follow the exact same path and that can be good.

• Maybe your way is more "natural" and makes the result fit nicely in a developing theory that didn't include it before. It's not such a bad idea to provide a new way of looking at something, even if the result per se is not new.
• Maybe your way is more fruitful, in the sense that during your quest for the answer to the original question you will come up with new interesting questions which did not stem clearly from the way the result was previously established.

It's still risky, but in research you constantly invest your time in stuff you don't know the outcome of.

Answer 3

To a certain extent this creates a false dichotomy. My mind is always turning with new ideas, and I may jot them down on paper and give then a bit of thought before I go and see if it is old hat with prior literature. The bit of thought may be a few hours of deliberation, or some simple experimentation or writing a few pages of a paper.

You need a general knowledge of the field to be able to understand how to make a contribution. You then have an idea, read on that specific prior literature, and this will often refine your idea. The more specific your idea becomes the easier it is to find prior literature and see if/how it is different. It is not a one before the other - they are constant and reciprocating.

The consumption of the pertinent scientific literature though is forever and endless.

Answer 4

Research before surveying the literature - is it ever a good idea?

If I reword that as:

Independent problem-solving before surveying the literature - is it ever a good idea?

Yes. I think in most cases it is a good idea to make an "initial effort" (the definition of "initial effort" depends on the scenario) to independently solve a problem before checking the literature. If you have not sat down and thought about the problem yourself, then:

1. you may not readily understand the solutions (if any) proposed in the literature unless you've worked through them yourself

2. you may not know the extent to which solutions are appropriate if you have not tried to apply your own ideas or thought of alternatives

3. you may not know what to search for ... the core problem may not be as it initially seems but may reduce to something more well-known

4. your own ideas on the problem may be influenced into the narrower-scope of what has already done (it might stifle creative problem-solving in the broader sense)

Of course, it is important to avoid working on a problem for months without referring to the literature at any point.

But certainly for a student, I would encourage them to think about and frame solutions for a problem strictly before they do a literature survey! Once they show understanding of the problem and of some approaches then I think they're ready to read papers ... doing it in this order, they're much better equipped to recognise relevant papers and interesting approaches when they see them.

Answer 5

While doing my Master's thesis my advisor said that (up to the wording):

It is better to spend the same time solving a problem than searching for its solution.

You learn more (compare: solving a problem in class vs googling for its solution) and more fun.

However, the question is:

• What is the time overhead (i.e. how much longer does it take to solve it)?
• Do you get the big picture?

If solving takes too much time, you may end up learning a lot but not doing new research. Also, without feedback (literature review or even better asking experts if the problem is novel and worth pursuing) you may end up with:

• solving already solved problems (perhaps with much more general methods)
• or solving problems no one cares about.

In any case, any sensible research strategy involves both.

Answer 6

A few thoughts. This is clearly a delicate balancing act by the individual and also an area of convention passed down from teacher to student in different fields. One of the main roles of an advisor is to keep the student on track and make sure they are not going off in "unproductive" directions. "Unproductive" involves extremes. The young researcher may be spending a lot of time experimenting without looking at the literature, or too much time looking at literature and not experimenting.

A young researcher who spends a lot of time on an idea and then the advisor quickly shoots it down as previously explored may find his/her balance is slightly off and need to "recalibrate". But also realize this is a common scenario that has happened to all fledgling scientists, and even senior ones. Research and experimentation are a feedback loop which of course go "hand in hand". One must be continually/flexibly "jumping between both worlds" at all stages of development.

Yes, each scientific field probably has slightly different conventions. Some fields are more empirical than others. Some are "younger", and yes there is some case to be made that computer science is a younger field than others, which means that the balance may be different in this field. Also, there is some case to be made that empirical research in CS is underexplored at this moment in the larger scheme.

However, computer science is a somewhat unique field in that, yes, the individual may be able to do scientific experiments without expensive laboratory equipment independent of his research laboratory, on "his/her" own time, and gain significant understanding/insight toward their research thesis.

Answer 7

I agree with your conclusion that it is an acceptable strategy when the individual is happy with the risk/reward balance. My supervisor used to always encourage his students not to do a literature survey but instead work on a concrete problem. Some students liked this approach, others did not like it and left to somewhere else within few months and still did very well in their research. I personally was undecided.

An obvious reward of this approach is that you may be able to publish something interesting within a year from the start of your PhD study. On the other hand you may end up with a negative result that can not be published and thus you may decide to change topic.

Spending a year or so on a literature survey may seem like a waste of time, however it can really help in the process of building an overall understanding of the field and boost your confidence and ability to discuss with more experienced people.

Answer 8

Knowing the state of the art is necessary, as well as having a clear definition of the problem that has to be solved.

Otherwise the student can:

• make old know mistakes, as you pointed
• solve old problems that have already been solved
• worse than that, solve problems nobody cares about and where the solution is completely useless (specially in middleware that will not be used by any top layer or lower layer).

In short, it can be a complete waste of an arbitrarily long time.

Whether this is done by the supervisor pointing at a few selected papers or the students reading in a uninformed way trying to understand what they got into is a different matter.

IMHO, the supervisor should guide the student in the first years. A few minutes to make a reading list can save months of reading not-so-interesting things, being lost in the broadness and vagueness of a not properly defined state of the art/problem/area, and a lot of frustration and misunderstanding.

Answer 9

I had a discussion like this with a seasoned professor. I think the best and also simplest thing to do when tackling a new problem is to first figure out multiple ways you think you could solve it WITHOUT doing a literature review. That shouldn't take too long, and doesn't involve actually doing any research, just some thinking and maybe some basic reading (but not a literature review). Then, review the literature, at which point you'll see if your ideas have been tried before and \ or have merit, based on your improved understanding.

What happens a lot in science is that people acquire a lot of knowledge in a field, and then often creativity is stifled, as every solution to a problem is seen in the light of the existing knowledge. It's hard to think originally about a problem when you've seen lots of other attempts to solve it. I call this the 'burden of knowledge'. So I would encourage contemplating a problem yourself before seeing what other people have done. This may lead to an original line of inquiry. But preferably before you spend a lot of time doing real research, do a thorough literature review. You'll also find that contemplating how to solve the problem yourself will help you understand the efforts of others better.

One more thing. In the field of mathematics at least, it has been commonly noted in history that some of the greatest mathematicians solved some big problems in their youth, before realizing that others had already solved that problem. I specifically recall reading this about Alan Turing, but he wasn't alone in this. Sometimes this may lead to alternate proofs that are more elegant or reveal other mathematics. For instance, Andrew Wiles solved Fermat's last theorem using tools that didn't exist in Fermat's day. So assuming Fermat did in fact solve his famous last theorem, then there is likely a much simpler solution which may reveal new mathematics. So sometimes solving an existing problem in a new way can be very valuable in its own light. That may be true even if your solution is not more elegant or more effective, if it's shows new lines of inquiry.

Answer 10

I would argue that it only becomes "research" when reading up on existing literature is involved. Basically, if inspiration hits you and you just fire up your IDE to test it, you are not aware if what you are doing is in any way novel. That's not really the nature of research.

Edit: (I should add that even if something is not novel in a scientific sense, it can still be extremely interesting and cool to do - however, it will likely never lead to a paper)

That is not to say that a researcher is never allowed to just follow her/his inspiration. It has certainly happened to me more than a few times that I had a crazy idea and just ran with it. Oftentimes, the quickly led nowhere and I just forgot about it. Sometimes, initial results proved interesting and then I went ahead and did proper research before going further.

Edit 2: I have just noticed that I did not actually answer your question:

Research before surveying the literature - is it ever a good idea?

No, not really. If you are doing "research" (in the sense that you are hoping to publish it, and not doing it out of personal interest, or as a training exercise, etc.), the only senseful way is to start with a literature review. One needs to keep in mind that in most fields, most of the "low-hanging fruits" (i.e., obvious ideas) are long solved. So just going ahead and doing something in the hopes that it may be novel has a significant error rate.