Is it normal to not care about the current literature?

Question

I am currently working on a research project with my PhD supervisor. The focus of this project is a sub-field that my supervisor has not previously published in and one that I am not familiar with. This sub-field has also been studied in the literature for more than 30 years. From past research experience, I am used to performing a literature review before a research project is started to answer the following questions:

1. What has already been done?
2. What is the current state-of-the-art (SOTA) algorithm?
3. What improvements can be done to the SOTA?

Answering questions 1 and 2 simply involves gathering information from different papers. Answering question 3, however, is a bit more complicated, since this would involve understanding how the SOTA algorithm precisely works, and then identifying an area of improvement.

Moreover, the SOTA algorithm in this case is the result of incremental improvements of a basic algorithm that was proposed 20-30 years ago. Since I am not familiar with the field, then understanding how the SOTA algorithm works would involve understanding how the basic algorithm works and then understanding the incremental improvements that were implemented.

The problem is that understanding and re-implementing the basic algorithm would take approximately 2-3 months. This is because it requires technical background that I do not yet have. My PhD supervisor is aware of this, and because of this, has proposed that we do not bother with re-implementing the basic algorithm or the SOTA algorithm and instead try to approach the problem from a completely different perspective, that has not been previously proposed in the literature, using the technical knowledge that we are familiar with.

While I do appreciate my supervisor's ambition, I have a few concerns:

1. This field has been studied for over 30 years. Complete books have been written about it. The fact that nobody bothered to go down the path that we are exploring strongly suggests that it is not a fruitful one.
2. Because we would essentially be ignoring the literature, we would be re-inventing the wheel a lot of the time. This means we would be making a lot of mistakes, learning from them, and then most likely end up implementing what was already done in the literature. This would waste a lot of time.
3. Even if we do end up designing an algorithm, there is no guarantee that it will improve on the SOTA. If it doesn't, then what?

Are my concerns justified? For my third concern, I am not sure if improving on the SOTA is always the goal of publication, and it is possible that some conferences will appreciate a new approach to an existing problem, even if it doesn't improve on the SOTA. However, I am not sure about this.

Answer 1

Let's start with your concerns:

Number 1: "The fact that nobody bothered to go down the path that we are exploring strongly suggests that it is not a fruitful one." - No. There may be many reasons why this has not been tried, e.g. fashion, lack of relevant expertise, or simply the tools for this route were not available at the time the topic began to develop and people pursued the - then - easier route.

Number 2: Ignoring the literature is not really an option on the long run, but it can be good idea to try making your own mistakes first before being too much "confused" by previous approaches. Once you tried and failed a few times, you will be able to appreciate much more the literature and understand much more easily why they did things the way they did - or, if you succeed, you now can try to understand where they failed or how they did things differently. It can be in general much easier to read the literature if you had the possibility to try a direction on your own. Ultimately, you will have to embed your insights into existing knowledge, but if your method is superior, it will supersede the work, and if it is not, you still may have understood something beyond the other approaches.

Number 3: Scientifically, it is perfectly ok to try a different algorithm principle, even if it does not supersede high-strung algorithms developed over decades of refining and improvement. The challenge will be getting it published, because some venues believe only in benchmark-breaking methods rather than novel insights and understanding. However, if you believe in the direction, go for it. You might want to try and get a sample implementation of the standard method, though, to be able to run comparisons with the existing models. I strongly recommend getting one, even if only as executable if nothing else.

Answer 2

I think your title is a bit misleading: "not wanting to invest 2-3 months into reimplementing an algorithm" and "not caring about the current literature" is not the same.

To the point. First, I don't see why an algorithm X must be necessarily constructed as an extension of an algorithm Y. You have to care about improving the results provided by SOTA, but it doesn't mean building on top of a SOTA algorithm. Second, you have a concern that your method won't be better than SOTA. It is of course valid, but you may as well fail to improve an existing solution. Third, I find it odd that in a well-research field you need to "reimplement" anything. Can't you get some ready-made implementation and work with it?

In general, I am afraid there is no "guaranteed" road in your case. There is some risk in both options. In such situations I usually try to find at least some "sellable" features of my solution, which can be considered along with SOTA results. For example, you can consider whether your approach is going to be faster / use less memory / less CPU / be better in certain specific cases, etc. This should give some backup options.