It might be a short story or it might be a novel. It must be before 1972 because I read it in my youth and I am OLD.
The following is very impressionistic.
There is a multidimensional/not of this Earth/alien artifact-like tunnel.
Picture a long, perhaps transparent, storm drain like structure sitting in a field but with an opening on only one end.
It is the subject of a lengthy investigation/exploration that has cost many lives. An instrumented investigator enters the "tunnel" and makes the exact movements that his predecessors have made, advancing until he must do something original at which point he survives and a new "move" is added to the list of correct moves or he is destroyed and a "move" is added to the list of things to be avoided at that point.
I remember absolutely nothing else about the work except that (I hope central) feature.