Let's approach this intuitively. I assume that before doing this problem you have tried to do other simple configurations. For example, if the current is directed along the surface of the tube in the longitudinal direction, the field inside would be zero, and outside it would go as in Fig. 1.
Figure 1: Jfmelero, CC BY-SA 4.0, via Wikimedia Commons
Another typical example is the magnetic field created by a long solenoid, shown in Fig. 2. Here we assume the turns are tightly wrapped and close enough so that the current can be thought of as going in the $\hat{\varphi}$ direction in cylindrical coordinates. Notice how in this case the field is zero outside. Inside, it goes in the $\hat{z}$-direction.
Figure 2: Public domain, via Wikimedia Commons
Now, in your problem, the current is tilted. There is a component $J_z=J\sin\theta$ which goes along the direction of the tube, and a component $J_\varphi =J\cos\theta$ that goes in the azimuthal direction. So we expect a field outside as in Fig. 1 caused by $J_z$ and a field inside as in Fig. 2 caused by $J_\varphi$. This explains the square root factor $$\sqrt{1-\left(\frac{h}{2\pi R}\right)^2}$$ that appears in the inside field. It's only that fraction of the total current that's responsible for the field inside.
If we apply Ampère's circuital law by taking a circle of radius $r$, then the current enclosed is 0, so why is the field non zero?
That is true. But that integration loop is not useful in this case, since the field inside goes along the tube, and therefore is perpendicular to the loop at every point, making $\int\mathbf B\cdot d\mathbf l$ identically zero. You need to choose a loop that has a nonzero contribution to $\int\mathbf B\cdot d\mathbf l$ along some part of it. It may be useful to revisit the case shown in Fig. 2 (the solenoid).
