Lucas reagent follows SN1 pathway. $\ce{Zn}$ is a lewis acid that tends to bond with $\ce{-OH}$ and form a partial positive charge leading to the formation of a carbocation.
According to Hammond Postulate, the stability of the transition state should allow successive formation of product or intermediate. The stability of the positive charge induced on the carbon if stabilised in the transition state should therefore stabilise the carbocation. The rate determining step is the formation of the carbocation.
Since tertiary carbocations are the most stable the carbocation is formed quite easily and the product formation happens quickly.
Similarly, the primary and secondary alcohols, need to form a primary and secondary carbocation, which is not stable. Hence the reaction is slow. Zinc forms a weaker bond with $\ce{-OH}$ and induces a partial charge on them, thus not forming the whole carbocation.
So I think that in such cases where the carbocation is not stable, the reaction is slow, and may not have a a very well defined SN1 or SN2 classification i.e. truly borderline reaction may not occur, due to accompanying facets like solvation and ion pairs.