"Technically", the answer is yes. The $\mathrm{pH}$ value is $+\infty$. $\mathrm{pH}$ is simply a logarithmic scale to reference the concentration of (cationic) ionized hydrogen in a sample, and is defined by(*)
$$\mathrm{pH} := -\log_{10}\left([\mathrm{H}^{+}]\right)$$
where the measurement unit is the usual SI derived $\mathrm{\frac{mol}{dm^3}}$ (equiv. $\mathrm{\frac{kmol}{m^3}}$). That's it. The reason this is typically seen in association with acids and bases is that most common (i.e. excluding "Lewis acids") acids are substances which can give up $\mathrm{H}^{+}$ and, moreover, which do so when they are dissolved, meaning that measuring the concentration of such ions in a solution gives a clue as to how much acid may be present. A solid crystal of acid material has its $\mathrm{H}^{+}$ still bound in the acid molecules and is not ionized, thus the concentration of $\mathrm{H}^{+}$ is zero, hence by the above definition, the $\mathrm{pH}$ is $+\infty$ (it is customary to use the extended reals instead of the usual reals when dealing with logarithmic measures as they provide exactly this capability so one can represent zero).
The usual rule that "$\mathrm{pH}$ of 7 is neutral" comes from solutions in water: Water has the property that it can be converted into, and self-converts between ("auto-ioniziation") separate $\mathrm{H}^{+}$ and $\mathrm{OH}^{-}$ ions and its usual molecular form, $\mathrm{H}_2\mathrm{O}$. When one is dealing with pure water with no adulterants present, there is always, due to this process, around $10^{-7}\ \mathrm{\frac{mol}{dm^3}}$ of $\mathrm{H}^{+}$ present (though actually, this depends on temperature, but around room temp, it is around this much). Decimal logarithm of $10^{-7}$ is -7, hence the $\mathrm{pH}$ is 7. When you throw some acid in and it releases its protonic payload, the concentration of $\mathrm{H}^{+}$ rises by that amount, thus the $\mathrm{pH}$ drops.
The key here is that $\mathrm{pH}$ itself is not inherently a measure of acidity or basicity. Rather, it is a measure that is typically associated with such, and thus, serves as a useful proxy therefor, at least under some common circumstances. When you are not dealing with a solution in water - i.e. either as you are talking about here a pure solid chunk of acid material, or you are dealing with acid dissolved in something other than water - the usual signifiers of $\mathrm{pH}$ beyond it being a logarithmic measure of the $\mathrm{H}^{+}$ ion concentration go out the window. Likewise, outside of water, $\mathrm{pH}$ cannot be used to measure bases, either, even simple (Arrhenius) bases since the presence of $\mathrm{OH}^{-}$ does not imply a corresponding deficit in (now non-existent) $\mathrm{H}^{+}$. Non-aqueous basic solutions, even of Arrhenius bases, will have $\mathrm{pH}$ of $+\infty$ as well, hence useless. Indeed, anything that has no free $\mathrm{H}^{+}$ ions knocking around has, by definition, a $\mathrm{pH}$ of $+\infty$.
Insofar as classification goes, that is not done using $\mathrm{pH}$, but rather the chemical behavior of the compound: an acid is signified by its ability to give up $\mathrm{H}^{+}$ under suitable circumstances, e.g. dissolution (Arrhenius' definition) or when brought into contact with a base (Bronsted-Lowry definition, as proton donor). Bases are the complement to this.
(*) ADD (2019-04-28, IE+1935.17 Ms): Upon review, I found and so should point out that this is not technically the "strictest" definition of pH. Technically, it is not log of concentration per se, but rather of the "activity" of $\mathrm{H}^{+}$, which is defined as a "modulated" concentration
$$a_{\mathrm{H}^{+}} := f_{\mathrm{int}}(S) \cdot [\mathrm{H}^{+}]$$
by a factor $f_{\mathrm{int}}(S) \in [0, 1]$ which accounts for interactivity between dissolved $\mathrm{H}^{+}$ due to the fact of their extended charge (Coulomb / electrostatic) interactions and that modifies the acid behavior. This factor depends on the thermodynamic state $S$ of the system which includes both temperature and the concentration itself and thus makes the "proper" $\mathrm{pH}$ nonlogarithmic in the concentration. Nonetheless, as concentration approaches zero, $f_{\mathrm{int}}(S)$ goes to $1$ and $a_{\mathrm{H}^{+}}$ still vanishes, hence the $\mathrm{pH}$ is still $+\infty$ and moreover, at low nonzero concentrations the two definitions are very close. The "concentration $\mathrm{pH}$" as given above is more "properly" written "$\mathrm{p[H]}$".
