The answer, as with all performance questions, is "it depends". I can go into a lot of detail about what it depends on, but Duarte Farrajota Ramos' comment nothing like a good old test is the only reliable way to get an answer.
There is a cost to using image textures, even in terms of GPU VRAM usage, but even it is difficult to quantify because it depends on the size of the image, the way it is UV mapped, and any in-shader manipulations applied to it.
The simplest procedural shaders are faster than using an image texture, so you can't say that image textures are always better. Moreover the cost of using any given texture in a procedural shader depends both on the implementation of the texture in Blender and how well it maps to the GPU. Brick textures are very fast and use very little VRAM. Noise textures are notoriously slow in Blender and can dramatically increase the time cost of the shader.
There is also a time / space tradeoff that varies. Some shader techniques use very little VRAM but consume relatively large amounts of time. Again, noise textures tend to be bad this way.
Finally, a lot has to do with the infamous "kernels" -- the bits of GPU code that blender generates as it builds the shader. Some GPUs are easier to build high performance kernels for, so the same texture might have very different costs on GPUs from two different families.
So, in the final analysis, all you can do is try it both ways, compare the render times, and decide which approach works best for you.
But that leads to a question your question implies but doesn't ask: what is the workflow cost. Image based materials have very little workflow impact. Set up the material, make a few adjustments and you're done. Procedural materials may take much more time to develop and you do have the time cost of baking the material to consider.
