GPU for deep learning

Question

I have a budget of ~10 kUSD to buy GPUs for neural network applications, mostly using Theano, Torch, TensorFlow and Caffe. Most programs I plan to run are for natural language processing. Computations are in float 32 and in CUDA. I live in the USA, and I don't pay for the electricity bills. The GPUs will be mounted to some computers running Ubuntu 14.04.3 LTS x64.

What are the graphic cards with the highest computational power / price ratio, given the above-mentioned conditions?

Answer 1

I haven't personally used CUDA for this, though I am planning to. From my research I concluded, that you for sure want to have 'Computing Capability' at least equal to 3.5, as some libraries requre it already. The (tough to find) list of computing capability is under this link. From this list one can conclude, that having GTX 980 or Titan (both 5.2 score) is as good as you can get, but note, that if you are buing a graphics card only for this, Nvidia has an answer for professional and academic use named Tesla - it's just a computing box, far from graphics card (it has no display ports even!), costs from 3k $ for K20, to 5k $ for K80 model, and is a behemoth:

A quick comparision (CC stands for Compute Capability):

• Tesla K20: for desktops, peek for float: 3.52 Tflops, 5 GB, 2496 CUDA cores, 2.9 k $, CC: 3.5
• Tesla K40: for desktops, peek for float: 4.29 Tflops, 12 GB, 2880 CUDA cores, 3.1k $, CC: 3.5
• Tesla K80: for servers, peek for float: 8 Tflops, 24 GB, 4992 CUDA Cores, 5k $, CC: 3.7

and customer grade, most popular and new graphics cards:

• GTX960: for desktops, peek for float: 2.3 Tflops, 2 or 4 GB, 1024 CUDA cores, 200$, CC: 5.2
• GTX980: for desktops, peek for float: 4.61 Tflops, 4 GB, 2048 CUDA cores, 500$, CC: 5.2
• Titan X: for desktops, peek for float: 6.1 Tflops, 12 GB, 3072 CUDA cores, 1000$, CC: 5.2

Also see comparision on wccftech.

To conclude: the commercial grade GPU seem to be more cost efficient, but only when we compare the specs. There can be other trade-offs, I am not awere of. Thus, I cannot confidently say "go with customer grade", but I can tell you what I would (will) do - I will buy GTX 960 or 970, because I plan on gaming and I'm quite cost limited, and this cards will do just fine for CUDA learning. If you buy for an institution, do not plan gaming, the calculations will go 24/7, consider the academic grade Teslas.

Also, if you'll be interested in boosting your 'conventional' integer based processing power on a high-end computation server, you may want to look up Xeon phi.

[EDIT] Please note, that switching from CPU based floating point arithmetic to GPU enchanced, is a change in quality, almost one order of magnitute, and will be very pronounced and noticible, but switching from ex. Tesla K20 to Tesla K40 will be just a change in quantity (K80 is just two K40 bundled together), so if you go for speed to price ratio, go with cheapest GPU acceleration, that'll work for you.

Answer 2

Nvidia has just announced the Nvidia GTX 1080, which is ~25% faster than the Titan X and significantly cheaper ( 600 USD vs 1000 USD).

From http://www.anandtech.com/show/10304/nvidia-announces-the-geforce-gtx-1080-1070/2:

From http://wccftech.com/nvidia-geforce-gtx-1080-launch/:

It's difficult to find exhaustive specification comparison tables between the GTX 1080 and Titan X (I guess they should appear soon). A few more comparisons GTX 1080 vs Titan X:

• 9 Tflops vs 6.1 Tflops

---

From some Nvidia official presentation:

The Nvidia GTX 1070 was also announced:

From cnn-benchmarks:

GTX 1080 > Titan X: Across all models, the GTX 1080 is 1.10x to 1.15x faster than the Titan X.

Answer 3

Nvidia announced on 2016-07-21 the new GTX Titan X:

• Under 1200 USD
• "Potentially 24% faster than GTX 1080; 60% faster than the old Titan X."
• 11 teraflops of FP32 performance
• 12GB of GDDR5X memory running at an effective 10GHz and attached to a wide 382-bit bus, resulting in a 480GB/s of memory bandwidth

A first benchmark (unfortunately they didn't keep the same CuDNN…):