Is the QuickDraw with Google neural net a convolutional neural network?

Question

Does anyone know, or can we deduce or infer with high probability from its characteristics, whether the neural network used on this site

https://quickdraw.withgoogle.com/

is a type of convolutional neural network (CNN)?

Answer 1

I believe they don't use CNNs. The most important reason why it's because they have more information than a regular image: time. The input they receive is a sequence of (x,y,t) as you draw on the screen, which they refer as "ink". This gives them the construction of the image for free, which a CNN would have to deduce by itself.

They tried two approaches. Their currently most successful approach does the following:

• Detect parts of the ink that are candidates of being a character
• Use a FeedForward Neural Network to do character recognition on those candidates
• Use beam search and a language model to find most the most likely combination of results that results into a word

Their second approach is using an LSTM (a type of Recurrent Neural Network) end-to-end. In their paper they say this was better in a couple languages.

Source: I was an intern in Google's handwriting team in summer 2015 (on which I believe quickdraw is based), but the techniques I explained can be found in this paper.

Answer 2

It looks like that it used convolution and recurrent neural network. There is a dataset webpage used in Quick Draw project and it led to Recurrent Neural Networks for Drawing Classification GitHub page. The GitHub link showed the neural network structure as well.

Here is the neural network structure from the GitHub page:

The current Quick Draw might be evolved from the neural network structure at the GitHub because the webpage written in 2020.