Exploring Google APIs with Python

Exploring Google APIs
with Python
Wesley Chun - @wescpy
Developer Advocate, Google
Adjunct CS Faculty, Foothill College
Developer Advocate, Google Cloud
● Mission: enable current and future
developers everywhere to be
successful using Google Cloud and
other Google developer tools & APIs
● Focus: GCP serverless (App Engine,
Cloud Functions, Cloud Run); higher
education, Google Workspace, GCP
AI/ML APIs; multi-product use cases
● Content: speak to developers globally;
make videos, create code samples,
produce codelabs (free, self-paced,
hands-on tutorials), publish blog posts
About the speaker
Previous experience / background
● Software engineer & architect for 20+ years
○ Yahoo!, Sun, HP, Cisco, EMC, Xilinx
○ Original Yahoo!Mail engineer/SWE
● Technical trainer, teacher, instructor
○ Taught Math, Linux, Python since 1983
○ Private corporate trainer
○ Adjunct CS Faculty at local SV college
● Python community member
○ Popular Core Python series author
○ Python Software Foundation Fellow
● AB (Math/CS) & CMP (Music/Piano), UC
Berkeley and MSCS, UC Santa Barbara
● Adjunct Computer Science Faculty, Foothill
College (Silicon Valley)

Agenda
1
Introduction to
Google APIs
2
Google Cloud
APIs overview
3
Serverless tools +
code samples
4
Inspiration
5
Wrap-up
01
Introduction to
Google APIs
Why are you here?

Cloud/GCP console
console.cloud.google.com
● Hub of all developer activity
● Applications == projects
○ New project for new apps
○ Projects have a billing acct
● Manage billing accounts
○ Financial instrument required
○ Personal or corporate credit cards,
Free Trial, and education grants
● Access GCP product settings
● Manage users & security
● Manage APIs in devconsole
● View application statistics
● En-/disable Google APIs
● Obtain application credentials
Using Google APIs
goo.gl/RbyTFD
API manager aka Developers Console (devconsole)
console.developers.google.com

Two different client library "styles"
● "Platform-level" client libraries (lower level)
○ Supports multiple products as a "lowest-common denominator"
○ Manage API service endpoints (setup & use)
○ Manage authorization (API keys, OAuth client IDs, service accounts)
○ Google Workspace, Google Analytics, YouTube, Google Ads APIs, etc.
○ Install: developers.google.com/api-client-library
● "Product-level" client libraries (higher level)
○ Custom client libraries made specifically for each product
○ Managing API service endpoints & security mostly taken care of
○ Only need to create a "client" to use API services
○ Install (Cloud/GCP & Firebase): cloud.google.com/apis/docs/cloud-client-libraries
○ Install (Maps): developers.google.com/places/web-service/client-library
● Some Google APIs families support both, e.g., Cloud
Google APIs client
libraries for common
languages; demos in
developers.google.com/api-
client-library
cloud.google.com/apis/docs
/cloud-client-libraries

Three different authz credential types
● Simple: API keys (to access public data)
○ Simplest form of authorization: an API key; tied to a project
○ Allows access to public data
○ Do not put in code, lose, or upload to GitHub! (can be restricted however)
○ Supported by: Google Maps, (some) YouTube, (some) GCP, etc.
● Authorized: OAuth client IDs (to access data owned by [human] user)
○ Provides additional layer of security via OAuth2 (RFC 6749)
○ Owner must grant permission for your app to access their data
○ Access granularity determined by requested permissions (user scopes)
○ Supported by: Google Workspace, (some) YouTube, (some) GCP, etc.
● Authorized: service accounts (to access data owned by an app/robot user)
○ Provides additional layer of security via OAuth2 or JWT (RFC 7519)
○ Project owning data grants permission implicitly; requires public-private key-pair
○ Access granularity determined by Cloud IAM permissions granted to service account key-pair
○ Supported by: GCP, (some) Google Workspace, etc.
SIMPLE
AUTHORIZED
Which do you choose?

OAuth2 or
API key
HTTP-based REST APIs 1
HTTP
2
Google APIs request-response workflow
● Application makes request
● Request received by service
● Process data, return response
● Results sent to application
(typical client-server model)
02
Google Cloud
APIs overview

Running Code: Compute Engine
>
Google Compute Engine delivers
configurable virtual machines
of all shapes and sizes, from
"micro" to 416 vCPUs, 11.776
TB RAM, 256 TB HDD or SSD
disk; GPUs & TPUs
(Debian, CentOS, CoreOS, SUSE, Red Hat Enterprise
Linux, Ubuntu, FreeBSD; Windows Server 2008 R2, 2012
R2, 2016, 1803, 1809, 1903/2019, 1909)
cloud.google.com/compute

Running Code: App Engine
Got a great app idea? Now what?
VMs? Operating systems? Big disk?
Web servers? Load balancing?
Database servers? Autoscaling?
With Google App Engine, you don't
think about those. Just upload
your code; we do everything else.
>
cloud.google.com/appengine
Running Code: Cloud Functions
Don't have an entire app? Just want
to deploy small microservices or
"RPCs" online globally? That's what
Google Cloud Functions are for!
(+Firebase version for mobile apps)
cloud.google.com/functions
firebase.google.com/products/functions

Running Code: Cloud Run
Got a containerized app? Want its
flexibility along with the convenience
of serverless that's fully-managed
plus auto-scales? Google Cloud Run is
exactly what you're looking for!
Need custom HW? Cloud Run on GKE
cloud.google.com/run
Managed containers: Kubernetes Engine
Got a containerized application?
Google Kubernetes Engine is an
enterprise-grade, fully-managed
container orchestration service.
cloud.google.com/kubernetes-engine

Storing Data: Cloud Storage & Cloud Filestore
cloud.google.com/storage
cloud.google.com/filestore
Storing Data: Cloud SQL
SQL servers in the cloud
High-performance, fully-managed
600MB to 416GB RAM; up to 64 vCPUs
Up to 10 TB storage; 40,000 IOPS
Types:
MySQL
Postgres
SQLServer (2019)
cloud.google.com/sql

Storing Data: Cloud Firestore
The best of both worlds: the next
generation of Cloud Datastore
(w/product rebrand) plus features
from the Firebase realtime database
(For choosing between Firebase & Cloud Firestore: see
firebase.google.com/docs/firestore/rtdb-vs-firestore;
for choosing between Cloud Datastore & Firestore: see
cloud.google.com/datastore/docs/firestore-or-datastore)
cloud.google.com/firestore
Storing and Analyzing Data: BigQuery
Google BigQuery is a fast, highly
scalable, fully-managed data
warehouse in the cloud for
analytics with built-in machine
learning (BQML); issue SQL queries
across multi-terabytes of data
cloud.google.com/bigquery

Passing Data & Events: Pub/Sub
Google Pub/Sub: a fast, highly
scalable, fully-managed multi
fan-in/fan-out publisher-subscriber
queuing system for messaging &
event ingestion (and processing)
cloud.google.com/pubsub
Machine Learning: Cloud Translation
Access Google Translate
programmatically through this
API; translate an arbitrary
string into any supported
language using state-of-the-art
Neural Machine Translation
cloud.google.com/translate

Machine Learning: Cloud Natural Language
Google Cloud Natural Language API
reveals the structure and meaning
of text, performing sentiment
analysis, content classification,
entity extraction, and syntactical
structure analysis; multi-lingual
cloud.google.com/language
Machine Learning: Cloud Vision & Video Intelligence
Google Cloud Vision & Video
Intelligence APIs enable developers
to extract metadata & understand the
content of images & videos, making
them searchable & discoverable.
cloud.google.com/vision
cloud.google.com/video-intelligence

Google Workspace
Top-level documentation and comprehensive developers
overview video at developers.google.com/gsuite
(formerly G Suite and Google Apps)
APIs
03
Serverless tools
+ code samples
Using Google APIs w/Python

> Google Compute Engine configurable
VMs of all shapes & sizes, from
"micro" to 416 vCPUs, 11.776 TB
RAM, 256 TB HDD/SSD plus Google
Cloud Storage for data lake "blobs"
(Debian, CentOS, CoreOS, SUSE, Red Hat Enterprise Linux,
Ubuntu, FreeBSD; Windows Server 2008 R2, 2012 R2, 2016, 1803,
1809, 1903/2019, 1909)
cloud.google.com/compute
cloud.google.com/storage
Yeah, we got VMs & big disk… but why*?
Serverless: what & why
● What is serverless?
○ Misnomer
○ "No worries"
○ Developers focus on writing code & solving business problems*
● Why serverless?
○ Fastest growing segment of cloud... per analyst research*:
■ $1.9B (2016) and $4.25B (2018) ⇒ $7.7B (2021) and $14.93B (2023)
○ What if you go viral? Autoscaling: your new best friend
○ What if you don't? Code not running? You're not paying.
* in USD; source:Forbes (May 2018), MarketsandMarkets™ & CB Insights (Aug 2018)

Why does App Engine exist?
● Focus on app not DevOps
○ Web app
○ Mobile backend
○ Cloud service
● Enhance productivity
● Deploy globally
● Fully-managed
● Auto-scaling
● Pay-per-use
● Familiar languages
● Test w/local dev server
Hello World (Python "MVP")
app.yaml
runtime: python38
main.py
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello():
return 'Hello World!'
requirements.txt
Flask>=1.1.2
Deploy:
$ gcloud app deploy
Access globally:
PROJECT_ID.appspot.com
cloud.google.com/appengine/docs/standard/python3/quickstart

Why does Cloud Functions exist?
● Don't have entire app?
○ No framework "overhead" (LAMP, MEAN...)
○ Deploy microservices
● Event-driven
○ Triggered via HTTP or background events
■ Pub/Sub, Cloud Storage, Firebase, etc.
○ Auto-scaling & highly-available; pay per use
● Flexible development environment
○ Cmd-line or developer console (in-browser)
○ Develop/test locally with Functions Framework
● Cloud Functions for Firebase
○ Mobile app use-cases
● Available runtimes
○ JS/Node.js 8, 10, 12, 14
○ Python 3.7, 3.8, 3.9
○ Go 1.11, 1.13
○ Java 11
○ Ruby 2.6, 2.7
○ .NET Core 3.1
main.py
def hello_world(request):
Deploy:
$ gcloud functions deploy hello --runtime python38 --trigger-http
Access globally (curl):
$ curl REGION-PROJECT_ID.cloudfunctions.net/hello
Access globally (browser):
https://REGION-PROJECT_ID.cloudfunctions.net/hello
cloud.google.com/functions/docs/quickstart-python

The rise of containers... ● Any language
● Any library
● Any binary
● Ecosystem of base images
● Industry standard
FLEXIBILITY
“We can’t be locked in.”
“How can we use
existing binaries?”
“Why do I have to choose between
containers and serverless?”
“Can you support language _______ ?”
Serverless inaccessible for some...
CONVENIENCE

Cloud Run: code, build, deploy
.js .rb .go
.sh
.py ...
● Any language, library, binary
○ HTTP port, stateless
● Bundle into container
○ Build w/Docker OR
○ Google Cloud Build
○ Image ⇒ Container Registry
● Deploy to Cloud Run (managed or GKE)
● GitOps: (CI/)CD Push-to-deploy from Git
State
HTTP
main.py
import os
from flask import Flask
@app.route('/')
def hello_world():
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0', port=int(os.environ.get('PORT', 8080)))
cloud.google.com/run/docs/quickstarts/build-and-deploy
requirements.txt
Flask>=1.1.2

Dockerfile
FROM python:3-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "main.py"]
.dockerignore
Dockerfile
README.md
*.pyc
*.pyo
.git/
__pycache__
Build (think docker build and docker push) then deploy (think docker run):
$ gcloud builds submit --tag gcr.io/PROJ_ID/IMG_NAME
$ gcloud run deploy SVC_NAME --image gcr.io/PROJ_ID/IMG_NAME
OR… Build and Deploy (1-line combo of above commands):
$ gcloud run deploy SVC_NAME --source .
Deploy (think docker push):
$ gcloud run deploy --image
gcr.io/PROJ_ID/IMG_NAME
--platform managed
Access globally:
SVC_NAME-HASH-REG_ABBR.a.run.app
Docker &
Dockerfile
OPTIONAL!!
BigQuery: querying Shakespeare words
from google.cloud import bigquery
TITLE = "The most common words in all of Shakespeare's works"
QUERY = '''
SELECT LOWER(word) AS word, sum(word_count) AS count
FROM `bigquery-public-data.samples.shakespeare`
GROUP BY word ORDER BY count DESC LIMIT 10
'''
rsp = bigquery.Client().query(QUERY).result()
print('n*** Results for %r:n' % TITLE)
print('t'.join(col.name.upper() for col in rsp.schema)) # HEADERS
print('n'.join('t'.join(str(x) for x in row.values()) for row in rsp)) # DATA

Top 10 most common Shakespeare words
$ python bq_shake.py
*** Results for "The most common words in all of Shakespeare's works":
WORD COUNT
the 29801
and 27529
i 21029
to 20957
of 18514
a 15370
you 14010
my 12936
in 11722
that 11519
● BigQuery public data sets: cloud.google.com/bigquery/public-data
● BQ sandbox (1TB/mo free): cloud.google.com/bigquery/docs/sandbox
● Other public data sets: cloud.google.com/public-datasets (Google Cloud),
research.google/tools/datasets (Google Research), and Kaggle (kaggle.com)
● COVID-19
○ How to use our data sets (see blog post)
○ JHU Coronavirus COVID-19 Global Cases data set
○ List of all COVID-19 data sets
● Cloud Life Sciences API: cloud.google.com/life-sciences (see blog post)
● Cloud Healthcare API: cloud.google.com/healthcare (see blog post)
BigQuery & public data sets
Spring 2020

from google.cloud import vision
image_uri = 'gs://cloud-samples-data/vision/using_curl/shanghai.jpeg'
client = vision.ImageAnnotatorClient()
image = vision.types.Image()
image.source.image_uri = image_uri
response = client.label_detection(image=image)
print('Labels (and confidence score):')
print('=' * 30)
for label in response.label_annotations:
print(label.description, '(%.2f%%)' % (label.score*100.))
Vision: label annotation/object detection
$ python3 label-detect.py
Labels (and confidence score):
==============================
People (95.05%)
Street (89.12%)
Mode of transport (89.09%)
Transport (85.13%)
Vehicle (84.69%)
Snapshot (84.11%)
Urban area (80.29%)
Infrastructure (73.14%)
Road (72.74%)
Pedestrian (68.90%)
Vision: label annotation/object detection
g.co/codelabs/vision-python

● Not just for conversations
● Create microservice utilities
● Build chat bots to...
○ Automate workflows
○ Query for information
○ Other heavy-lifting
● Plain text or rich UI "cards"
● Very flexible ("any")
development environment
○ POST to HTTP port
Hangouts Chat bots
(bot framework & API)
"Hello World" (echo bot)
Python+Flask: GAE or other hosting
from flask import Flask, request, json
@app.route('/', methods=['POST'])
def on_event():
event = request.get_json()
msg = {}
if event['type'] == 'MESSAGE':
text = 'Hi {}. You sent: {}'.format(
event['user']['displayName'], event['message']['text'])
msg = {'text': text}
return json.jsonify(msg)
Hangouts Chat bots
goo.gl/jt3FqK

Search YouTube for videos
from __future__ import print_function
from googleapiclient import discovery
from settings import API_KEY
QUERY = 'python -snake'
trim = lambda x, ct: ('%s%s' % (x[:ct],
'...' if len(x)>ct else '')).ljust(ct+3)
print('n** Searching for %r videos...' % QUERY)
YOUTUBE = discovery.build('youtube', 'v3', developerKey=API_KEY)
res = YOUTUBE.search().list(q=QUERY, type='video',
part='id,snippet').execute().get('items', [])
for item in res:
print('http://youtu.be/%st%s' % (
trim(item['id']['videoId'], 24),
trim(item['snippet']['title'], 48)))
Maps APIs geocoding & places queries
import googlemaps
from settings import API_KEY
GMAPS = googlemaps.Client(key=API_KEY)
print('n** Geocode address:')
rsp = GMAPS.geocode('1600 Amphitheatre Pkwy 94043')
latlong = rsp[0]['geometry']['location']
print('tGeocode:', latlong['lat'], ',', latlong['lng'])
print('n** Address lookup:')
rsp = GMAPS.reverse_geocode((37.4222934, -122.0841409))
print('tAddress:', rsp[0]['formatted_address'])
print('n** Place query:')
pl_id = GMAPS.find_place('Villa Tugendhat',
input_type='textquery')['candidates'][0]['place_id']
place = GMAPS.place(pl_id)['result']
print('tPlace:t', place['name'])
print('tAddress:t', place['formatted_address'])
print('tWebsite:t', place['website'])
print('tMaps URL:t', place['url'])
$ python3 maps-demo-pub.py
** Geocode address:
Geocode: 37.4222934 , -122.0841409
** Address lookup:
Address: 1600 Amphitheatre Pkwy, Mountain View, CA
94043, USA
** Place query:
Place: Villa Tugendhat
Address: Černopolní 45, 613 00 Brno, Czechia
Website: tugendhat.eu
Maps URL: maps.google.com/?cid=5889127146370224362
Google Maps APIs
● Directions API
● Distance Matrix API
● Elevation API
● Geocoding API
● Geolocation API
● Time Zone API
● Roads API
● Places API
● Maps Static API

Other Google APIs & platforms
● Firebase (mobile development platform + RT DB; ML Kit)
○ firebase.google.com & firebase.google.com/docs/ml-kit
● Google Data Studio (data visualization, dashboards, etc.)
○ datastudio.google.com/overview
○ goo.gle/datastudio-course
● Actions on Google/Assistant/DialogFlow (voice apps)
○ developers.google.com/actions
● YouTube (Data, Analytics, and Livestreaming APIs)
○ developers.google.com/youtube
● Google Maps (Maps, Routes, and Places APIs)
○ developers.google.com/maps
● Flutter (native apps [Android, iOS, web] w/1 code base[!])
○ flutter.dev
04
All of Cloud
(inspiration)
Build powerful solutions
with GCP and G Suite

Cloud image processing workflow
Archive and analyze Google Workspace
(formerly G Suite) images with GCP

Cloud
Vision
Google
Workspac
e
(formerly
G Suite)
GCP
Cloud
Storage
Drive
Sheets
Archive
image
Categorize
image
Record
results

def drive_get_file(fname):
rsp = DRIVE.files().list(q="name='%s'" % fname).execute().get['files'][0]
fileId, fname, mtype = rsp['id'], rsp['name'], rsp['mimeType']
blob = DRIVE.files().get_media(fileId).execute()
return fname, mtype, rsp['modifiedTime'], blob
def gcs_blob_upload(fname, bucket, blob, mimetype):
body = {'name': fname, 'uploadType': 'multipart',
'contentType': mimetype}
return GCS.objects().insert(bucket, body, blob).execute()
def vision_label_img(img, top):
body = {'requests': [{'image': {'content': img}, 'features':
[{'type': 'LABEL_DETECTION', 'maxResults': top}]}]}
rsp = VISION.images().annotate(
body=body).execute().get['responses'][0]
return ', '.join('%s (%.2f%%)' % (label['description'],
label['score']*100.) for label in rsp['labelAnnotations'])
def sheet_append_rows(sheet, rows):
rsp = SHEETS.spreadsheets().values().append(
spreadsheetId=sheet, range='Sheet1',
body={'values': rows}).execute()
return rsp.get('updates').get('updatedCells')
def main(fname, bucket, sheet_id, folder, top):
fname, mtype, ftime, data = drive_get_img(fname)
gcs_blob_upload(fname, bucket, data, mtype)
vision_label_img(data, top)
sheet_append_row([sheet_id, fname, mtype,
ftime, len(data), rsp])
API method calls in Bold
Driver calls in Bold Italics
● Project goal: Imagine enterprise use cases Workspace & GCP; got one!
● Specific goals: free-up highly-utilized resource, archive data to
colder/cheaper storage, analyze images, generate report for mgmt
● Download image binary from Google Drive
● Upload object to Cloud Storage bucket
● Send payload for analysis by Cloud Vision
● Write back-up location & analysis results into Google Sheets
● Blog post: goo.gle/3nPxmlc (original post); Cloud X-post
● Codelab: self-paced (1+-hour) hands-on tutorial
● g.co/codelabs/drive-gcs-vision-sheets
● Application source code
● github.com/googlecodelabs/analyze_gsimg
App summary

05
Wrap-up
Summary & resources
Session Summary
● Google provides more than just apps
○ We're more than search, YouTube, Android, Chrome, and Gmail
○ Much of our tech available to developers through our APIs
● Tour of Google (Cloud) APIs & developer tools
○ Workspace: not just a set of productivity apps… you can code them too!
○ GCP: compute, storage, networking, security, data & machine learning tools
■ Serverless: frees developers from infrastructure
■ So you can focus on building solutions
● Interesting possibilities using ALL of Cloud (GCP + Workspace)
● Also explore other Google developer products & APIs

● Documentation
○ GCP: cloud.google.com/{docs,appengine,functions,run,vision,automl,translate,language,
speech,texttospeech,video-intelligence,firestore,bigquery,compute,storage,gpu,tpu}
○ Workspace: developers.google.com/{gsuite,drive,calendar,gmail,docs,sheets,slides,apps-script}
● Introductory "codelabs" ([free] self-paced, hands-on tutorials)
○ Workspace APIs: g.co/codelabs/gsuite-apis-intro (featuring Drive API)
○ Apps Script: g.co/codelabs/apps-script-intro
○ App Engine: codelabs.developers.google.com/codelabs/cloud-app-engine-python
○ Cloud Functions: codelabs.developers.google.com/codelabs/cloud-starting-cloudfunctions
○ Cloud Run: codelabs.developers.google.com/codelabs/cloud-run-hello-python3
○ Others: g.co/codelabs (all Google codelabs) and gcplab.me (GCP-only)
● Videos: youtube.com/GoogleCloudPlatform (GCP) and goo.gl/JpBQ40 (Google Workspace)
● Code samples: github.com/GoogleCloudPlatform (GCP) and github.com/gsuitedevs (Google Workspace)
● Free trial (new users) and Always Free (daily/monthly tier) - cloud.google.com/free
● Know AWS/Azure? Compare w/GCP at cloud.google.com/docs/compare/{aws,azure}
Resources (industry)
Thank you! Questions?
Wesley Chun
@wescpy
Progress bars: goo.gl/69EJVw
Slides: bit.ly/2SHybQg

Exploring Google APIs with Python

Related slideshows

More Related Content

Exploring Google APIs with Python