SlideShare a Scribd company logo

Image archive, analysis & report generation with Google Cloud

•
•
wesley chun
wesley chun

Google Cloud provides a diverse array of services to realize the ambition of solving real business problems, like constrained resources. An image archive & analysis plus report generation use-case can be realized with just Google Workspace & GCP APIs. The principle of mixing-and-matching Google technologies is applicable to many other challenges faced by you, your organization, or your customers. These slides are from a half- to 1-hour presentation about this case study.

1 of 25
Download to read offline
Image archive, analysis & report generation with Google Cloud Wesley Chun - @wescpy Developer Advocate, Google Adjunct CS Faculty, Foothill College Why and Agenda ● Organizations have real-life business problems seeking solutions ● Google Cloud (the organization) produces 2 main product groups ○ Google Cloud Platform (GCP) and Google Workspace (ex G Suite) ● May know GCP for compute, storage, data & AI/ML cloud services ● While Workspace known for its apps... also for developers(!) ● Use both to build novel solutions to unique business problems 1 Google APIs intro 2 Google Cloud APIs 3 Image processing workflow 4 More inspiration 5 Wrap-up
Developer Advocate, Google Cloud ● Mission: enable current and future developers everywhere to be successful using Google Cloud and other Google developer tools & APIs ● Focus: GCP serverless (App Engine, Cloud Functions, Cloud Run); higher education, Google Workspace, GCP AI/ML APIs; multi-product use cases ● Content: speak to developers globally; make videos, create code samples, produce codelabs (free, self-paced, hands-on tutorials), publish blog posts About the speaker Previous experience / background ● Software engineer & architect for 20+ years ○ Yahoo!, Sun, HP, Cisco, EMC, Xilinx ○ Original Yahoo!Mail engineer/SWE ● Technical trainer, teacher, instructor ○ Taught Math, Linux, Python since 1983 ○ Private corporate trainer ○ Adjunct CS Faculty at local SV college ● Python community member ○ Popular Core Python series author ○ Python Software Foundation Fellow ● AB (Math/CS) & CMP (Music/Piano), UC Berkeley and MSCS, UC Santa Barbara ● Adjunct Computer Science Faculty, Foothill College (Silicon Valley) 01 Introduction to Google APIs Why are you here?
Image archive, analysis & report generation with Google Cloud
General steps 1. Go to Cloud Console 2. Login to Google/Gmail account (Workspace domain may require admin approval) 3. Create project (per application) 4. Enable APIs to use 5. Enable billing (CC, Free Trial, etc.) 6. Download client library(ies) 7. Create & download credentials 8. Write code 9. Run code (may need to authorize) Google APIs: how to use Costs and pricing ● GCP: pay-per-use ● Google Workspace: subscription ● GCP Free Trial ($300/1Q, CC req'd) ● GCP "Always Free" tier ○ Most products have free tier ○ Daily or monthly quota ○ Must exceed to incur billing ● More on both programs at cloud.google.com/free Cloud/GCP console console.cloud.google.com ● Hub of all developer activity ● Applications == projects ○ New project for new apps ○ Projects have a billing acct ● Manage billing accounts ○ Financial instrument required ○ Personal or corporate credit cards, Free Trial, and education grants ● Access GCP product settings ● Manage users & security ● Manage APIs in devconsole
● View application statistics ● En-/disable Google APIs ● Obtain application credentials Using Google APIs goo.gl/RbyTFD API manager aka Developers Console (devconsole) console.developers.google.com Three different authz credential types ● Simple: API keys (to access public data) ○ Simplest form of authorization: an API key; tied to a project ○ Allows access to public data ○ Do not put in code, lose, or upload to GitHub! (can be restricted however) ○ Supported by: Google Maps, (some) YouTube, (some) GCP, etc. ● Authorized: OAuth client IDs (to access data owned by [human] user) ○ Provides additional layer of security via OAuth2 (RFC 6749) ○ Owner must grant permission for your app to access their data ○ Access granularity determined by requested permissions (user scopes) ○ Supported by: Google Workspace, (some) YouTube, (some) GCP, etc. ● Authorized: service accounts (to access data owned by an app/robot user) ○ Provides additional layer of security via OAuth2 or JWT (RFC 7519) ○ Project owning data grants permission implicitly; requires public-private key-pair ○ Access granularity determined by Cloud IAM permissions granted to service account key-pair ○ Supported by: GCP, (some) Google Workspace, etc.
Google APIs client libraries for common languages; demos in developers.google.com/api- client-library cloud.google.com/apis/docs /cloud-client-libraries ● Why Google Believes in Open Cloud (Jun 2018 blog post & home page) ○ cloud.google.com/blog/products/gcp/why-google-believes-in-open-cloud ○ cloud.google.com/open-cloud ● Bringing the Best of Open Source to Google Cloud customers (Apr 2019) ○ Strategic partnerships w/leading {open source,data}-centric organizations ■ Confluent, DataStax, Elastic, InfluxData, MongoDB, Neo4j, Redis Labs ○ cloud.google.com/blog/products/open-source/bringing-the-best-of-open- source-to-google-cloud-customers ○ tcrn.ch/2Z3z1qS ● Google Cloud open source blog ○ cloud.google.com/blog/products/open-source Google Cloud open source references
Two different client library "styles" ● "Platform-level" client libraries (lower-level) ○ Supports multiple products as a "lowest-common denominator" ○ Manage API service endpoints (setup & use) ○ Manage authorization (API keys, OAuth client IDs, service accounts) ○ Google Workspace, Google Analytics, YouTube, Google Ads APIs, etc. ○ Install: developers.google.com/api-client-library ● "Product-level" client libraries (higher-level) ○ Custom client libraries made specifically for each product ○ Managing API service endpoints & security mostly taken care of ○ Only need to create a "client" to use API services ○ Install (Cloud/GCP & Firebase): cloud.google.com/apis/docs/cloud-client-libraries ○ Install (Maps): developers.google.com/places/web-service/client-library ● Some Google APIs families support both, e.g., Cloud OAuth2 or API key HTTP-based REST APIs 1 HTTP 2 Google APIs request-response workflow ● Application makes request ● Request received by service ● Process data, return response ● Results sent to application (typical client-server model)
02 Google Cloud Developer tools & APIs formerly ( )
Google Workspace Top-level documentation and comprehensive developers overview video at developers.google.com/gsuite (formerly G Suite and Google Apps) APIs Google Compute Engine, Google Cloud Storage AWS EC2 & S3; Rackspace; Joyent SaaS Software as a Service PaaS Platform as a Service IaaS Infrastructure as a Service Google Apps Script Salesforce1/force.com Google Workspace (was G Suite/Google Apps) Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365 Google App Engine, Cloud Functions Heroku, Cloud Foundry, Engine Yard, AWS Lambda Google BigQuery, Cloud SQL, Cloud Datastore, NL, Vision, Pub/Sub AWS Kinesis, RDS; Windows Azure SQL, Docker Google Cloud Platform vs. Google Workspace Workspace APIs GCP APIs
List (first 100) files/folders in Google Drive from __future__ import print_function from googleapiclient import discovery from httplib2 import Http from oauth2client import file, client, tools SCOPES = 'https://www.googleapis.com/auth/drive.metadata.readonly' store = file.Storage('storage.json') creds = store.get() if not creds or creds.invalid: flow = client.flow_from_clientsecrets('client_secret.json', SCOPES) creds = tools.run_flow(flow, store) DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http())) files = DRIVE.files().list().execute().get('files', []) for f in files: print(f['name'], f['mimeType']) Listing your files goo.gl/ZIgf8k Migrate SQL data to a Sheet # read SQL data then create new spreadsheet & add rows into it FIELDS = ('ID', 'Customer Name', 'Product Code', 'Units Ordered', 'Unit Price', 'Status') cxn = sqlite3.connect('db.sqlite') cur = cxn.cursor() rows = cur.execute('SELECT * FROM orders').fetchall() cxn.close() rows.insert(0, FIELDS) DATA = {'properties': {'title': 'Customer orders'}} SHEET_ID = SHEETS.spreadsheets().create(body=DATA, fields='spreadsheetId').execute().get('spreadsheetId') SHEETS.spreadsheets().values().update(spreadsheetId=SHEET_ID, range='A1', body={'values': rows}, valueInputOption='RAW').execute() Migrate SQL data to Sheets goo.gl/N1RPwC
Storage: listing buckets from __future__ import print_function from googleapiclient import discovery GCS = discovery.build('storage', 'v1') BUCKET = YOUR_BUCKET # send bucket name & return fields to API, display results print('n** Objects in bucket %r...' % BUCKET) FIELDS = 'items(name,size)' files = GCS.objects().list(bucket=BUCKET, fields=FIELDS ).execute().get('items') or [{'name': '(none)', 'size': 'NaN'}] for f in files: print(' %s (%s)' % (f['name'], f['size'])) Vision: image analysis & metadata extraction # set Vision API request body and create endpoint to API IMG = 'https://google.com/services/images/section-work-card-img_2x.jpg' body = {'requests': [ { 'image': { 'source': {'imageUri': IMG}, }, 'features': [ {'type': 'LABEL_DETECTION'}, {'type': 'FACE_DETECTION'}, ], }, ]} VISION = discovery.build('vision', 'v1', developerKey=API_KEY)
labeling = VISION.images().annotate(body=body).execute().get('responses') for labels in labeling: if 'labelAnnotations' in labels: print('** Labels detected (and confidence score):') for label in labels['labelAnnotations']: print(' {0:.2f}%t{1}'.format(label['score']*100., label['description'])) if 'faceAnnotations' in labels: print('n** Facial features detected (and likelihood):') for label, value in labels['faceAnnotations'][0].items(): if label.endswith('Likelihood'): print(' {}:t{}'.format( label.split('Likelihood')[0].title(), value.lower().replace('_', ' '))) Vision: image analysis & metadata extraction $ python viz_demo.py ** Labels detected (and confidence score): 96.84% Table 95.62% Furniture 95.35% Smile 93.17% Window 91.78% Tableware 88.44% Lamp 86.70% Comfort 84.65% Interior design 83.56% Sky 83.16% Desk ** Facial features detected (and likelihood): Joy: very likely Sorrow: very unlikely Anger: very unlikely Surprise: very unlikely Underexposed: very unlikely Blurred: very unlikely Headwear: very unlikely Vision: image analysis & metadata extraction g.co/codelabs/vision-python
03 Image processing workflow Main use case
Image archive, analysis & report generation with Google Cloud
Image: Gerd Altmann from Pixabay
Cloud Vision Google Workspace GCP Cloud image processing workflow Cloud Storage Drive Sheets Archive image Categorize image Record results Cloud image processing workflow from __future__ import print_function import argparse, base64, io, webbrowser from googleapiclient import discovery, http from httplib2 import Http from oauth2client import file, client, tools k_ize = lambda b: '%6.2fK' % (b/1000.) # bytes to kBs FILE = 'YOUR_IMG_ON_DRIVE' BUCKET = 'YOUR_BUCKET_NAME' PARENT = '' # YOUR IMG FILE PREFIX SHEET = 'YOUR_SHEET_ID' TOP = 5 # TOP # of VISION LABELS TO SAVE DEBUG = False # process credentials for OAuth2 tokens SCOPES = ( 'https://www.googleapis.com/auth/drive.readonly', 'https://www.googleapis.com/auth/devstorage', 'https://www.googleapis.com/auth/cloud-vision', 'https://www.googleapis.com/auth/spreadsheets', ) store = file.Storage('storage.json') creds = store.get() if not creds or creds.invalid: flow = client.flow_from_clientsecrets( 'client_secret.json', SCOPES) creds = tools.run_flow(flow, store) # create API service endpoints HTTP = creds.authorize(Http()) DRIVE = discovery.build('drive', 'v3', http=HTTP) GCS = discovery.build('storage', 'v1', http=HTTP) VISION = discovery.build('vision', 'v1', http=HTTP) SHEETS = discovery.build('sheets', 'v4', http=HTTP)
Cloud image processing workflow def drive_get_file(fname): rsp = DRIVE.files().list(q="name='%s'" % fname).execute().get['files'][0] fileId, fname, mtype = rsp['id'], rsp['name'], rsp['mimeType'] blob = DRIVE.files().get_media(fileId).execute() return mtype, rsp['modifiedTime'], blob def gcs_blob_upload(fname, bucket, blob, mimetype): body = {'name': fname, 'uploadType': 'multipart', 'contentType': mimetype} return GCS.objects().insert(bucket, body, blob).execute() def vision_label_img(img, top): body = {'requests': [{'image': {'content': img}, 'features': [{'type': 'LABEL_DETECTION', 'maxResults': top}]}]} rsp = VISION.images().annotate( body=body).execute().get['responses'][0] return ', '.join('%s (%.2f%%)' % (label['description'], label['score']*100.) for label in rsp['labelAnnotations']) def sheet_append_row(sheet, row): rsp = SHEETS.spreadsheets().values().append( spreadsheetId=sheet, range='Sheet1', body={'values': rows}).execute() return rsp.get('updates').get('updatedCells') def main(fname, bucket, sheet_id, top): mtype, ftime, data = drive_get_img(fname) gcs_blob_upload(fname, bucket, data, mtype) rsp = vision_label_img(data, top) sheet_append_row(sheet_id, [fname, mtype, ftime, len(data), rsp]) API method calls in Bold Driver calls in Bold Italics Cloud image processing workflow def drive_get_file(fname): rsp = DRIVE.files().list( q="name='%s'" % fname).execute().get['files'][0] fileId, fname, mtype = rsp['id'], rsp['name'], rsp['mimeType'] blob = DRIVE.files().get_media(fileId).execute() return mtype, rsp['modifiedTime'], blob API method calls in Bold Driver calls in Bold Italics
Cloud image processing workflow def gcs_blob_upload(fname, bucket, blob, mimetype): body = { 'name': fname, 'uploadType': 'multipart', 'contentType': mimetype } return GCS.objects().insert( bucket, body, blob).execute() Cloud image processing workflow def vision_label_img(img, top): body = {'requests': [{'image': { 'content': img}, 'features': [ {'type': 'LABEL_DETECTION', 'maxResults': top}]}]} rsp = VISION.images().annotate( body=body).execute().get['responses'][0] return ', '.join('%s (%.2f%%)' % ( label['description'], label['score']*100.) for label in rsp['labelAnnotations'])
Cloud image processing workflow def sheet_append_row(sheet, row): rsp = SHEETS.spreadsheets().values().append( spreadsheetId=sheet, range='Sheet1', body={'values': row}).execute() return rsp.get('updates').get('updatedCells') API method calls in Bold Driver calls in Bold Italics Cloud image processing workflow def main(fname, bucket, sheet_id, top): fname, mtype, ftime, data = drive_get_img(fname) gcs_blob_upload(fname, bucket, data, mtype) rsp = vision_label_img(data, top) sheet_append_row(sheet_id, [fname, mtype, ftime, len(data), rsp]) API method calls in Bold Driver calls in Bold Italics
● Project goal: Imagining an actual enterprise use case and solve it! ● Specific goals: free-up highly-utilized resource, archive data to colder/cheaper storage, analyze images, generate report for mgmt ● Download image binary from Google Drive ● Upload object to Cloud Storage bucket ● Send payload for analysis by Cloud Vision ● Write back-up location & analysis results into Google Sheets ● Blog post: goo.gle/3nPxmlc (original post); Cloud X-post ● Codelab: free, online, self-paced, hands-on tutorial ● g.co/codelabs/drive-gcs-vision-sheets ● Application source code ● github.com/googlecodelabs/analyze_gsimg App summary 04 More Inspiration Build powerful solutions with GCP and G Suite
Custom intelligence in Gmail Analyze Google Workspace (formerly G Suite) data with GCP
Gmail message processing with GCP Gmail Cloud Pub/Sub Cloud Functions Cloud Vision Workspace (formerly G Suite) GCP Star message Message notification Trigger function Extract images Categorize images Inbox augmented with Cloud Function
● Gmail API: sets up notification forwarding to Cloud Pub/Sub ● developers.google.com/gmail/api/guides/push ● Pub/Sub: triggers logic hosted by Cloud Functions ● cloud.google.com/functions/docs/calling/pubsub ● Cloud Functions: "orchestrator" accessing GCP (and Google Workspace/G Suite) APIs ● Combine all of the above to add custom intelligence to Gmail ● Deep dive code blog post ● cloud.google.com/blog/products/application-development/ adding-custom-intelligence-to-gmail-with-serverless-on-gcp ● Application source code ● github.com/GoogleCloudPlatform/cloud-functions-gmail-nodejs App summary 05 Wrap-up Summary & resources
Session Summary ● Google provides more than just apps ○ We're more than search, YouTube, Android, Chrome, and Gmail ○ Much of our tech available to developers through our APIs ● Tour of Google (Cloud) APIs & developer tools ○ Workspace: not just a set of productivity apps… you can code them too! ○ GCP: compute, storage, networking, security, data & machine learning tools ○ Google Cloud serverless frees developers from infrastructure ■ 4 unique services so you can focus on building solutions ● Interesting possibilities using both GCP + Workspace together ● Other Google developer products to consider Other Google APIs & platforms ● Firebase (mobile development platform + RT DB; ML Kit) ○ firebase.google.com & firebase.google.com/docs/ml-kit ● Google Data Studio (data visualization, dashboards, etc.) ○ datastudio.google.com/overview ○ goo.gle/datastudio-course ● Actions on Google/Assistant/DialogFlow (voice apps) ○ developers.google.com/actions ● YouTube (Data, Analytics, and Livestreaming APIs) ○ developers.google.com/youtube ● Google Maps (Maps, Routes, and Places APIs) ○ developers.google.com/maps ● Flutter (native apps [Android, iOS, web] w/1 code base[!]) ○ flutter.dev
● Documentation ○ GCP: cloud.google.com/{docs,appengine,functions,vision,automl,sql,compute.storage, language,speech,text-to-speech,translate,video-intelligence,firestore,bigquery,filestore} ○ Workspace: developers.google.com/{gsuite,drive,calendar,gmail,docs,sheets,slides,apps-script} ● Introductory "codelabs" ([free] self-paced, hands-on tutorials) ○ Workspace REST APIs: g.co/codelabs/gsuite-apis-intro (featuring Drive API) ○ Apps Script: g.co/codelabs/apps-script-intro ○ App Engine: codelabs.developers.google.com/codelabs/cloud-app-engine-python ○ Cloud Vision: g.co/codelabs/vision-python (or C# or Ruby) ○ Cloud Functions: codelabs.developers.google.com/codelabs/cloud-starting-cloudfunctions ○ All others: gcplab.me (GCP) and g.co/codelabs (non-GCP) ● Videos: youtube.com/GoogleCloudPlatform (GCP), goo.gl/JpBQ40 (Workspace), and all others ● Code samples: github.com/GoogleCloudPlatform (GCP) and github.com/gsuitedevs (Workspace) ● GCP Free trial (new users) and Always Free (daily/monthly tier) - cloud.google.com/free Know AWS/Azure? Compare w/GCP: cloud.google.com/docs/compare/{aws,azure} Online resources Thank you! Questions? Wesley Chun @wescpy Codelab: g.co/codelabs/drive-gcs-vision-sheets Progress bars: goo.gl/69EJVw Slides: bit.ly/2SHybQg

More Related Content

Image archive, analysis & report generation with Google Cloud

  • 1. Image archive, analysis & report generation with Google Cloud Wesley Chun - @wescpy Developer Advocate, Google Adjunct CS Faculty, Foothill College Why and Agenda ● Organizations have real-life business problems seeking solutions ● Google Cloud (the organization) produces 2 main product groups ○ Google Cloud Platform (GCP) and Google Workspace (ex G Suite) ● May know GCP for compute, storage, data & AI/ML cloud services ● While Workspace known for its apps... also for developers(!) ● Use both to build novel solutions to unique business problems 1 Google APIs intro 2 Google Cloud APIs 3 Image processing workflow 4 More inspiration 5 Wrap-up
  • 2. Developer Advocate, Google Cloud ● Mission: enable current and future developers everywhere to be successful using Google Cloud and other Google developer tools & APIs ● Focus: GCP serverless (App Engine, Cloud Functions, Cloud Run); higher education, Google Workspace, GCP AI/ML APIs; multi-product use cases ● Content: speak to developers globally; make videos, create code samples, produce codelabs (free, self-paced, hands-on tutorials), publish blog posts About the speaker Previous experience / background ● Software engineer & architect for 20+ years ○ Yahoo!, Sun, HP, Cisco, EMC, Xilinx ○ Original Yahoo!Mail engineer/SWE ● Technical trainer, teacher, instructor ○ Taught Math, Linux, Python since 1983 ○ Private corporate trainer ○ Adjunct CS Faculty at local SV college ● Python community member ○ Popular Core Python series author ○ Python Software Foundation Fellow ● AB (Math/CS) & CMP (Music/Piano), UC Berkeley and MSCS, UC Santa Barbara ● Adjunct Computer Science Faculty, Foothill College (Silicon Valley) 01 Introduction to Google APIs Why are you here?
  • 4. General steps 1. Go to Cloud Console 2. Login to Google/Gmail account (Workspace domain may require admin approval) 3. Create project (per application) 4. Enable APIs to use 5. Enable billing (CC, Free Trial, etc.) 6. Download client library(ies) 7. Create & download credentials 8. Write code 9. Run code (may need to authorize) Google APIs: how to use Costs and pricing ● GCP: pay-per-use ● Google Workspace: subscription ● GCP Free Trial ($300/1Q, CC req'd) ● GCP "Always Free" tier ○ Most products have free tier ○ Daily or monthly quota ○ Must exceed to incur billing ● More on both programs at cloud.google.com/free Cloud/GCP console console.cloud.google.com ● Hub of all developer activity ● Applications == projects ○ New project for new apps ○ Projects have a billing acct ● Manage billing accounts ○ Financial instrument required ○ Personal or corporate credit cards, Free Trial, and education grants ● Access GCP product settings ● Manage users & security ● Manage APIs in devconsole
  • 5. ● View application statistics ● En-/disable Google APIs ● Obtain application credentials Using Google APIs goo.gl/RbyTFD API manager aka Developers Console (devconsole) console.developers.google.com Three different authz credential types ● Simple: API keys (to access public data) ○ Simplest form of authorization: an API key; tied to a project ○ Allows access to public data ○ Do not put in code, lose, or upload to GitHub! (can be restricted however) ○ Supported by: Google Maps, (some) YouTube, (some) GCP, etc. ● Authorized: OAuth client IDs (to access data owned by [human] user) ○ Provides additional layer of security via OAuth2 (RFC 6749) ○ Owner must grant permission for your app to access their data ○ Access granularity determined by requested permissions (user scopes) ○ Supported by: Google Workspace, (some) YouTube, (some) GCP, etc. ● Authorized: service accounts (to access data owned by an app/robot user) ○ Provides additional layer of security via OAuth2 or JWT (RFC 7519) ○ Project owning data grants permission implicitly; requires public-private key-pair ○ Access granularity determined by Cloud IAM permissions granted to service account key-pair ○ Supported by: GCP, (some) Google Workspace, etc.
  • 6. Google APIs client libraries for common languages; demos in developers.google.com/api- client-library cloud.google.com/apis/docs /cloud-client-libraries ● Why Google Believes in Open Cloud (Jun 2018 blog post & home page) ○ cloud.google.com/blog/products/gcp/why-google-believes-in-open-cloud ○ cloud.google.com/open-cloud ● Bringing the Best of Open Source to Google Cloud customers (Apr 2019) ○ Strategic partnerships w/leading {open source,data}-centric organizations ■ Confluent, DataStax, Elastic, InfluxData, MongoDB, Neo4j, Redis Labs ○ cloud.google.com/blog/products/open-source/bringing-the-best-of-open- source-to-google-cloud-customers ○ tcrn.ch/2Z3z1qS ● Google Cloud open source blog ○ cloud.google.com/blog/products/open-source Google Cloud open source references
  • 7. Two different client library "styles" ● "Platform-level" client libraries (lower-level) ○ Supports multiple products as a "lowest-common denominator" ○ Manage API service endpoints (setup & use) ○ Manage authorization (API keys, OAuth client IDs, service accounts) ○ Google Workspace, Google Analytics, YouTube, Google Ads APIs, etc. ○ Install: developers.google.com/api-client-library ● "Product-level" client libraries (higher-level) ○ Custom client libraries made specifically for each product ○ Managing API service endpoints & security mostly taken care of ○ Only need to create a "client" to use API services ○ Install (Cloud/GCP & Firebase): cloud.google.com/apis/docs/cloud-client-libraries ○ Install (Maps): developers.google.com/places/web-service/client-library ● Some Google APIs families support both, e.g., Cloud OAuth2 or API key HTTP-based REST APIs 1 HTTP 2 Google APIs request-response workflow ● Application makes request ● Request received by service ● Process data, return response ● Results sent to application (typical client-server model)
  • 8. 02 Google Cloud Developer tools & APIs formerly ( )
  • 9. Google Workspace Top-level documentation and comprehensive developers overview video at developers.google.com/gsuite (formerly G Suite and Google Apps) APIs Google Compute Engine, Google Cloud Storage AWS EC2 & S3; Rackspace; Joyent SaaS Software as a Service PaaS Platform as a Service IaaS Infrastructure as a Service Google Apps Script Salesforce1/force.com Google Workspace (was G Suite/Google Apps) Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365 Google App Engine, Cloud Functions Heroku, Cloud Foundry, Engine Yard, AWS Lambda Google BigQuery, Cloud SQL, Cloud Datastore, NL, Vision, Pub/Sub AWS Kinesis, RDS; Windows Azure SQL, Docker Google Cloud Platform vs. Google Workspace Workspace APIs GCP APIs
  • 10. List (first 100) files/folders in Google Drive from __future__ import print_function from googleapiclient import discovery from httplib2 import Http from oauth2client import file, client, tools SCOPES = 'https://www.googleapis.com/auth/drive.metadata.readonly' store = file.Storage('storage.json') creds = store.get() if not creds or creds.invalid: flow = client.flow_from_clientsecrets('client_secret.json', SCOPES) creds = tools.run_flow(flow, store) DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http())) files = DRIVE.files().list().execute().get('files', []) for f in files: print(f['name'], f['mimeType']) Listing your files goo.gl/ZIgf8k Migrate SQL data to a Sheet # read SQL data then create new spreadsheet & add rows into it FIELDS = ('ID', 'Customer Name', 'Product Code', 'Units Ordered', 'Unit Price', 'Status') cxn = sqlite3.connect('db.sqlite') cur = cxn.cursor() rows = cur.execute('SELECT * FROM orders').fetchall() cxn.close() rows.insert(0, FIELDS) DATA = {'properties': {'title': 'Customer orders'}} SHEET_ID = SHEETS.spreadsheets().create(body=DATA, fields='spreadsheetId').execute().get('spreadsheetId') SHEETS.spreadsheets().values().update(spreadsheetId=SHEET_ID, range='A1', body={'values': rows}, valueInputOption='RAW').execute() Migrate SQL data to Sheets goo.gl/N1RPwC
  • 11. Storage: listing buckets from __future__ import print_function from googleapiclient import discovery GCS = discovery.build('storage', 'v1') BUCKET = YOUR_BUCKET # send bucket name & return fields to API, display results print('n** Objects in bucket %r...' % BUCKET) FIELDS = 'items(name,size)' files = GCS.objects().list(bucket=BUCKET, fields=FIELDS ).execute().get('items') or [{'name': '(none)', 'size': 'NaN'}] for f in files: print(' %s (%s)' % (f['name'], f['size'])) Vision: image analysis & metadata extraction # set Vision API request body and create endpoint to API IMG = 'https://google.com/services/images/section-work-card-img_2x.jpg' body = {'requests': [ { 'image': { 'source': {'imageUri': IMG}, }, 'features': [ {'type': 'LABEL_DETECTION'}, {'type': 'FACE_DETECTION'}, ], }, ]} VISION = discovery.build('vision', 'v1', developerKey=API_KEY)
  • 12. labeling = VISION.images().annotate(body=body).execute().get('responses') for labels in labeling: if 'labelAnnotations' in labels: print('** Labels detected (and confidence score):') for label in labels['labelAnnotations']: print(' {0:.2f}%t{1}'.format(label['score']*100., label['description'])) if 'faceAnnotations' in labels: print('n** Facial features detected (and likelihood):') for label, value in labels['faceAnnotations'][0].items(): if label.endswith('Likelihood'): print(' {}:t{}'.format( label.split('Likelihood')[0].title(), value.lower().replace('_', ' '))) Vision: image analysis & metadata extraction $ python viz_demo.py ** Labels detected (and confidence score): 96.84% Table 95.62% Furniture 95.35% Smile 93.17% Window 91.78% Tableware 88.44% Lamp 86.70% Comfort 84.65% Interior design 83.56% Sky 83.16% Desk ** Facial features detected (and likelihood): Joy: very likely Sorrow: very unlikely Anger: very unlikely Surprise: very unlikely Underexposed: very unlikely Blurred: very unlikely Headwear: very unlikely Vision: image analysis & metadata extraction g.co/codelabs/vision-python
  • 15. Image: Gerd Altmann from Pixabay
  • 16. Cloud Vision Google Workspace GCP Cloud image processing workflow Cloud Storage Drive Sheets Archive image Categorize image Record results Cloud image processing workflow from __future__ import print_function import argparse, base64, io, webbrowser from googleapiclient import discovery, http from httplib2 import Http from oauth2client import file, client, tools k_ize = lambda b: '%6.2fK' % (b/1000.) # bytes to kBs FILE = 'YOUR_IMG_ON_DRIVE' BUCKET = 'YOUR_BUCKET_NAME' PARENT = '' # YOUR IMG FILE PREFIX SHEET = 'YOUR_SHEET_ID' TOP = 5 # TOP # of VISION LABELS TO SAVE DEBUG = False # process credentials for OAuth2 tokens SCOPES = ( 'https://www.googleapis.com/auth/drive.readonly', 'https://www.googleapis.com/auth/devstorage', 'https://www.googleapis.com/auth/cloud-vision', 'https://www.googleapis.com/auth/spreadsheets', ) store = file.Storage('storage.json') creds = store.get() if not creds or creds.invalid: flow = client.flow_from_clientsecrets( 'client_secret.json', SCOPES) creds = tools.run_flow(flow, store) # create API service endpoints HTTP = creds.authorize(Http()) DRIVE = discovery.build('drive', 'v3', http=HTTP) GCS = discovery.build('storage', 'v1', http=HTTP) VISION = discovery.build('vision', 'v1', http=HTTP) SHEETS = discovery.build('sheets', 'v4', http=HTTP)
  • 17. Cloud image processing workflow def drive_get_file(fname): rsp = DRIVE.files().list(q="name='%s'" % fname).execute().get['files'][0] fileId, fname, mtype = rsp['id'], rsp['name'], rsp['mimeType'] blob = DRIVE.files().get_media(fileId).execute() return mtype, rsp['modifiedTime'], blob def gcs_blob_upload(fname, bucket, blob, mimetype): body = {'name': fname, 'uploadType': 'multipart', 'contentType': mimetype} return GCS.objects().insert(bucket, body, blob).execute() def vision_label_img(img, top): body = {'requests': [{'image': {'content': img}, 'features': [{'type': 'LABEL_DETECTION', 'maxResults': top}]}]} rsp = VISION.images().annotate( body=body).execute().get['responses'][0] return ', '.join('%s (%.2f%%)' % (label['description'], label['score']*100.) for label in rsp['labelAnnotations']) def sheet_append_row(sheet, row): rsp = SHEETS.spreadsheets().values().append( spreadsheetId=sheet, range='Sheet1', body={'values': rows}).execute() return rsp.get('updates').get('updatedCells') def main(fname, bucket, sheet_id, top): mtype, ftime, data = drive_get_img(fname) gcs_blob_upload(fname, bucket, data, mtype) rsp = vision_label_img(data, top) sheet_append_row(sheet_id, [fname, mtype, ftime, len(data), rsp]) API method calls in Bold Driver calls in Bold Italics Cloud image processing workflow def drive_get_file(fname): rsp = DRIVE.files().list( q="name='%s'" % fname).execute().get['files'][0] fileId, fname, mtype = rsp['id'], rsp['name'], rsp['mimeType'] blob = DRIVE.files().get_media(fileId).execute() return mtype, rsp['modifiedTime'], blob API method calls in Bold Driver calls in Bold Italics
  • 18. Cloud image processing workflow def gcs_blob_upload(fname, bucket, blob, mimetype): body = { 'name': fname, 'uploadType': 'multipart', 'contentType': mimetype } return GCS.objects().insert( bucket, body, blob).execute() Cloud image processing workflow def vision_label_img(img, top): body = {'requests': [{'image': { 'content': img}, 'features': [ {'type': 'LABEL_DETECTION', 'maxResults': top}]}]} rsp = VISION.images().annotate( body=body).execute().get['responses'][0] return ', '.join('%s (%.2f%%)' % ( label['description'], label['score']*100.) for label in rsp['labelAnnotations'])
  • 19. Cloud image processing workflow def sheet_append_row(sheet, row): rsp = SHEETS.spreadsheets().values().append( spreadsheetId=sheet, range='Sheet1', body={'values': row}).execute() return rsp.get('updates').get('updatedCells') API method calls in Bold Driver calls in Bold Italics Cloud image processing workflow def main(fname, bucket, sheet_id, top): fname, mtype, ftime, data = drive_get_img(fname) gcs_blob_upload(fname, bucket, data, mtype) rsp = vision_label_img(data, top) sheet_append_row(sheet_id, [fname, mtype, ftime, len(data), rsp]) API method calls in Bold Driver calls in Bold Italics
  • 20. ● Project goal: Imagining an actual enterprise use case and solve it! ● Specific goals: free-up highly-utilized resource, archive data to colder/cheaper storage, analyze images, generate report for mgmt ● Download image binary from Google Drive ● Upload object to Cloud Storage bucket ● Send payload for analysis by Cloud Vision ● Write back-up location & analysis results into Google Sheets ● Blog post: goo.gle/3nPxmlc (original post); Cloud X-post ● Codelab: free, online, self-paced, hands-on tutorial ● g.co/codelabs/drive-gcs-vision-sheets ● Application source code ● github.com/googlecodelabs/analyze_gsimg App summary 04 More Inspiration Build powerful solutions with GCP and G Suite
  • 21. Custom intelligence in Gmail Analyze Google Workspace (formerly G Suite) data with GCP
  • 22. Gmail message processing with GCP Gmail Cloud Pub/Sub Cloud Functions Cloud Vision Workspace (formerly G Suite) GCP Star message Message notification Trigger function Extract images Categorize images Inbox augmented with Cloud Function
  • 23. ● Gmail API: sets up notification forwarding to Cloud Pub/Sub ● developers.google.com/gmail/api/guides/push ● Pub/Sub: triggers logic hosted by Cloud Functions ● cloud.google.com/functions/docs/calling/pubsub ● Cloud Functions: "orchestrator" accessing GCP (and Google Workspace/G Suite) APIs ● Combine all of the above to add custom intelligence to Gmail ● Deep dive code blog post ● cloud.google.com/blog/products/application-development/ adding-custom-intelligence-to-gmail-with-serverless-on-gcp ● Application source code ● github.com/GoogleCloudPlatform/cloud-functions-gmail-nodejs App summary 05 Wrap-up Summary & resources
  • 24. Session Summary ● Google provides more than just apps ○ We're more than search, YouTube, Android, Chrome, and Gmail ○ Much of our tech available to developers through our APIs ● Tour of Google (Cloud) APIs & developer tools ○ Workspace: not just a set of productivity apps… you can code them too! ○ GCP: compute, storage, networking, security, data & machine learning tools ○ Google Cloud serverless frees developers from infrastructure ■ 4 unique services so you can focus on building solutions ● Interesting possibilities using both GCP + Workspace together ● Other Google developer products to consider Other Google APIs & platforms ● Firebase (mobile development platform + RT DB; ML Kit) ○ firebase.google.com & firebase.google.com/docs/ml-kit ● Google Data Studio (data visualization, dashboards, etc.) ○ datastudio.google.com/overview ○ goo.gle/datastudio-course ● Actions on Google/Assistant/DialogFlow (voice apps) ○ developers.google.com/actions ● YouTube (Data, Analytics, and Livestreaming APIs) ○ developers.google.com/youtube ● Google Maps (Maps, Routes, and Places APIs) ○ developers.google.com/maps ● Flutter (native apps [Android, iOS, web] w/1 code base[!]) ○ flutter.dev
  • 25. ● Documentation ○ GCP: cloud.google.com/{docs,appengine,functions,vision,automl,sql,compute.storage, language,speech,text-to-speech,translate,video-intelligence,firestore,bigquery,filestore} ○ Workspace: developers.google.com/{gsuite,drive,calendar,gmail,docs,sheets,slides,apps-script} ● Introductory "codelabs" ([free] self-paced, hands-on tutorials) ○ Workspace REST APIs: g.co/codelabs/gsuite-apis-intro (featuring Drive API) ○ Apps Script: g.co/codelabs/apps-script-intro ○ App Engine: codelabs.developers.google.com/codelabs/cloud-app-engine-python ○ Cloud Vision: g.co/codelabs/vision-python (or C# or Ruby) ○ Cloud Functions: codelabs.developers.google.com/codelabs/cloud-starting-cloudfunctions ○ All others: gcplab.me (GCP) and g.co/codelabs (non-GCP) ● Videos: youtube.com/GoogleCloudPlatform (GCP), goo.gl/JpBQ40 (Workspace), and all others ● Code samples: github.com/GoogleCloudPlatform (GCP) and github.com/gsuitedevs (Workspace) ● GCP Free trial (new users) and Always Free (daily/monthly tier) - cloud.google.com/free Know AWS/Azure? Compare w/GCP: cloud.google.com/docs/compare/{aws,azure} Online resources Thank you! Questions? Wesley Chun @wescpy Codelab: g.co/codelabs/drive-gcs-vision-sheets Progress bars: goo.gl/69EJVw Slides: bit.ly/2SHybQg