This document describes how to implement the reference architecture that's described in Integrate microservices with Pub/Sub and GKE. The architecture is designed to handle long-running processes by using containers and asynchronous messaging.
The document uses an example photo-sharing application that generates photo thumbnails. You deploy the application using Google Kubernetes Engine (GKE) and use Pub/Sub to invoke long-running processes asynchronously. You also use Pub/Sub notifications for Cloud Storage to add side jobs without modifying the application's code.
The application is containerized by Cloud Build and stored in Artifact Registry. It uses Cloud Vision to detect inappropriate images.
Architecture
The following diagram illustrates the design of the example photo album application that implements the reference architecture.
Figure 1. Architecture for image processing that's based on using containers and asynchronous messaging.
The preceding diagram illustrates how the thumbnail is generated:
- A client uploads an image to the application.
- The application stores the image in Cloud Storage.
- A request is generated for the thumbnail.
- The thumbnail generator generates the thumbnail.
- The successful response is sent to the photo album application.
- The successful response is sent to the client and you can find the thumbnail in Cloud Storage.
The following diagram shows how the application implements thumbnail generation as a separate service in an asynchronous manner.
Figure 2. Architecture of the thumbnail extraction process.
You use Pub/Sub to send service requests to the thumbnail generation service. This new architecture makes the service call asynchronous so that a thumbnail is created in the background after the application sends the response back to a client. This design also allows the thumbnail generation service to scale so that multiple jobs can run in parallel.
Objectives
- Deploy an example photo album application on GKE.
- Make asynchronous service calls from the application.
- Use Pub/Sub notifications for Cloud Storage to trigger the application when a new file is uploaded to the Cloud Storage bucket.
- Use Pub/Sub to perform more tasks without modifying the application.
Costs
In this document, you use the following billable components of Google Cloud:
- Cloud Storage
- Cloud SQL
- Pub/Sub
- Compute Engine instances used by GKE
- Cluster-management fee for GKE
- Cloud Load Balancing
- Cloud Build
- Artifact Registry
- Vision
To generate a cost estimate based on your projected usage,
use the pricing calculator.
When you finish building the example application, you can avoid continued billing by deleting the resources you created. For more information, see Clean up.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the GKE, Cloud SQL, Cloud Build, Artifact Registry, and Cloud Vision APIs.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the GKE, Cloud SQL, Cloud Build, Artifact Registry, and Cloud Vision APIs.
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
Set up the environment
In this section, you assign default settings for values that are used throughout the document. If you close your Cloud Shell session, you lose these environment settings.
In Cloud Shell, set your default Google Cloud project:
gcloud config set project PROJECT_ID
Replace
PROJECT_ID
with your Google Cloud project ID.Set your default Compute Engine region:
gcloud config set compute/region REGION export REGION=REGION
Replace
REGION
with a region that is close to you. For more information, see Regions and zones.Set your default Compute Engine zone:
gcloud config set compute/zone ZONE export ZONE=ZONE
Replace
ZONE
with a zone that is close to you.Download the example application files and set your current directory:
git clone https://github.com/GoogleCloudPlatform/gke-photoalbum-example cd gke-photoalbum-example
Create a Cloud Storage bucket and upload the default thumbnail image
In Cloud Shell, create a Cloud Storage bucket to store the original images and thumbnails:
export PROJECT_ID=$(gcloud config get-value project) gsutil mb -c regional -l ${REGION} gs://${PROJECT_ID}-photostore
Upload the default thumbnail file:
gsutil cp ./application/photoalbum/images/default.png \ gs://${PROJECT_ID}-photostore/thumbnails/default.png
- The uploaded images are stored in the following format:
gs://PROJECT_ID-photostore/FILENAME
, whereFILENAME
represents the name of the image file that is uploaded. - The generated thumbnails are stored in the following format:
gs://PROJECT_ID-photostore/thumbnails/FILENAME
. - The original image and the corresponding thumbnail have the
same
FILENAME
value, but the thumbnail is stored in thethumbnails
bucket. While the thumbnail is being created, the following
default.png
placeholder thumbnail image is displayed in the photo album application.
- The uploaded images are stored in the following format:
Make the thumbnail file public:
gsutil acl ch -u AllUsers:R \ gs://${PROJECT_ID}-photostore/thumbnails/default.png
Create a Cloud SQL instance and a MySQL database
In Cloud Shell, create the Cloud SQL instance:
gcloud sql instances create photoalbum-db --region=${REGION} \ --database-version=MYSQL_8_0
Retrieve the connection name:
gcloud sql instances describe photoalbum-db \ --format="value(connectionName)"
Make a note of the name because you'll use it later.
Set the password for the
root@%
MySQL user:gcloud sql users set-password root --host=% --instance=photoalbum-db \ --password=PASSWORD
Replace
PASSWORD
with a secure password for theroot@%
user.Connect to the Cloud SQL instance:
gcloud sql connect photoalbum-db --user=root --quiet
When you're prompted, enter the password that you set up in the preceding step.
Create a database called
photo_db
, where the user isappuser
and the password ispas4appuser
:create database photo_db; create user 'appuser'@'%' identified by 'pas4appuser'; grant all on photo_db.* to 'appuser'@'%' with grant option; flush privileges;
Confirm the result and exit from MySQL:
show databases; select user from mysql.user; exit
In the output, confirm that the
photo_db
database and theappuser
user are created:mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | photo_db | | sys | +--------------------+ 5 rows in set (0.16 sec) mysql> \t Outfile disabled. mysql> select user from mysql.user; +-------------------+ | user | +-------------------+ | appuser | | cloudsqlreplica | | cloudsqlsuperuser | | root | | cloudsqlexport | | cloudsqlimport | | cloudsqloneshot | | root | | cloudsqlexport | | cloudsqlimport | | cloudsqloneshot | | root | | cloudsqlapplier | | cloudsqlimport | | mysql.infoschema | | mysql.session | | mysql.sys | | root | +-------------------+ 18 rows in set (0.16 sec) mysql> exit Bye
Create a Pub/Sub topic and a subscription
In Cloud Shell, create a Pub/Sub topic called
thumbnail-service
:gcloud pubsub topics create thumbnail-service
The photo album application sends requests to the thumbnail generation service by publishing a message on the
thumbnail-service
topic.Create a Pub/Sub subscription called
thumbnail-workers
:gcloud pubsub subscriptions create --topic thumbnail-service thumbnail-workers
The thumbnail generation service receives requests from the
thumbnail-workers
subscription.
Create a GKE cluster
In Cloud Shell, create a GKE cluster that has permission to call APIs:
gcloud container clusters create "photoalbum-cluster" \ --scopes "https://www.googleapis.com/auth/cloud-platform" \ --num-nodes "5"
Get access credentials configured so that you can manage the cluster using the
kubectl
command in later steps:gcloud container clusters get-credentials photoalbum-cluster
Show the list of nodes:
kubectl get nodes
In the output, confirm that there are five nodes whose
STATUS
value isReady
:NAME STATUS ROLES AGE VERSION gke-photoalbum-cluster-default-pool-d637570a-2pfh Ready <none> 2m55s v1.24.10-gke.2300 gke-photoalbum-cluster-default-pool-d637570a-3rm4 Ready <none> 2m55s v1.24.10-gke.2300 gke-photoalbum-cluster-default-pool-d637570a-f7l4 Ready <none> 2m55s v1.24.10-gke.2300 gke-photoalbum-cluster-default-pool-d637570a-qb2z Ready <none> 2m53s v1.24.10-gke.2300 gke-photoalbum-cluster-default-pool-d637570a-rvnp Ready <none> 2m54s v1.24.10-gke.2300
Create the Artifact Registry repository
In Cloud Shell, create a repository to store container images:
gcloud artifacts repositories create photoalbum-repo \ --repository-format=docker \ --location=us-central1 \ --description="Docker repository"
Build images for the application
In a text editor, open the
application/photoalbum/src/auth_decorator.py
file and update the username and password:USERNAME = 'username' PASSWORD = 'passw0rd'
In Cloud Shell, build an image for the photo album application by using the Cloud Build service:
gcloud builds submit ./application/photoalbum -t \ us-central1-docker.pkg.dev/${PROJECT_ID}/photoalbum-repo/photoalbum-app
Build an image for the
thumbnail-worker
thumbnail generation service by using the Cloud Build service:gcloud builds submit ./application/thumbnail -t \ us-central1-docker.pkg.dev/${PROJECT_ID}/photoalbum-repo/thumbnail-worker
Deploy the photo album application
In Cloud Shell, update the Kubernetes Deployment manifests for the photo album and the thumbnail generator with values from your environment:
connection_name=$(gcloud sql instances describe photoalbum-db \ --format "value(connectionName)") digest_photoalbum=$(gcloud container images describe \ us-central1-docker.pkg.dev/${PROJECT_ID}/photoalbum-repo/photoalbum-app:latest --format \ "value(image_summary.digest)") sed -i.bak -e "s/\[PROJECT_ID\]/${PROJECT_ID}/" \ -e "s/\[CONNECTION_NAME\]/${connection_name}/" \ -e "s/\[DIGEST\]/${digest_photoalbum}/" \ config/photoalbum-deployment.yaml digest_thumbnail=$(gcloud container images describe \ us-central1-docker.pkg.dev/${PROJECT_ID}/photoalbum-repo/thumbnail-worker:latest --format \ "value(image_summary.digest)") sed -i.bak -e "s/\[PROJECT_ID\]/${PROJECT_ID}/" \ -e "s/\[CONNECTION_NAME\]/${connection_name}/" \ -e "s/\[DIGEST\]/${digest_thumbnail}/" \ config/thumbnail-deployment.yaml
Create deployment resources to launch the photo album application and the thumbnail generation service:
kubectl create -f config/photoalbum-deployment.yaml kubectl create -f config/thumbnail-deployment.yaml
Create a service resource to assign an external IP address to the application:
kubectl create -f config/photoalbum-service.yaml
Check the results for the pods:
kubectl get pods
In the output, confirm that there are three pods for each
photoalbum-app
andthumbail-worker
pod and that theirSTATUS
value isRunning
:NAME READY STATUS RESTARTS AGE photoalbum-app-555f7cbdb7-cp8nw 2/2 Running 0 2m photoalbum-app-555f7cbdb7-ftlc6 2/2 Running 0 2m photoalbum-app-555f7cbdb7-xsr4b 2/2 Running 0 2m thumbnail-worker-86bd95cd68-728k5 2/2 Running 0 2m thumbnail-worker-86bd95cd68-hqxqr 2/2 Running 0 2m thumbnail-worker-86bd95cd68-xnxhc 2/2 Running 0 2m
The
thumbnail-worker
pods subscribe thumbnail generation requests from thethumbnail-workers
subscription. For more information, see how thecallback
function is used in the source code.Check the results for the services:
kubectl get services
In the output, confirm that there is an external IP address in the
EXTERNAL-IP
column for thephotoalbum-service
service. It might take a few minutes until the services are all running.NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.23.240.1 <none> 443/TCP 20m photoalbum-service LoadBalancer 10.23.253.241 146.148.111.115 80:32657/TCP 2m
Make a note of the external IP address because it's used later. In this example, it's
146.148.111.115
.
Test the photo album application
To access the deployed application in a web browser, go to the following URL and enter the username and password that you previously set up:
http://EXTERNAL_IP
Replace
EXTERNAL_IP
with the IP address that you copied in the previous step.To upload an image file, click Upload.
The thumbnail placeholder appears on the screen.
In the background, the thumbnail generation service creates a thumbnail of the uploaded image. To see the generated thumbnail, click Refresh. The Cloud Vision API adds image labels that it detects.
To see the original image, click the thumbnail.
Add an inappropriate-image detection feature
The following image illustrates how you can use Pub/Sub notifications for Cloud Storage to trigger a service that detects inappropriate content. This feature blurs the image when a new file with inappropriate content is stored in the Cloud Storage bucket.
In the preceding image, the service uses the Safe Search Detection feature from Vision API to detect inappropriate content within an image.
The photo application triggers the thumbnail generator and the image checker asynchronously. As a result, there is no guarantee that they will be executed in a specific order. If the thumbnail generation occurs before the image is blurred, you might see an inappropriate thumbnail for a short while. However, the image checker eventually blurs both the inappropriate image and the inappropriate thumbnail.
Create a Pub/Sub topic, subscription, and notification
In Cloud Shell, create a Pub/Sub topic called
safeimage-service
:gcloud pubsub topics create safeimage-service
Create a Pub/Sub subscription called
safeimage-workers
:gcloud pubsub subscriptions create --topic safeimage-service \ safeimage-workers
Configure a Pub/Sub notification so that a message is sent to the
safeimage-service
topic when a new file is uploaded to the Cloud Storage bucket:gsutil notification create -t safeimage-service -f json \ gs://${PROJECT_ID}-photostore
Build and deploy the worker image
In Cloud Shell, build a container image for the
safeimage-workers
subscription by using Cloud Build:gcloud builds submit ./application/safeimage \ -t us-central1-docker.pkg.dev/${PROJECT_ID}/photoalbum-repo/safeimage-worker
Update the Kubernetes Deployment manifests for the safe-image service with your Google Cloud project ID, Cloud SQL connection name, and container image digests:
digest_safeimage=$(gcloud container images describe \ us-central1-docker.pkg.dev/${PROJECT_ID}/photoalbum-repo/safeimage-worker:latest --format \ "value(image_summary.digest)") sed -i.bak -e "s/\[PROJECT_ID\]/${PROJECT_ID}/" \ -e "s/\[CONNECTION_NAME\]/${connection_name}/" \ -e "s/\[DIGEST\]/${digest_safeimage}/" \ config/safeimage-deployment.yaml
Create a deployment resource
Create a deployment resource called
safeimage-deployment
to deploy thesafeimage-service
topic:kubectl create -f config/safeimage-deployment.yaml
Check the results:
kubectl get pods
In the output, confirm that there are three
safeimage-worker
pods whoseSTATUS
value isRunning
.NAME READY STATUS RESTARTS AGE photoalbum-app-555f7cbdb7-cp8nw 2/2 Running 0 30m photoalbum-app-555f7cbdb7-ftlc6 2/2 Running 0 30m photoalbum-app-555f7cbdb7-xsr4b 2/2 Running 8 30m safeimage-worker-7dc8c84f54-6sqzs 1/1 Running 0 2m safeimage-worker-7dc8c84f54-9bskw 1/1 Running 0 2m safeimage-worker-7dc8c84f54-b7gtp 1/1 Running 0 2m thumbnail-worker-86bd95cd68-9wrpv 2/2 Running 0 30m thumbnail-worker-86bd95cd68-kbhsn 2/2 Running 2 30m thumbnail-worker-86bd95cd68-n4rj7 2/2 Running 0 30m
The
safeimage-worker
pods subscribe ro inappropriate-image detection requests from thesafeimage-workers
subscription. For more information, see how thecallback
function is used in the source code.
Test the inappropriate-image detection feature
In this section, you upload a test image to verify that the Safe Search Detection feature blurs out an inappropriate image. The test image is a picture of a girl who's dressed up as a zombie (licensed under a CC0 license from Pixaby).
- Download the test image.
- To upload the image, go to
http://EXTERNAL_IP
, and then click Upload. Click Refresh. The application displays a blurred thumbnail.
To see that the uploaded image is also blurred, click the thumbnail.
Clean up
If you don't want to keep the Google Cloud resources that you created for the example application, you can remove them so you won't be billed for them in the future. You can either delete the project entirely or delete cluster resources and then delete the cluster.
Delete the project
- In the Google Cloud console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down to delete the project.
Delete the individual resources
Instead of deleting the project, you can delete the individual resources that you created.
Delete resources from GKE:
kubectl delete -f config/safeimage-deployment.yaml kubectl delete -f config/photoalbum-service.yaml kubectl delete -f config/thumbnail-deployment.yaml kubectl delete -f config/photoalbum-deployment.yaml
Delete the cluster from GKE:
gcloud container clusters delete photoalbum-cluster --quiet
Delete the repository from Artifact Registry:
gcloud artifacts repositories delete photoalbum-repo --location us-central1 --quiet
Delete subscriptions and topics from Pub/Sub:
gcloud pubsub subscriptions delete safeimage-workers gcloud pubsub topics delete safeimage-service gcloud pubsub subscriptions delete thumbnail-workers gcloud pubsub topics delete thumbnail-service
Delete the Cloud SQL instance:
gcloud sql instances delete photoalbum-db --quiet
Delete the Cloud Storage bucket:
gsutil rm -r gs://${PROJECT_ID}-photostore gsutil rm -r gs://${PROJECT_ID}_cloudbuild
Delete the files:
cd .. rm -rf gke-photoalbum-example
What's next
- Read about DevOps, and learn more about the Architecture capability that's related to this reference architecture.
- Take the DevOps quick check to understand where you stand in comparison with the rest of the industry.
- For more reference architectures, diagrams, and best practices, explore the Cloud Architecture Center.