Almost two years ago I wrote the first installment of what was supposed to be a 3 part series on Serverless Monitoring. Parts two and three never materialized. Today, however, I am revisiting that original idea and expanding upon it. I hope to succeed this time in making it a full three-part series. For this first installment (Revisited), I will again work with Google Cloud Run to monitor MongoDB Atlas. I stated in that original blog that there are many reasons that someone may want to run monitoring in a serverless state, which still holds today. However, we will take it further this time and connect this Cloud Run system to our Bindplane platform. I will be using our SaaS offering: Bindplane Cloud.

Environment Prerequisites

• MongoDB Atlas Target
  • This target already set up correctly with the API Access Keys.
• Access to Google Cloud Run
• Access to Google Secret Manager
  • Secrets Created for Atlas public key and private key
• Access to Google Cloud Storage
  • Bucket named cloudrun-atlas that contains
    • Generic bindplane-otel-collector config.yaml for bootstrapping
    • A logging.yaml file
    • A manager.yaml file
• Container Images in Google Artifact Registry
• Bindplane Cloud Access or On-Prem Bindplane Server
  • This can be done via a free license, which you can obtain on our website.

Resources

• https://github.com/observIQ/bindplane-otel-collector/blob/main/docs/google-cloud-run.md
• https://bindplane.com/docs/resources/sources/mongodb-atlas
• https://bindplane.com/docs/resources/destinations/google-cloud

The first task on our agenda is to get our container image transferred from Docker Hub to the Google Artifact Registry. To do this, we need a system with Docker installed. Additionally, we need to have the project already created in Google Cloud. For this blog, I’ve created a temporary project called dm-cloudrun-blog. I have also chosen to use our minimal container image, which is Scratch, instead of a traditional Linux container. Now that we’re ready with Docker and our Google Cloud project, we can run the following commands to import the image into the Artifact Registry:

1docker pull observiq/bindplane-agent:1.75.0-minimal
2docker tag observiq/bindplane-agent:1.75.0-minimal us-central1-docker.pkg.dev/dm-cloudrun-blog/bindplane-agent/bindplane-agent:1.75.0-minimal
3gcloud auth print-access-token | docker login   -u oauth2accesstoken   --password-stdin https://us-central1-docker.pkg.dev
4docker push us-central1-docker.pkg.dev/dm-cloudrun-blog/bindplane-agent/bindplane-agent:1.75.0-minimal

Our second prerequisite task is to set up our secrets. For this piece of the puzzle, I go to the Google Cloud Secret Manager and create two secrets: mongo-atlas-priv-key and mongo-atlas-pub-key. The values of the secrets are the ones set up on the MongoDB Atlas site for the 2 keys.

The third prerequisite task is to set up a Cloud Storage Bucket that contains three files. Sample files can be found:

• config.yaml
• logging.yaml
• manager.yaml
  • This needs to be edited with your endpoint and secret key
    • Alternatively, these can be defined as environment variables the same way we did with the Mongo Atlas keys above

Creation of the Cloud Run Deployment

Now that the prerequisites are completed, we can focus on creating our deployment. In the Google Cloud Console, under Cloud Run, we click Create Service.

On the next page, we will need to fill in several values. Under the initial display, fill in the following:

• Container Image URL (we created above): us-central1-docker.pkg.dev/dm-cloudrun-blog/bindplane-agent/bindplane-agent:1.75.0-minimal
• Service Name: I’m using bindplane-mongo
• Authentication: Set as desired
• Set Billing to Instance Based
• Set Minimum Instances to 1
• Ingress: Set to Internal

Now we need to expand the section called Container, Variables & Secrets, Connections, Security by clicking the dropdown arrow to the right of that heading. Once it expands, set the Container Port to 13133.

With the port set, we can access the Variables & Secrets sub tab. Click the Reference a secret link. Using the Secret dropdown, select mongo-atlas-priv-key, change the Reference Method to Exposed as environment variable. Finally, the Name should be set to MONGODB_ATLAS_PRIVATE_KEY. Repeat this process for mongo-atlas-pub-key, replacing it with MONGODB_ATLAS_PUBLIC_KEY.

After the environment variables are set, we click into the Volumes section. Click the Add Volume link. Set the volume type to Cloud Storage Bucket, the volume name to etc-otel, and Bucket to cloudrun-atlas. Now return to the Container tab, and enter the Volume Mounts subtab. Click the Mount Volume button. In the Name dropdown, select the etc-otel we defined above. Set the Mount path to etc/otel (note, the preceding root / is already there).

That is all of the container's special parameters set. All that remains is to set the starting CPU and Memory. I recommend setting the CPU to 2 and the Memory to 1 GiB. Set the Revision Scaling Minimum and Maximum to a value of 1; this will prevent horizontal scaling because we do not want more than a single agent connecting to the Atlas API with a given configuration. Also, check Startup CPU Boost. All other parameters can be left at the default, and we can click the blue CREATE button at the bottom of the page.

Reviewing the Container

Now that our image is deployed, we can click on it in the list of Cloud Run services. Doing so, brings us to a dashboard of metrics for the container. We can choose from other tabs, such as logs, revisions, and triggers. The metrics here can tell us if our container needs to be edited to have more CPU and/or Memory. The logs will display the logs from inside the container, where we can see what is happening with the collector and rectify any issues it has by editing the configuration file secret.

Creating Config in Bindplane

With the container set up, and connected to Bindplane, we can now create a configuration and attach it to the agent. To do this, we click on Configurations on the top banner in Bindplane and click the Create Configuration button. Give this new configuration a name; I used MongoAtlas. Set the Platform to Linux and the Agent Type to the current stable version. Click Next.

On this new page, click Add Source. Select the MongoDB Atlas source from the list. When the source configuration opens, set the API Public Key as ${MONGODB_ATLAS_PUBLIC_KEY} and the API Private Key to ${MONGODB_ATLAS_PRIVATE_KEY}. Set your Project Name, and any other options you want. Click the Save button at the bottom of the dialog. Click Next.

Click the Add Destination button. For testing, I added the Dev Null destination. You should add whatever is appropriate for your environment. Or use Dev Null if this is just to test collection. Click Save.

Everything is ready; the final step is to click Start Rollout.

With the rollout completed, our Cloud Run agent will start collecting telemetry data from MongoDB Atlas.

Conclusion

Most teams will eventually need to monitor a serverless computing resource. Running an instance of a telemetry collector inside another serverless computing platform can often be an inexpensive and effective way to address this need.

I look forward to the next installment of this three-part series: AWS Elastic Container Service. I will repeat what we achieved with Google Cloud Run over in AWS in that installment. Hope to see you there.