Schedule MongoDB Backup to S3 using Kubernetes CronJob

Introduction

Kubernetes CronJob makes it very easy to run Jobs on a time-based schedule. These automated jobs run like Cron tasks on a Linux or UNIX system.

In this post, we’ll make use of Kubernetes CronJob to schedule a recurring backup of MongoDB database and upload the backup archive to AWS S3. All the source code is available in GitHub Repository.

Get Started

Let’s go ahead and first create a user in MongoDB dedicated to perform the backup with minimum privileges.

Login to the MongoDB shell as a root user.

mongo admin --host <hostname> --authenticationDatabase admin -u root

Run the following command to create the backup user.

db.createUser({
user: 'backup_user',
pwd: 'oO9eV5cG6cF2oM1r',
roles: [{ role: 'backup',db:'admin'}]
})

Kubernetes Namespace

Create a dedicated namespace in Kubernetes to deploy the cronjob.

kubectl apply -f https://raw.githubusercontent.com/tuladhar/k8s-backup-mongodb/main/kubernetes/namespace.yaml

The output is similar to this:

namespace/backup-mongodb created

Let’s save the namespace for all subsequent kubectl commands to run in that context.

kubectl config set-context --current --namespace=backup-mongodb

Kubernetes Secrets

Kubernetes Secrets allows us to store and manage sensitive information. Storing confidential information in a Secret is safer and more flexible than putting it verbatim in a Pod definition or in a container image.

Store MongoDB URI

export MONGODB_URI=mongodb://backup_user:oO9eV5cG6cF2oM1r@<mongodb-hostname>:27017kubectl create secret generic mongodb-uri --from-literal=MONGODB_URI=$MONGODB_URI

Store AWS credentials and S3 bucket URI

export AWS_ACCESS_KEY_ID=***
export AWS_SECRET_ACCESS_KEY=***
export BUCKET_URI=s3://bucket-name
export AWS_DEFAULT_REGION=us-east-1
kubectl create secret generic aws --from-literal=AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_IDkubectl create secret generic aws --from-literal=AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEYkubectl create secret generic aws --from-literal=BUCKET_URI=$BUCKET_URIkubectl create secret generic aws --from-literal=AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION

Deploy CronJob

Now we can go ahead and deploy the MongoDB backup cronjob by running the following command:

kubectl apply -f https://raw.githubusercontent.com/tuladhar/k8s-backup-mongodb/main/kubernetes/cronjob.yaml

The output is similar to this:

cronjob.batch/backup-mongodb created

The default schedule is to run every hour. To adjust the schedule, run the following command and modify the schedule property:

kubectl edit cronjob backup-mongodb
Fig: Adjust the schedule

After creating the cronjob, you can get its status by running the following command:

kubectl get cronjob

The output is similar to this:

NAME             SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULEbackup-mongodb   0 */1 * * *   False     0        <none>

As you can see from the results of the command, the cronjob has not scheduled or run any jobs yet. You can list the jobs by running the following command:

kubectl get jobs

To view the Pod logs for a job, run the following command:

pods=$(kubectl get pods --selector=job-name=<job-name> --output=jsonpath={.items[*].metadata.name})kubectl logs $pods

Conclusion

--

--

--

Make Complex Simple (https://tuladhar.github.io)

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Work can flow across the Sprint boundary

How to type the not equals (≠) symbol in 2022

Did a Garmin watch really need to be price optimized 129 times in 30 days?

Everything you need to know about Cocoapods in iOS

From Monolith to Microservices: A lesson learnt — review your tests!

AC Daily Log Part 5: Update Google Sheet Daily Log

[JustForFunEconomics] Dynamic Programming for Cake Eating Problem — (1) Intro

Consuming a RESTful Web Service using Spring’s RestTemplate class

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Puru Tuladhar

Puru Tuladhar

Make Complex Simple (https://tuladhar.github.io)

More from Medium

About Kubernetes architecture(1)

Prometheus Alertmanager — Custom email Alert configuration & templating

Kubernetes Inter-pod anti-affinity and de-schedule

Confluent cloud integration with Logz.io.