Syncing Prod & Staging Data With Dokku

Sep 2, 2024 | Automation, Docker, Programming

If you’re anything like me, you’ve probably faced the classic problem of everything working swimmingly on your test / staging environment, but the end users hitting some weird issue on production. This is often caused by your production data being (obviously) accurate real-world customer data and your staging data being developer created data that almost certainly follows a Goldielocks path; let’s face it, it’s nearly impossible to predict the JarJar Binks level weird stuff under users will put our code through. Today, I want to share a simple but powerful approach using Dokku’s apps:clone functionality that has been working well for me to avoid this very issue. 

Why Bother With Dokku’s apps:clone?

Dokku’s apps:clone command is a fantastic utility that allows you to replicate an entire app, including all code, databases and environment variables. Breaking it down further for the certified PM types: 

  • Simplicity and Speed: It’s just easy and leverages the core advantages of containerization; if you didn’t already know, Dokku is an open source re-implementation of Heroku’s deployment APIs based fully on Docker. 
  • Automated Syncing: The problem I was actually trying to solve when I first started doing this was that I had a customer who wanted staging to mirror production data and sync on a weekly basis. There are of course other ways to achieve this, but I’ve found many of them to be as elegant as the dialogue in most Middle Schoolers’ fan-fiction. I also don’t want to have to remember to run a script every week for obvious reasons, such as having a life and other customers.
  • Consistency: This probably should go without saying, but manual data administration is generally bad, yet we see it all too often and it generally leads to irritation all-around. That’s why it’s so important to automate this process. 

Taking It a Step at a Time

Begin the Clone Wars: Use Dokku’s apps:clone command. This will recreate the staging app from production, including the database, files, and environment configurations. Keep in mind that for the sample below `staging_app` and `production_app` should be the actual names of your staging and production apps.

dokku apps:destroy staging_app --force
dokku apps:clone production_app staging_app

Keep the Clones Coming: To automate this process, add it to your cron jobs. Here’s an example of how to run the clone every Sunday at midnight via good old Bash. There are fancier ways to handle this, but for the purposes of this tutorial we’re going to keep it simple.

crontab -e

0 0 * * 0 /path/to/your/clone_and_setup_script.sh >> /var/log/dokku_clone.log 2>&1

You will also need to include re-configuring your staging app’s LetsEncrypt config in your Bash script. 

letsencrypt:set staging email $LETSENCRYPT_EMAIL
letsencrypt:enable $STAGING_APP
dokku ps:restart $STAGING_APP

That’s it! Not too bad, right? If you want to see more DevOps and Automation tips, follow me on LinkedIn and if your organization needs help automating processes even with legacy data, checkout Alice.

 

More from Mike:

About Me

Hi! I’m Mike! I’m a software engineer who codes at The Mad Botter INC. You might know me from Coder Radio or The Mike Dominick Show.  Drop me a line if you’re interested in having some custom mobile or web development done.

Follow Me

© 2024 Copyright Michael Dominick | All rights reserved