The Couchbase Autonomous Operator is a Kubernetes operator that provides cloud-native integration with open source Kubernetes and Red Hat Openshift. It allows a user to use Kubernetes’ declarative functionality to define what the Couchbase Server Cluster will be and manage that cluster’s attributes. This declarative functionality is useful as it allows you to store your environment definitions in source control.

The operator’s goal is to manage one or more Couchbase Server deployments fully. It manages the cluster lifecycle (provisioning, scaling, upgrade, auto-recovery) and configuration (persistent volumes, server groups, XDCR, TLS, RBAC, backup/restore). The operator is a potent tool in managing a large-scale Couchbase environment and it is highly recommended you read the Couchbase Operator’s Documentation.

Cloud-native Automation with Couchbase Documentation

Save and Restore: How to migrate a non-managed topology

We added the Save and Restore feature In Couchbase Autonomous Operator v2.3. This feature allows users to migrate a non-managed Couchbase Cluster and turn it into a managed cluster. This functionality allows you to probe a Couchbase Cluster created by the Autonomous operator with un-managed data topology and retrieve a data topology YAML file that matches that Couchbase cluster’s current data topology. A user can take this deployment YAML file and apply it to the existing cluster to lock down the topology or apply it to a new cluster environment to migrate the environment. 

The use cases for this feature are numerous! Still, the one we will focus on in this post is migrating an environment from a manually managed data topology in a rapid/agile R/D environment to a more stable production environment.

Whether you are new to Couchbase or a seasoned pro, the new Save and Restore feature of the Couchbase Autonomous Operator significantly simplifies your CI/CD pipeline. Being able to transition easily from an unmanaged cluster to a managed cluster allows you to continue fast-paced development while maintaining control of environments down the pipeline.

Demonstrating a Couchbase cluster schema migration  

To demonstrate this functionality, we first need to create a Couchbase server cluster in Kubernetes in an unmanaged state. We’ll be using this unmanaged.yaml file:

To create this cluster, use the command: This creates a simple unmanaged cluster with all the services, but turns off bucket and RBAC management. It also includes a secret to access the cluster with a username of Administrator and a password of password

To access this cluster, we port-forward port 8091 so that we can access the Couchbase server Admin UI just for quick development access. If you want to access the admin port regularly, see the docs about setting up a load balancer port. 

To create this port forwarding, use the command: 

Now we can access the web UI on port 8091 using the administrator credentials and then create some buckets, scopes and collections. In this example, we create a BlogApp bucket, with a scope for en-US and some collections within that scope.

Next, we add the scope for en-US.

Add scope to Couchbase Kubernetes Operator

Lastly, we add the Blogs collections.

Add collection to Couchbase Autonomous Operator Kubernetes

Now we have a tree structure that looks like this:

Once we have our data topology, we can generate a YAML file using the cao binary with the command:

This will output our topology.yaml:

Cloning a cluster topology

With the topology file we can now restore this to another cluster, cloning the structure to a new environment. To do this create a managed.yaml with this content:

Next, we create the managed cluster using: This is similar to our unmanaged.yaml, but instead, the buckets are managed and we re-used our secret from the unmanaged cluster. 

Restoring topology to managed cluster

Once that cluster has spun up, you can use the cao binary to restore the topology YAML to the newly created managed cluster:

Restoring managed cluster topology with YAML file for migration
Lastly, to verify, we’ll create a simple port-forward to the new, managed cluster and verify the data topology. Use the following command to create the port-forward and open your browser to http://127.0.0.1:8091.  

You should be greeted with the appropriate data topology.

Confirm schema migration by viewing scopes and collection in Couchbase

Now that we have migrated the data topology, we may want to migrate the data from one environment to another. I recommend using Cross Data Center Replication (XDCR) to facilitate data movement from one cluster to another. You can find more information about setting up XDCR from the Couchbase Operator Documentation.

Next steps and resources

Some things to keep in mind:

    • Save does not save RBAC roles/groups. You will need to migrate those on your own.
    • The cao tool will inform you of all changes to be made to the destination cluster. Any item marked as delete will be deleted, potentially resulting in data loss.

This post touched on the following topics:

 

Author

Posted by Justin Ashworth - Senior Software Engineer

Leave a reply