Seppo Jaakola, CEO of Codership, makers of Galera Cluster, recently gave a talk titled Introduction to MariaDB Galera Cluster at the MariaDB Server Fest 2020, and it is truly one of the best introductions to the software that is currently available and up-to-date as of September 2020. There is an overview, configuration, feature differences between asynchronous replication as well as the releases and release cycles that are available.
As Seppo says in the talk, we work closely with MariaDB to ensure consistent support and services around Galera Cluster, which is also why we have a good strong partnership, from an engineering to services standpoint.
Galera is a replication plugin and in the configuration you need to configure my.cnf to where the Galera plugin resides. This is wsrep_provider and you can also have wsrep_provider_options. You start by having one node, and then after that you start another node, by telling the other cluster addresses via wsrep_cluster_address. In principle, all nodes have the wsrep_cluster_address as a full list (this is a good idea to ensure that your my.cnf is synced with each other). You also naturally need an State Snapshot Transfer (SST) method, via wsrep_sst_method (which you can do such as: wsrep_sst_method=rsync) – this is how the new joining nodes get a copy of the entire database. In this case the joiner node gets an rsync copy of all the data once the handshake occurs, you get a fresh copy of the database, and once the copy is completed, the server pairs and will operate as a node in the synchronous database cluster.
Two nodes are not naturally very functional for a Galera Cluster, as we prefer 3-nodes as a minimum recommend cluster size. Three nodes are the minimum number of nodes when it comes to effective voting. Every node in a cluster is a full backup of each other due to synchronous replication. All writes in the cluster are replicated; all nodes can be used for reading.
Galera Cluster is based on a generic replication plugin for database servers, and uses the replication API to interact with the Database Management Server (DBMS) via the wsrep API (project is open source on GitHub). MariaDB 10.1 and later have had built-in Galera Cluster which makes it easy to get started. Seppo goes through the complete list of wsrep configuration options, but in practice you need very few to get going.
MariaDB 10.4 have 66 wsrep-specific status variables, and when it comes to monitoring, the most common variables are: wsrep_ready, wsrep_cluster_status and wsrep_cluster_size. Besides configuration and status variables, there are also 3 tables for Galera, located in the mysql schema, and they are wsrep_cluster, wsrep_cluster_members (active members of the cluster) and wsrep_streaming_log (a new feature in Galera 4, and in MariaDB 10.4, streaming replication is a method for replicating very big/long transactions).
With streaming replication, transactions are processing in small chunks in each node in the cluster. Status is stored and kept in the mysql.wsrep_streaming_log table. In case of issues, state information is kept here when it comes to long-running transactions. It can also be used for monitoring of transactions and how long they have been processing. This is a good method for troubleshooting ill-behaving transactions.
Some interesting features that Galera Cluster brings: synchronous replication (close to fully synchronous, but the commit is currently only done in one node in the cluster; so it is the look & feel of synchronous replication – there is wsrep_sync_wait to help with this). Flow control keeps node progress even and all nodes are equal, allowing read and write access. Conflicting writes are supported (hence multi-master use is possible – Galera picks which row to commit to, and can ensure rollback for the conflicting write with a deadlock error, and retry the transaction based on configuration). There is obviously also automatic node joining.
What is different with asynchronous replication (standard in MariaDB) and synchronous replication (Galera replication)? Basically, there are several simple reasons for this:
- Asynchronous replication allows writes only to the master (primary) server and you end up with a master-slave (primary-secondary) topology.
- Secondary nodes may fall behind and encounter secondary lag as there is also no flow control.
- Changing the primary server will require failover management which is a burden one has to manage via external software.
Galera development started in 2008 and MariaDB support came in 5.5 and 10.0 versions. MariaDB 10.1 includes Galera Cluster in the main branch (so one download is all you need that comes with the relevant libraries). Feature wise, Galera replication has major version numbers: 1, 2, 3, and 4, and the current major version 4 has been present in MariaDB Server 10.4 and greater. Changes to the wsrep API also mean that you will need to perform a rolling upgrade (e.g. if you were migrating from MariaDB Server 10.3 to MariaDB Server 10.4). It is important to note that Galera can be upgraded, but rolling downgrades are not supported (i.e. if you decided to go from MariaDB Server 10.4 with Galera 4 to MariaDB Server 10.3 with Galera 3).
For MariaDB Server 10.6, there may be some major changes that go into Galera and this is something that will be decided quite soon within the next 1-2 months.
Galera Cluster also works in WAN and cloud installations, and MariaDB 10.4 has significant new Galera 4 features as stated in this blog post, so it is highly recommended that you use MariaDB 10.4 and later for evaluation.
Watch out for our next post about the other session that Seppo gave that focuses on what is around in MariaDB Server 10.5 and what is coming in MariaDB Server 10.6.