For MySQL, MyISAM and InnoDB storage engines are very popular. Currently, we are mostly using InnoDB engines for high reliability and high performance. Apart from those engines, we also have some other alternative engines and they have some nice features in them. In this blog, I am going to explain some of those engines, which I have listed below.
- FEDERATED Storage Engine
- Merge or MRG_MyISAM Engine
- Blackhole Engine
- CSV Engine
FEDERATED Storage Engine
Overview:
- FEDERATED Storage Engine allows you to access the data remotely without replication and cluster technologies.
- Using the FEDERATED tables, you can scale your server load. Queries for the given table will be sent over the network to another MySQL instance. In this case, to scale the DB, you can use many MySQL instances without changing the application code.
- FEDERATED tables are a security concern because you will need to save the host and user information in the table. It can be viewed using SHOW CREATE TABLE command.
- Query optimization is limited and JOINs are slow.
- Doing the bulk transaction may crash the local server.
By default, FEDERATED Storage Engine support is disabled. To enable it, you need to manually enable the variable “federated = ON” in the MySQL config file and restart the MySQL service.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | mysql> select * from information_schema.engines where engine='federated'\G *************************** 1. row *************************** ENGINE: FEDERATED SUPPORT: NO COMMENT: Federated MySQL storage engine TRANSACTIONS: NULL XA: NULL SAVEPOINTS: NULL 1 row in set (0.00 sec) #vi /etc/my.cnf [mysqld] federated = ON [root@mass ~]# service mysqld restart Redirecting to /bin/systemctl restart mysqld.service [root@mass ~]# mysql> select * from information_schema.engines where engine='federated'\G *************************** 1. row *************************** ENGINE: FEDERATED SUPPORT: YES COMMENT: Federated MySQL storage engine TRANSACTIONS: NO XA: NO SAVEPOINTS: NO 1 row in set (0.00 sec) |
How Does it work?
- FEDERATED tables need to be created on a local server and the remote table needs to be created on a remote server.
- Make sure that you have the MySQL port and user access between the local and remote servers.
- Remote tables can be created as MyISAM or InnoDB storage engines.
- The FEDERATED table will not store any data. Data will be stored on the remote server.
- Both local and remote servers should have the same columns and structure.
- You can execute the query on both local or remote servers to modify or retrieve the data.
Example
I have two servers:
- 172.28.128.16 (local server)
- 172.28.128.17 (remote server)
On the local server, I am creating the FEDERATED table:
1 2 3 4 5 6 7 8 9 10 11 | mysql> create table fed_source(id int, name varchar(16)) engine=federated connection="mysql://fed:Fede4!i&[email protected]/percona/fed_destination"; Query OK, 0 rows affected (0.02 sec) mysql> show create table fed_source\G *************************** 1. row *************************** Table: fed_source Create Table: CREATE TABLE `fed_source` ( `id` int(11) DEFAULT NULL, `name` varchar(16) DEFAULT NULL ) ENGINE=FEDERATED DEFAULT CHARSET=latin1 CONNECTION='mysql://fed:Fede4!i&[email protected]/percona/fed_destination' 1 row in set (0.00 sec) |
Syntax is:
1 | “connection=mysql://<user>:<password>@<remote_host_ip>/<remote_database>/<remote_table>” |
On the remote server, I am creating the table with InnoDB engine:
1 2 3 4 5 6 7 8 9 10 11 | mysql> create table fed_destination(id int, name varchar(16)); Query OK, 0 rows affected (0.00 sec) mysql> show create table fed_destination\G *************************** 1. row *************************** Table: fed_destination Create Table: CREATE TABLE `fed_destination` ( `id` int(11) DEFAULT NULL, `name` varchar(16) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=latin1 1 row in set (0.00 sec) |
As I mentioned earlier, the data will be physically stored on the remote server. FEDERATED tables will not store the data. From the below example, you can see the data file (.ibd) was created on the remote server and the local server just has the table structure file ( .frm ).
Local server:
1 2 3 4 5 | [root@mass percona]# pwd /var/lib/mysql/percona [root@mass percona]# ls -lrth total 16K -rw-r-----. 1 mysql mysql 8.4K Mar 19 18:00 fed_source.frm |
Remote server:
1 2 3 4 5 6 | root@repl percona]# pwd /var/lib/mysql/percona [root@repl percona]# ls -lrth total 112K -rw-r-----. 1 mysql mysql 8.4K Mar 19 18:00 fed_destination.frm -rw-r-----. 1 mysql mysql 96K Mar 19 18:01 fed_destination.ibd |
Let’s do this experiment. On the local server, I am inserting the record.
1 2 3 4 5 6 7 8 9 10 | mysql> insert into fed_source values (1,'herc'); Query OK, 1 row affected (0.00 sec) mysql> select * from fed_source; +------+------+ | id | name | +------+------+ | 1 | herc | +------+------+ 1 row in set (0.00 sec) |
And on the remote server:
1 2 3 4 5 6 7 | mysql> select * from fed_destination; +------+------+ | id | name | +------+------+ | 1 | herc | +------+------+ 1 row in set (0.00 sec) |
Now, I am going to update the data on a remote server.
1 2 3 4 5 6 7 8 9 10 11 | mysql> update fed_destination set name='hercules7sakthi' where name='herc'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> select * from fed_destination; +------+-----------------+ | id | name | +------+-----------------+ | 1 | hercules7sakthi | +------+-----------------+ 1 row in set (0.00 sec) |
At the local server:
1 2 3 4 5 6 7 | mysql> select * from fed_source; +------+-----------------+ | id | name | +------+-----------------+ | 1 | hercules7sakthi | +------+-----------------+ 1 row in set (0.00 sec) |
It seems that you can execute the query on both local and remote servers. The FEDERATED Engine is mostly supported for data manipulation languages (INSERT/UPDATE/DELETE/TRUNCATE).
Merge or MRG_MyISAM Engine
Overview:
- The collection of identical MyISAM tables can be used as a single table for better performance.
- Only supported for MyISAM tables.
- Merge tables will use more file descriptors.
- You can’t perform the FULL TEXT SEARCH using the merge tables.
- Merge tables used extremely rare since partitions came around.
How Does it work?
- It works only for the MyISAM tables.
- The columns order, index, data types should be the same on all the tables.
Example
I have created two tables:
1 2 3 4 5 | mysql> create table merge_1(id int, name varchar(16)) engine = myisam; Query OK, 0 rows affected (0.00 sec) mysql> create table merge_2(id int, name varchar(16)) engine = myisam; Query OK, 0 rows affected (0.01 sec) |
Inserting some data on both tables:
1 2 3 4 5 6 7 | mysql> insert into merge_1 values (1,'herc'),(2,'sakthi'),(3,'sri'); Query OK, 3 rows affected (0.00 sec) Records: 3 Duplicates: 0 Warnings: 0 mysql> insert into merge_2 values (4,'jc'),(5,'xxx'),(3,'yyy'); Query OK, 3 rows affected (0.00 sec) Records: 3 Duplicates: 0 Warnings: 0 |
Now, creating the merge table:
1 2 | mysql> create table merge_1_and_2 (id int, name varchar(16)) engine = mrg_myisam union=(merge_1,merge_2); Query OK, 0 rows affected (0.01 sec) |
Let’s query the merge table:
1 2 3 4 5 6 7 8 9 10 11 12 | mysql> select * from merge_1_and_2; +------+--------+ | id | name | +------+--------+ | 1 | herc | | 2 | sakthi | | 3 | sri | | 4 | jc | | 5 | xxx | | 3 | yyy | +------+--------+ 6 rows in set (0.00 sec) |
It seems, when I query the merge table, it merges both the tables (merge_1, merge_2) and displays the results.
Physically, the MERGE table will not occupy any disk space. When querying the table, it will just merge the data from the configured tables and display the result.
1 2 3 4 5 6 7 8 9 10 11 12 13 | [root@mass percona]# ls -lrth total 72K -rw-r-----. 1 mysql mysql 8.4K Mar 19 18:44 merge_1.frm -rw-r-----. 1 mysql mysql 1.9K Mar 19 18:51 merge_1.MYD -rw-r-----. 1 mysql mysql 1.0K Mar 19 18:51 merge_1.MYI -rw-r-----. 1 mysql mysql 8.4K Mar 19 18:44 merge_2.frm -rw-r-----. 1 mysql mysql 1.6K Mar 19 18:51 merge_2.MYD -rw-r-----. 1 mysql mysql 1.0K Mar 19 18:51 merge_2.MYI -rw-r-----. 1 mysql mysql 8.4K Mar 19 18:48 merge_1_and_2.frm -rw-r-----. 1 mysql mysql 16 Mar 19 18:48 merge_1_and_2.MRG |
Blackhole Engine
Overview:
- Blackhole Engine will accept the data from SQL. The accepted data will not be stored, whenever you are querying the data it will give the empty result.
- Can be used for SQL syntax checking purposes.
- Can be used for the replication filter purpose.
- You have to be very careful when you use the table in a replication environment. Because the SQL will be logged in the binary log.
How Does it work?
Example
Creating the blackhole table:
1 2 | mysql> create table black_hole (id int, name varchar(16)) engine = blackhole; Query OK, 0 rows affected (0.00 sec) |
Inserting and retrieving the data:
1 2 3 4 5 6 | mysql> insert into black_hole values (1,'sri'),(2,'jc'); Query OK, 2 rows affected (0.00 sec) Records: 2 Duplicates: 0 Warnings: 0 mysql> select * from black_hole; Empty set (0.00 sec) |
The data will be stored on the binary logs:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | # at 23445 #210319 19:19:15 server id 10 end_log_pos 23497 CRC32 0x36e22a05 Write_rows: table id 115 flags: STMT_END_F ### INSERT INTO `percona`.`black_hole` ### SET ### @1=1 /* INT meta=0 nullable=1 is_null=0 */ ### @2='sri' /* VARSTRING(16) meta=16 nullable=1 is_null=0 */ ### INSERT INTO `percona`.`black_hole` ### SET ### @1=2 /* INT meta=0 nullable=1 is_null=0 */ ### @2='jc' /* VARSTRING(16) meta=16 nullable=1 is_null=0 */ # at 23497 #210319 19:19:15 server id 10 end_log_pos 23573 CRC32 0x4d79cba4 Query thread_id=5 exec_time=0 error_code=0 SET TIMESTAMP=1616181555/*!*/; COMMIT |
Syntax Checking Purposes
If you want to check any syntax of the SQL statements, you can directly execute them against the blackhole tables as it is not going to do anything with the data.
Replication Filter Purpose
Let’s consider that I have a source-replica setup. At the source, I have created the below table.
1 2 | mysql> create table test_blackhole(id int, name varchar(16)) engine=innodb; Query OK, 0 rows affected (0.01 sec) |
I don’t want to replicate this table to replica nodes. In this case, I just converted the table to BLACKHOLE engine on the replica node.
At replica node:
1 2 3 4 5 6 7 8 9 10 11 12 | mysql> alter table test_blackhole engine=blackhole; Query OK, 0 rows affected (0.00 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> show create table test_blackhole\G *************************** 1. row *************************** Table: test_blackhole Create Table: CREATE TABLE `test_blackhole` ( `id` int(11) DEFAULT NULL, `name` varchar(16) DEFAULT NULL ) ENGINE=BLACKHOLE DEFAULT CHARSET=latin1 1 row in set (0.00 sec) |
Now, at source, I am inserting some records:
1 2 3 4 5 6 7 8 9 10 | mysql> insert into test_blackhole values (1,'aaa'); Query OK, 1 row affected (0.00 sec) mysql> select * from test_blackhole; +------+------+ | id | name | +------+------+ | 1 | aaa | +------+------+ 1 row in set (0.00 sec) |
At replica, the data is not available. The data has been ignored as the table was converted to the blackhole engine.
1 2 | mysql> select * from test_blackhole; Empty set (0.00 sec) |
CSV Engine
Overview:
- The CSV storage engine stores data in csv files.
- If you need the data into a CSV file, you can just copy the table physical file and use it. No need to export the data using the command SELECT INTO OUTFILE.
- It will not support nullable columns.
- The data will store comma-separated values.
Example
Creating the CSV table:
1 2 | mysql> create table csv_test (id int not null, name varchar(16) not null, cur_time datetime default current_timestamp not null) engine = csv; Query OK, 0 rows affected (0.00 sec) |
Inserting data:
1 2 3 4 5 6 7 8 9 10 11 12 13 | mysql> insert into csv_test (id,name) values (1,'jc'),(2,'sri'),(3,'herc'); Query OK, 3 rows affected (0.00 sec) Records: 3 Duplicates: 0 Warnings: 0 mysql> select * from csv_test; +----+------+---------------------+ | id | name | cur_time | +----+------+---------------------+ | 1 | jc | 2021-03-19 19:40:40 | | 2 | sri | 2021-03-19 19:40:40 | | 3 | herc | 2021-03-19 19:40:40 | +----+------+---------------------+ 3 rows in set (0.00 sec) |
Physically, you can see the data is stored as the .csv file. You can view the data from the file itself.
1 2 3 4 5 6 7 8 9 | [root@mass percona]# ls -lrth | grep -i csv -rw-r-----. 1 mysql mysql 8.5K Mar 19 19:38 csv_test.frm -rw-r-----. 1 mysql mysql 90 Mar 19 19:40 csv_test.CSV -rw-r-----. 1 mysql mysql 35 Mar 19 19:40 csv_test.CSM [root@mass percona]# cat csv_test.CSV 1,"jc","2021-03-19 19:40:40" 2,"sri","2021-03-19 19:40:40" 3,"herc","2021-03-19 19:40:40" |
As you see, MySQL alternative engines are having some good features. Based on my point of view, I would not suggest having them on production until finding a valid reason. But, it is still good to know about those engines and understand their features.
No mention of rocksdb? Merge and federated engine sound like dead ends. Rocksdb sounds interesting as developed by and used by facebook and potentially allows considerable space savings, useful on large datasets. The blackhole engine does have a few handy use cases for odd situations.
Hi Simon,
Thanks for the comment, In this blog I tried to cover up the engines , which supported by Oracle MySQL Team. Will write the separate blog to describe about the RocksDB. Thanks!