Checking MyRocks 5.6 for regressions with sysbench and a small server, part 2

This post has results from sysbench and many MyRocks versions (old and new) in my search for performance regressions. It uses a small server and my previous post on that is here, which has results for the Insert Benchmark with the same server and same MyRocks versions. I also have results here for sysbench on a medium server.

The tests (microbenchmarks) are split into 5 groups -- 2 for point queries, 2 for range queries, 1 for writes. Using data from the Summary statistics section at the end of the post and the median of the relative throughput values per group to compare modern MyRocks with classic where modern uses source from 2023/05/29 with RocksDB 8.5.0 and classic uses source from 2021/04/07 with RocksDB 6.19. Finally, relative throughput is (QPS for modern / QPS for classic).

• QPS for point queries is 1.01x and 1.07x larger in modern MyRocks vs classic
• QPS for range queries is 1.02x and 1.12x larger in modern MyRocks vs classic
• QPS for writes is similar between modern MyRocks and classic.

Builds

The builds tested are described in a previous post. MyRocks was compiled from source. All tests used the y10a5_bee config.

Benchmark

I used sysbench and my usage is explained here. RocksDB was configured to cached all tables.

This benchmark used the Beelink server explained here that has 8 cores, 16G RAM and 1TB of NVMe SSD with XFS and Ubuntu 22.04. 

The benchmark is run with 1 client and 1 table with 20M rows. The read-only tests ran for 600 seconds each and the other tests ran for 1200 seconds each.

Results

All of the results are here and each chart below is there too (and easier to read there). For the charts below I split the results into two groups of point query tests, two groups of range query tests and one group for read-write/write-only tests. 

The charts use relative QPS which is: (QPS for a given version / QPS for the base case) and the base case is the 20210407 build that has a much longer name, fbmy5635_rel_202104072149, in my previous post. The names used below are the date for which builds were done, YYYMMDD, and then for the last three the three digits at the end (832, 843, 850) are the RocksDB version when I used a more recent version of RocksDB than the build would otherwise use.

Note 

• the y-axis for all charts starts at 0.8 rather than 0.0 to improve readability
• the test names under the x-axis are truncated in the images here but readable here

Point query, group 1

Point query, group 2

Range query, group 1

Range query, group 1

Writes

Summary statistics

Point-1	20230529_850
Average	1.19
Median	1.07
Min	0.94
Max	2.06
Stddev	0.353
Point-2	20230529_850
Average	1.01
Median	1.01
Min	0.95
Max	1.06
Stddev	0.051
Range-1	20230529_850
Average	1.10
Median	1.02
Min	0.80
Max	1.50
Stddev	0.260
Range-2	20230529_850
Average	1.13
Median	1.12
Min	0.85
Max	1.62
Stddev	0.265
Writes	20230529_850
Average	1.01
Median	1.00
Min	0.90
Max	1.14
Stddev	0.075