Measuring the potential overhead of PMM Client on MySQL workloads

Having good historial metrics monitoring in place is critical for properly operating, maintaining and troubleshooting database systems, and
Percona Monitoring and Management is one of the options we recommend to our clients for this. One common concern among potential users is how using this may impact their database's performance. As I could not find any conclusive information about this, I set out to do some basic tests and this post shows my results. To begin, let me describe my setup. I used the following Google Cloud instances:
We can see that with SSL enabled there is a noticeable drop in throughput when the exporters are running, while this is not the case when SSL is disabled. I arrived at the conclusion that it was worth repeating the tests with SSL disabled after creating
Flame Graphs from perf captures during sample runs. On them, the only significant increases were due to the exporters (mysqld_exporter and node_exporter, the qan exporter did not have any noticeable impact during my tests). The results from the tests show that this analysis pointed me in the right direction so while they are worth of separate blog posts, it is worth to at least recommend our readers to get familiar with this performance analysis tool. Next is a scatter plot of throughput over time with ssl enabled:
On it we get a more clear picture of the impact of having the exporters running during the test. Next is the same graphs but with SSL disabled:
Now it is much more difficult to differentiate the runs. This is confirmed if we look at the 99 percentile of throughput for each case (here for 32 threads):
- One 4 vCPU instance for the MySQL server
- One 2 vCPU instance for the sysbench client
- One 1 vCPU instance for the PMM server



PMM | SSL | tps (p99) |
enabled | enabled | 1167.518 |
enabled | disabled | 1397.397 |
disabled | disabled | 1429.097 |