Change a TServer gFlag on the Fly

Gflags, also known as configuration flags, are used in YugabyteDB to manage configurations and feature flags for its primary applications: yb-master (master), yb-tserver (tserver), and postgres.

These gFlags allow you to resolve issues, improve performance, and customize functionality. 

Modifying a gFlag typically involves a rolling restart of the cluster so that the change is persistent after subsequent restarts.

However,  you may want to change a gFlag on a running cluster, and then later during a less busy time, apply the gFlag changes to the configuration file that the server uses to start.

yb-ts-cli is a command line tool that can be used to perform an operation on a particular tablet server (yb-tserver), where one of those operations (set_flag) is to change a gFlag in memory.

Example:

Let’s change the value of the TServer gFlag ysql_log_statement from its current value to “all”.

First, we can view the current value of the gFlag using this YugabyteDB tip.

				
					[root@localhost ~]# curl -s http://127.0.0.1:9000/varz?raw | grep "ysql_log_statement"
--ysql_log_statement=none
				
			

And here’s how to change it (in memory):

				
					[root@localhost ~]# yb-ts-cli --server_address=127.0.0.1:9100 set_flag ysql_log_statement all

[root@localhost ~]# curl -s http://127.0.0.1:9000/varz?raw | grep "ysql_log_statement"
--ysql_log_statement=all
				
			

We need to make the change on all TServers.

For that, we can query the yb_servers() function to get a list of cluster hosts, then build our yb-ts-cli commands dynamically and run them!

				
					[root@localhost ~]# ysqlsh -h 127.0.0.1 -Atc "SELECT 'yb-ts-cli --server_address=' || host || ' set_flag ysql_log_statement all' FROM yb_servers();"
yb-ts-cli --server_address=127.0.0.3 set_flag ysql_log_statement all
yb-ts-cli --server_address=127.0.0.2 set_flag ysql_log_statement all
yb-ts-cli --server_address=127.0.0.1 set_flag ysql_log_statement all

[root@localhost ~]# ysqlsh -h 127.0.0.1 -Atc "SELECT 'yb-ts-cli --server_address=' || host || ' set_flag ysql_log_statement all' FROM yb_servers();" | bash
				
			

And we can simularly verify the changes on all nodes…

				
					[root@localhost ~]# ysqlsh -h 127.0.0.1 -Atc "SELECT 'curl -s ' || host || ':9000/varz?raw | grep "ysql_log_statement"' FROM yb_servers();"
curl -s 127.0.0.3:9000/varz?raw | grep ysql_log_statement
curl -s 127.0.0.2:9000/varz?raw | grep ysql_log_statement
curl -s 127.0.0.1:9000/varz?raw | grep ysql_log_statement
[root@localhost ~]# ysqlsh -h 127.0.0.1 -Atc "SELECT 'curl -s ' || host || ':9000/varz?raw | grep "ysql_log_statement"' FROM yb_servers();" | bash
--ysql_log_statement=all
--ysql_log_statement=all
--ysql_log_statement=all
				
			

Please keep in mind that changing a gFlag in this manner does not persist when a node restarts; you’ll want to make it permanent at some point. 

Have Fun!

My daughter's dog squeezed into my much smaller dog's bed!