Automatic tablet splitting allows a cluster to reshard data online and transparently once a specified size threshold is reached.
This feature is available for both YSQL and YCQL tables.
Note that tablet splitting is currently disabled for YCQL tables utilizing the TTL file expiration feature.
What does that mean exactly?
The best way to understand when tablet splitting is disabled for YCQL tables is through an example.
Let’s start a YugabyteDB cluster with a single node using yugabyted. For this cluster, we’ll enable the TTL file expiration feature and set an artificially low threshold for tablet splitting during the ‘high phase’ of automatic tablet splitting.
To enable the TTL file expiration feature, we must set the TServer and Master gFlag tablet_enable_ttl_file_filter to true and the TServer and Master gFlag rocksdb_max_file_size_for_compaction to a value greater than 0.
[root@cloud-server-0 ~]# yugabyted start --tserver_flags="tablet_enable_ttl_file_filter=true,tablet_split_high_phase_size_threshold_bytes=300000000,rocksdb_max_file_size_for_compaction=100" --master_flags="tablet_enable_ttl_file_filter=true,tablet_split_high_phase_size_threshold_bytes=300000000,rocksdb_max_file_size_for_compaction=100" > start.log
Next, we’ll create two YCQL tables: one with TTL enabled and the other without.
[root@cloud-server-0 ~]# ycqlsh $(hostname -I) -e "CREATE KEYSPACE ks;"
[root@cloud-server-0 ~]# ycqlsh $(hostname -I) -e "CREATE TABLE ks.demo(id INT PRIMARY KEY, data TEXT);"
[root@cloud-server-0 ~]# ycqlsh $(hostname -I) -e "CREATE TABLE ks.demo_ttl(id INT PRIMARY KEY, data TEXT) WITH default_time_to_live = 86400;"
We can use the YSQL API to query the yb_local_tablets view and check the tablet count for each of the YCQL tables we just created.
[root@cloud-server-0 ~]# ysqlsh -h $(hostname -I) -c "SELECT table_name, COUNT(tablet_id) tablet_count FROM yb_local_tablets WHERE table_type = 'YCQL' AND table_name IN ('demo', 'demo_ttl') GROUP BY table_name ORDER BY table_name;"
table_name | tablet_count
------------+--------------
demo | 1
demo_ttl | 1
(2 rows)
Next, let’s load data into both tables and observe if either splits into new tablets.
[root@cloud-server-0 ~]# ls -lh /root/demo.txt
-rw-r--r--. 1 root root 1.2G Jan 1 17:15 /root/demo.txt
root@cloud-server-0 ~]# ycqlsh $(hostname -I) -e "COPY ks.demo FROM '/root/demo.txt' WITH DELIMITER='|';"
Using 5 child processes
Starting copy of ks.demo with columns [id, data].
Processed: 76800 rows; Rate: 458 rows/s; Avg. rate: 1007 rows/s
76800 rows imported from 1 files in 0 day, 0 hour, 1 minutes, and 16.244 seconds (0 skipped).
[root@cloud-server-0 ~]# ycqlsh $(hostname -I) -e "COPY ks.demo_ttl FROM '/root/demo.txt' WITH DELIMITER='|';"
Using 5 child processes
Starting copy of ks.demo_ttl with columns [id, data].
Processed: 76800 rows; Rate: 461 rows/s; Avg. rate: 1007 rows/s
76800 rows imported from 1 files in 0 day, 0 hour, 1 minutes, and 16.255 seconds (0 skipped).
[root@cloud-server-0 ~]# ysqlsh -h $(hostname -I) -c "SELECT table_name, COUNT(tablet_id) tablet_count FROM yb_local_tablets WHERE table_type = 'YCQL' AND table_name IN ('demo', 'demo_ttl') GROUP BY table_name ORDER BY table_name;"
table_name | tablet_count
------------+--------------
demo | 6
demo_ttl | 1
(2 rows)
Note that the demo table with the default TTL of 0 seconds has split, while the demo_ttl table with a TTL greater than 0 has not.
Have Fun!
