Check for Zombie Processes After OS Patching

Routine OS patching should be boring. Patch the host, reboot or restart services, confirm YugabyteDB is healthy, and move on.

But there is one small pre-start check that can save a lot of confusion:

  • Make sure no leftover YugabyteDB processes are still holding the master or tserver RPC ports.

The Issue

YugabyteDB processes bind to specific TCP ports.

By default:

Process Default RPC Port
yb-master 7100
yb-tserver 9100

Because these are TCP ports, the operating system only allows one process to bind to each port at a time.

So if a previous yb-master or yb-tserver process is still running after patching, the real service may fail to start with an error like:

				
					Address already in use
				
			

That message can send you down the wrong troubleshooting path, especially after a maintenance window.

Why This Can Happen

After an OS patch, reboot, or service restart, a previous YugabyteDB process may still be holding the port if:

  • ● The process did not shut down cleanly.
  • ● A monitoring job, cron job, or external automation restarted the process during the patch window.
  • ● Multiple startup mechanisms are competing with each other, such as systemd plus a custom script.
  • ● A maintenance workflow stopped the service, but another automation layer brought it back too early.

The result is that when the intended service startup occurs, the port is already taken.

Quick Pre-Start Check

Before starting YugabyteDB services after OS patching, check that the expected ports are free.

For YB-Master:

				
					sudo lsof -i :7100
				
			

For YB-TServer:

				
					sudo lsof -i :9100
				
			

If the commands returns nothing, the ports are free.

If either returns a process, review it before proceeding. For example:

				
					COMMAND     PID      USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
yb-master  12345 yugabyte  123u  IPv4  ...    0t0      TCP *:7100 (LISTEN)
				
			

That means something is already listening on the port.

Clear the Leftover Process

If you confirm the processes are stale and should not be running, stop them cleanly when possible:

				
					sudo systemctl stop yb-master
sudo systemctl stop yb-tserver
				
			

Then re-check the ports:

				
					sudo lsof -i :7100
sudo lsof -i :9100
				
			

If a stale process is still holding the port, identify the PID and terminate it:

				
					sudo kill <pid>
				
			

If it does not exit, use the stronger option only after confirming it is safe:

				
					sudo kill -9 <pid>
				
			

Then confirm the port is free again:

				
					sudo lsof -i :7100
sudo lsof -i :9100
				
			

Recommended Maintenance Flow

A simple post-patching checklist can help avoid confusing startup failures:

				
					# Stop services
sudo systemctl stop yb-master
sudo systemctl stop yb-tserver

# Confirm ports are free
sudo lsof -i :7100
sudo lsof -i :9100

# Kill only confirmed stale processes, if needed
sudo kill <pid>

# Start services
sudo systemctl start yb-master
sudo systemctl start yb-tserver

# Confirm status
sudo systemctl status yb-master
sudo systemctl status yb-tserver
				
			

Best Practice

Before restarting YugabyteDB after OS patching, verify that no stale yb-master or yb-tserver process is still listening on the default RPC ports.

This check only takes a few seconds, but it can prevent a misleading startup failure.

Key Insight

An Address already in use error after patching does not always mean YugabyteDB has a configuration problem. It may simply mean an old or accidentally restarted process is still holding the port.

Final Takeaway

After OS patching, do not assume the ports are free just because the service was supposed to stop.

Check first:

				
					sudo lsof -i :7100
sudo lsof -i :9100
				
			

If a stale process is still holding the port, clear it before starting the real service.

It is a small operational habit that can save time, reduce confusion, and make YugabyteDB maintenance windows smoother.

Have Fun!

Great day in Omaha with my favorite customer. Always appreciate the conversations, the partnership, and the chance to spend time with such a fantastic team.