Making yb-admin list_snapshots Output Valid JSON

Jim Knicely
🧑‍💼 Customer Story

A customer recently automated their backup validation pipeline and wanted to parse snapshot metadata using:

				
					yb-admin list_snapshots show_details JSON

They expected a clean, single JSON document from yb-admin.

Instead, they got:

  • ● A stream of JSON Lines (one per table/namespace), followed by

  • ● A final JSON block containing the list of snapshots

…which means the whole output is not valid JSON and cannot be parsed directly.

They asked:

  • “Is this a bug in yb-admin?”

Short answer:
👉 No … the JSON keyword is undocumented, and the output is not guaranteed to be structured JSON.

But we can reliably produce the JSON structure they expect.

This tip shows how.

❗ Why the output is not valid JSON

Here’s what the customer saw:

				
					yb-admin list_snapshots show_details JSON

Output (simplified):

				
					{detail JSON}
{detail JSON}
{detail JSON}
{
  "snapshots": [
    { "id": "...", ... },
    { "id": "...", ... }
  ]
}

Each detail line is a valid JSON object individually, but the stream as a whole is not valid JSON.

✅ The JSON structure customers actually want
				
					{
  "snapshots": [
    {
      "id": "uuid",
      "state": "COMPLETE",
      "snapshot_time": "...",
      "details": [
        { "type":"NAMESPACE", ... },
        { "type":"TABLE", ... },
        ...
      ]
    }
  ]
}

To get that, we must combine the output of:

  • ● yb-admin list_snapshots

  • ● yb-admin list_snapshots show_details

🐍 Option 1: Python Script (recommended)

Save as yb_snapshots_to_json.py:

				
					#!/usr/bin/env python3
import subprocess, json, re, sys, os

def yb_admin(args):
    master = os.environ.get("YB_MASTER", "127.0.0.1")
    cmd = ["yb-admin", "--init_master_addrs", master] + args
    return subprocess.check_output(cmd, text=True)

def parse_summary(text):
    snaps = {}
    # id state snapshot_time [previous_snapshot_time]
    pat = re.compile(r"^(\S+)\s+(\S+)\s+(\S+\s+\S+)(?:\s+(\S+\s+\S+))?")

    for line in text.splitlines():
        s = line.strip()
        if not s:
            continue
        # skip header / footer lines
        if s.startswith("Snapshot UUID") or s.startswith("No snapshot restorations"):
            continue

        m = pat.match(s)
        if not m:
            continue

        sid, state, ts, prev = m.groups()
        snaps[sid] = {
            "id": sid,
            "state": state,
            "snapshot_time": ts,
            "previous_snapshot_time": prev or None,
            "details": []
        }
    return snaps

def parse_details(text):
    # snapshot header lines start with UUID
    snap_pat = re.compile(r"^([0-9a-f-]+)\s+")
    grouped = {}
    current = None

    for line in text.splitlines():
        s = line.strip()
        if not s:
            continue
        if s.startswith("Snapshot UUID") or s.startswith("No snapshot restorations"):
            continue

        m = snap_pat.match(s)
        if m:
            current = m.group(1)
            grouped.setdefault(current, [])
            continue

        if s.startswith("{") and current:
            grouped[current].append(json.loads(s))
    return grouped

def main():
    summary_txt = yb_admin(["list_snapshots"])
    details_txt = yb_admin(["list_snapshots", "show_details"])

    summary = parse_summary(summary_txt)
    details = parse_details(details_txt)

    for sid in summary:
        summary[sid]["details"] = details.get(sid, [])

    # preserve insertion order from list_snapshots
    snapshots = list(summary.values())

    print(json.dumps({"snapshots": snapshots}, indent=2))

if __name__ == "__main__":
    main()

Usage:

✅ Option 1: Make the script executable (recommended)
				
					chmod +x yb_snapshots_to_json.py
export YB_MASTER=xxx.xxx.xxx.xxx
./yb_snapshots_to_json.py > snapshots.json
✅ Option 2: Run it explicitly with Python (also works)
				
					export YB_MASTERxxx.xxx.xxx.xxx
python3 yb_snapshots_to_json.py > snapshots.json

or, depending on the system:

				
					export YB_MASTERxxx.xxx.xxx.xxx
python yb_snapshots_to_json.py > snapshots.json

Example run:

First, here is the standard output from the yb-admin list_snapshots show_details command: 
				
					[root@localhost ~]# yb-admin --init_master_addrs 127.0.0.1 list_snapshots show_details
Snapshot UUID                    	State 	 	Creation Time
e3cd52fb-0350-48e7-ba8f-b82ea26df28a 	COMPLETE 	2025-11-23 03:52:56.308819
 	{"type":"NAMESPACE","id":"000034cb000030008000000000000000","data":{"name":"yugabyte","database_type":"YQL_DATABASE_PGSQL","next_normal_pg_oid":16640,"colocated":false,"state":"RUNNING","ysql_next_major_version_state":"NEXT_VER_RUNNING"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004012","data":{"name":"orders","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004012"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004019","data":{"name":"risk","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004019"}}
 	{"type":"TABLE","id":"000034cb00003000800000000000401e","data":{"name":"risk_envelopes","version":0,"state":"RUNNING","next_column_id":4,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb00003000800000000000401e"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004025","data":{"name":"test","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004025"}}
4444301c-5e66-4832-b8f3-4b0030b84188 	COMPLETE 	2025-11-23 03:55:46.552879
 	{"type":"NAMESPACE","id":"000034cb000030008000000000000000","data":{"name":"yugabyte","database_type":"YQL_DATABASE_PGSQL","next_normal_pg_oid":16640,"colocated":false,"state":"RUNNING","ysql_next_major_version_state":"NEXT_VER_RUNNING"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004012","data":{"name":"orders","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004012"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004019","data":{"name":"risk","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004019"}}
 	{"type":"TABLE","id":"000034cb00003000800000000000401e","data":{"name":"risk_envelopes","version":0,"state":"RUNNING","next_column_id":4,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb00003000800000000000401e"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004025","data":{"name":"test","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004025"}}
No snapshot restorations

And here is the output with the additional JSON argument:

				
					[root@localhost ~]# yb-admin --init_master_addrs 127.0.0.1 list_snapshots show_details JSON
 	{"type":"NAMESPACE","id":"000034cb000030008000000000000000","data":{"name":"yugabyte","database_type":"YQL_DATABASE_PGSQL","next_normal_pg_oid":16640,"colocated":false,"state":"RUNNING","ysql_next_major_version_state":"NEXT_VER_RUNNING"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004012","data":{"name":"orders","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004012"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004019","data":{"name":"risk","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004019"}}
 	{"type":"TABLE","id":"000034cb00003000800000000000401e","data":{"name":"risk_envelopes","version":0,"state":"RUNNING","next_column_id":4,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb00003000800000000000401e"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004025","data":{"name":"test","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004025"}}
 	{"type":"NAMESPACE","id":"000034cb000030008000000000000000","data":{"name":"yugabyte","database_type":"YQL_DATABASE_PGSQL","next_normal_pg_oid":16640,"colocated":false,"state":"RUNNING","ysql_next_major_version_state":"NEXT_VER_RUNNING"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004012","data":{"name":"orders","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004012"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004019","data":{"name":"risk","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004019"}}
 	{"type":"TABLE","id":"000034cb00003000800000000000401e","data":{"name":"risk_envelopes","version":0,"state":"RUNNING","next_column_id":4,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb00003000800000000000401e"}}
 	{"type":"TABLE","id":"000034cb000030008000000000004025","data":{"name":"test","version":0,"state":"RUNNING","next_column_id":2,"table_type":"PGSQL_TABLE_TYPE","namespace_id":"000034cb000030008000000000000000","namespace_name":"yugabyte","pg_table_id":"000034cb000030008000000000004025"}}
{
    "snapshots": [
        {
            "id": "e3cd52fb-0350-48e7-ba8f-b82ea26df28a",
            "state": "COMPLETE",
            "snapshot_time": "2025-11-23 03:52:56.308819",
            "previous_snapshot_time": "2112-09-17 23:53:47.370495"
        },
        {
            "id": "4444301c-5e66-4832-b8f3-4b0030b84188",
            "state": "COMPLETE",
            "snapshot_time": "2025-11-23 03:55:46.552879",
            "previous_snapshot_time": "2112-09-17 23:53:47.370495"
        }
    ]
}

Now let’s try the handy Python script…

				
					[root@localhost ~]# export YB_MASTER=127.0.0.1

[root@localhost ~]# chmod +x yb_snapshots_to_json.py

[root@localhost ~]# ./yb_snapshots_to_json.py > snapshots.json

[root@localhost ~]# cat snapshots.json
{
  "snapshots": [
    {
      "id": "e3cd52fb-0350-48e7-ba8f-b82ea26df28a",
      "state": "COMPLETE",
      "snapshot_time": "2025-11-23 03:52:56.308819",
      "previous_snapshot_time": null,
      "details": [
        {
          "type": "NAMESPACE",
          "id": "000034cb000030008000000000000000",
          "data": {
            "name": "yugabyte",
            "database_type": "YQL_DATABASE_PGSQL",
            "next_normal_pg_oid": 16640,
            "colocated": false,
            "state": "RUNNING",
            "ysql_next_major_version_state": "NEXT_VER_RUNNING"
          }
        },
        {
          "type": "TABLE",
          "id": "000034cb000030008000000000004012",
          "data": {
            "name": "orders",
            "version": 0,
            "state": "RUNNING",
            "next_column_id": 2,
            "table_type": "PGSQL_TABLE_TYPE",
            "namespace_id": "000034cb000030008000000000000000",
            "namespace_name": "yugabyte",
            "pg_table_id": "000034cb000030008000000000004012"
          }
        },
        {
          "type": "TABLE",
          "id": "000034cb000030008000000000004019",
          "data": {
            "name": "risk",
            "version": 0,
            "state": "RUNNING",
            "next_column_id": 2,
            "table_type": "PGSQL_TABLE_TYPE",
            "namespace_id": "000034cb000030008000000000000000",
            "namespace_name": "yugabyte",
            "pg_table_id": "000034cb000030008000000000004019"
          }
        },
        {
          "type": "TABLE",
          "id": "000034cb00003000800000000000401e",
          "data": {
            "name": "risk_envelopes",
            "version": 0,
            "state": "RUNNING",
            "next_column_id": 4,
            "table_type": "PGSQL_TABLE_TYPE",
            "namespace_id": "000034cb000030008000000000000000",
            "namespace_name": "yugabyte",
            "pg_table_id": "000034cb00003000800000000000401e"
          }
        },
        {
          "type": "TABLE",
          "id": "000034cb000030008000000000004025",
          "data": {
            "name": "test",
            "version": 0,
            "state": "RUNNING",
            "next_column_id": 2,
            "table_type": "PGSQL_TABLE_TYPE",
            "namespace_id": "000034cb000030008000000000000000",
            "namespace_name": "yugabyte",
            "pg_table_id": "000034cb000030008000000000004025"
          }
        }
      ]
    },
    {
      "id": "4444301c-5e66-4832-b8f3-4b0030b84188",
      "state": "COMPLETE",
      "snapshot_time": "2025-11-23 03:55:46.552879",
      "previous_snapshot_time": null,
      "details": [
        {
          "type": "NAMESPACE",
          "id": "000034cb000030008000000000000000",
          "data": {
            "name": "yugabyte",
            "database_type": "YQL_DATABASE_PGSQL",
            "next_normal_pg_oid": 16640,
            "colocated": false,
            "state": "RUNNING",
            "ysql_next_major_version_state": "NEXT_VER_RUNNING"
          }
        },
        {
          "type": "TABLE",
          "id": "000034cb000030008000000000004012",
          "data": {
            "name": "orders",
            "version": 0,
            "state": "RUNNING",
            "next_column_id": 2,
            "table_type": "PGSQL_TABLE_TYPE",
            "namespace_id": "000034cb000030008000000000000000",
            "namespace_name": "yugabyte",
            "pg_table_id": "000034cb000030008000000000004012"
          }
        },
        {
          "type": "TABLE",
          "id": "000034cb000030008000000000004019",
          "data": {
            "name": "risk",
            "version": 0,
            "state": "RUNNING",
            "next_column_id": 2,
            "table_type": "PGSQL_TABLE_TYPE",
            "namespace_id": "000034cb000030008000000000000000",
            "namespace_name": "yugabyte",
            "pg_table_id": "000034cb000030008000000000004019"
          }
        },
        {
          "type": "TABLE",
          "id": "000034cb00003000800000000000401e",
          "data": {
            "name": "risk_envelopes",
            "version": 0,
            "state": "RUNNING",
            "next_column_id": 4,
            "table_type": "PGSQL_TABLE_TYPE",
            "namespace_id": "000034cb000030008000000000000000",
            "namespace_name": "yugabyte",
            "pg_table_id": "000034cb00003000800000000000401e"
          }
        },
        {
          "type": "TABLE",
          "id": "000034cb000030008000000000004025",
          "data": {
            "name": "test",
            "version": 0,
            "state": "RUNNING",
            "next_column_id": 2,
            "table_type": "PGSQL_TABLE_TYPE",
            "namespace_id": "000034cb000030008000000000000000",
            "namespace_name": "yugabyte",
            "pg_table_id": "000034cb000030008000000000004025"
          }
        }
      ]
    }
  ]
}

That’s more like it!!!

🔎 Note on previous_snapshot_time

previous_snapshot_time is only populated for snapshots created by a snapshot schedule (create_snapshot_schedule).

Manual snapshots (create_database_snapshot, create_table_snapshot) will always show null for this field.
This is expected behavior.

This is expected behavior.

🛠 Option 2: Bash + jq (no Python required)

Save as yb_snapshots_to_json.sh:

				
					#!/bin/bash
set -e

MASTER="${YB_MASTER:-127.0.0.1}"

summary=$(yb-admin --init_master_addrs "$MASTER" list_snapshots)
details=$(yb-admin --init_master_addrs "$MASTER" list_snapshots show_details)

# ---- 1) Extract summary rows as JSON array ----
summary_json=$(echo "$summary" |
  awk '
    /^[0-9a-f-]+\s/ {
      sid=$1
      state=$2
      ts=$3" "$4
      prev=""

      # If there are 6+ fields, assume previous_snapshot_time is fields 5+6
      if (NF >= 6) {
        prev=$5" "$6
        gsub(/^ +| +$/, "", prev)
      }

      if (prev == "") {
        prev_json="null"
      } else {
        prev_json="\""prev"\""
      }

      printf("{\"id\":\"%s\",\"state\":\"%s\",\"snapshot_time\":\"%s\",\"previous_snapshot_time\":%s}\n",
             sid, state, ts, prev_json)
    }
  ' | jq -s '.')

# ---- 2) Extract detail JSON lines grouped by snapshot ID ----
details_json=$(echo "$details" |
  awk '
    /^[0-9a-f-]+[ \t]/ { sid=$1; next }
    /^[ \t]*\{/ {
      # strip leading whitespace before the JSON
      gsub(/^[ \t]+/, "", $0)
      # print "snapshot_id<TAB>{json...}"
      print sid "\t" $0
    }
  ' |
  jq -R '
    # split on TAB into [id, json_text]
    split("\t") |
    { id: .[0], detail: (.[1] | fromjson) }
  ' |
  jq -s '
    group_by(.id)
    | map({id: .[0].id, details: map(.detail)})
  ')

# ---- 3) Merge summary + details by id ----
jq -n \
  --argjson summary "$summary_json" \
  --argjson details "$details_json" '
{
  snapshots:
    $summary
    | map(. as $s |
        . + {
          details: (
            $details
            | map(select(.id == $s.id))
            | if length > 0 then .[0].details else [] end
          )
        }
      )
}
'

Usage:

				
					export YB_MASTER=xxx.xxx.xxx.xxx
chmod +x yb_snapshots_to_json.sh
./yb_snapshots_to_json.sh > snapshots.json

The Bash + jq script produces the exact same JSON structure as the Python helper script, so the output is shown only once.

🎉 Summary

  • ● list_snapshots show_details JSON is not a bug, but not documented and doesn’t produce valid JSON.

  • ● You can get fully structured JSON using:

    • ○ list_snapshots

    • ○ list_snapshots show_details

    • ○ One of the helper scripts above

  • ● Using --init_master_addrs means you only need one master IP, making scripts portable and simple.

Have Fun!

Meet my new at-home YB mascots. They don’t say much, but they’re solid listeners. 🐻🐘