document using ldb to drop rocksdb column families for downgrade scenarios (#26424)

docs/src/running-validator/validator-troubleshoot.md

@@ -15,3 +15,37 @@ testnet participants, [https://discord.gg/pquxPsq](https://discord.gg/pquxPsq).
 - [Core software repo](https://github.com/solana-labs/solana)
 
 Can't find what you're looking for? Send an email to ryan@solana.com or reach out to @rshea\#2622 on Discord.
+
+## Blockstore
+
+The validator blockstore rocksdb database can be inspected using the `ldb` tool.
+`ldb` is part of the `rocksdb` code base and is also available in the `rocksdb-tools`
+package.
+
+[RocksDB Administration and Data Access Tool](https://github.com/facebook/rocksdb/wiki/Administration-and-Data-Access-Tool)
+
+## Upgrade
+
+If a new software version introduces a new column family to the blockstore,
+that new (empty) column will be automatically created. This is the same logic
+that allows a validator to start fresh without the blockstore directory.
+
+## Downgrade
+
+If a new column family has been introduced to the validator blockstore, a
+subsequent downgrade of the validator to a version that predates the new column
+family will cause the validator to fail while opening the blockstore during
+startup.
+
+List column families:
+```
+ldb --db=<validator ledger path>/rocksdb/ list_column_families
+```
+
+**Warning**: Please seek guidance on discord before modifying the validator
+blockstore.
+
+Drop a column family:
+```
+ldb --db=<validator ledger path>/rocksdb drop_column_family <column family name>
+```