Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-8097 Improved ozone debug and repair tools
  3. HDDS-10295

Provide an "ozone repair" subcommand to update the snapshot info in transactionInfoTable

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • None
    • A new command "ozone repair update-transaction" is added to update the highest index in OM transactionInfoTable.

    Description

      The issue found in HDDS-9342 caused the snapshot info in OM transactionInfoTable not get updated timely, so that OM restart failed at update ID check during raft log reapply.

      The recover solution is to find the largest update ID, and update the snapshot info in transactionInfoTable with this it.

      The task aims to provide such an CLI to update the table.  Be noted, the largest update ID and its term currently should still need manual find.

      Attachments

        Issue Links

          Activity

            Recently erose and I had a discussion about adding state manipulation commands under ozone debug. Based on that, it may be better to add this one under ozone repair instead (which does not exist yet, see HDDS-8100).

            adoroszlai Attila Doroszlai added a comment - Recently erose and I had a discussion about adding state manipulation commands under ozone debug . Based on that, it may be better to add this one under ozone repair instead (which does not exist yet, see HDDS-8100 ).
            szetszwo Tsz-wo Sze added a comment -

            We may use the rocksdb_ldb tool (provided by Rocksdb) for the moment:

            $ rocksdb_ldb --db=/var/lib/hadoop-ozone/om/data/om.db/ scan --column_family=transactionInfoTable
            #TRANSACTIONINFO ==> 11#282615
            $rocksdb_ldb --db=/var/lib/hadoop-ozone/om/data/om.db/ put "#TRANSACTIONINFO" "11#573145" --column_family=transactionInfoTable
            OK
            $rocksdb_ldb --db=/var/lib/hadoop-ozone/om/data/om.db/ scan --column_family=transactionInfoTable
            #TRANSACTIONINFO ==> 11#573145
            

            See also https://github.com/facebook/rocksdb/wiki/RocksDB-Troubleshooting-Guide

            szetszwo Tsz-wo Sze added a comment - We may use the rocksdb_ldb tool (provided by Rocksdb) for the moment: $ rocksdb_ldb --db=/ var /lib/hadoop-ozone/om/data/om.db/ scan --column_family=transactionInfoTable #TRANSACTIONINFO ==> 11#282615 $rocksdb_ldb --db=/ var /lib/hadoop-ozone/om/data/om.db/ put "#TRANSACTIONINFO" "11#573145" --column_family=transactionInfoTable OK $rocksdb_ldb --db=/ var /lib/hadoop-ozone/om/data/om.db/ scan --column_family=transactionInfoTable #TRANSACTIONINFO ==> 11#573145 See also https://github.com/facebook/rocksdb/wiki/RocksDB-Troubleshooting-Guide

            People

              dteng Dave Teng
              Sammi Sammi Chen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: