Uploaded image for project: 'Hadoop Distributed Data Store'
  1. Hadoop Distributed Data Store
  2. HDDS-4315

Ensure ObjectIDs are unique across restarts

    XMLWordPrintableJSON

    Details

    • Target Version/s:

      Description

      In a non-Ratis OM, the transaction index used to generate ObjectID is reset on OM restart. This can lead to duplicate ObjectIDs when the OM is restarted. ObjectIDs should be unique. 
      HDDS-2939 and NFS are some of the features which depend on ObjectIds being unique.

      To ensure that objectIDs are unique across restarts in non-ratis OM cluster, the transaction index should be updated in DB on every flush to DB. This can be done in a similar fashion to what is being done for ratis enabled cluster today. TransactionInfo table is updated with transaction index as part of every batch write operation to DB.

      Also, and epoch number is introduced to ensure that objectIDs do not clash with older clusters in which this fix does not exist. From the 64 bits of ObjectID (long variable), 2 bits are reserved for epoch and 8 bits for recursive directory creation, if required. The most significant 2 bits of objectIDs is set to epoch. For clusters before HDDS-4315 there is no epoch as such. But it can be safely assumed that the most significant 2 bits of the objectID will be 00 (as it unlikely to reach trxn index > 2^62 in an existing cluster). From HDDS-4315 onwards, the Epoch for non-ratis OM clusters will be binary 01 (= decimal 1) and for ratis enabled OM cluster will be binary 10 (= decimal 2).

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hanishakoneru Hanisha Koneru
                Reporter:
                hanishakoneru Hanisha Koneru
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: