Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: docs
    • Labels:

      Description

      Document bulk-import based recovery of accumulo.
      Include a stern warning and possible data loss and unwanted data recovery.

        Activity

        Hide
        elserj Josh Elser added a comment -

        Eric Newton re-importing files that were candidates for deletion by the garbage collector is one example that I can think about of unwanted data being reintroduced. Likewise, WALs that haven't been recovered could result in data loss.

        Any other cases that I'm missing?

        Show
        elserj Josh Elser added a comment - Eric Newton re-importing files that were candidates for deletion by the garbage collector is one example that I can think about of unwanted data being reintroduced. Likewise, WALs that haven't been recovered could result in data loss. Any other cases that I'm missing?
        Hide
        elserj Josh Elser added a comment -

        First stab at documentation on how to recover from ZK failure.

        Show
        elserj Josh Elser added a comment - First stab at documentation on how to recover from ZK failure.
        Hide
        ecn Eric Newton added a comment -

        Josh Elser you can also get data back that had been compacted out.

        Let's say we have a table in a single tablet that contains rows [a, b, d... x, y, z]. The tablet splits to (-inf, m] and (m, +inf). Both tablets use the file that contains [a...z].

        Now I delete row a with a bunch of delete markers. The table contains [b, c, ... z].

        I compact (-inf, m]. So I have [b...m] and [n...z] in separate tablets. But remember, row a still lives in a file pointed to by (m, +inf).

        If zookeeper dies, and I re-import files. Row a will return.

        Show
        ecn Eric Newton added a comment - Josh Elser you can also get data back that had been compacted out. Let's say we have a table in a single tablet that contains rows [a, b, d... x, y, z] . The tablet splits to (-inf, m] and (m, +inf). Both tablets use the file that contains [a...z] . Now I delete row a with a bunch of delete markers. The table contains [b, c, ... z] . I compact (-inf, m]. So I have [b...m] and [n...z] in separate tablets. But remember, row a still lives in a file pointed to by (m, +inf). If zookeeper dies, and I re-import files. Row a will return.
        Hide
        elserj Josh Elser added a comment -

        Eric Newton, gotcha. I have a brief call-out to deletions being reintroduced when they haven't been written out to disk. I'll re-read with your response in mind the get this pushed. Thanks!

        Show
        elserj Josh Elser added a comment - Eric Newton , gotcha. I have a brief call-out to deletions being reintroduced when they haven't been written out to disk. I'll re-read with your response in mind the get this pushed. Thanks!
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1c516193342acfa838df25bc880e3c594a659282 in accumulo's branch refs/heads/1.6.0-SNAPSHOT from Josh Elser
        [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1c51619 ]

        ACCUMULO-1218 Overview on how to recover an instance from failed zookeepers

        Ample warning given to the reintroduction of stale data (from files
        that should be deleted but have not yet been deleted) or omission
        of new data only present in WALs.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1c516193342acfa838df25bc880e3c594a659282 in accumulo's branch refs/heads/1.6.0-SNAPSHOT from Josh Elser [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1c51619 ] ACCUMULO-1218 Overview on how to recover an instance from failed zookeepers Ample warning given to the reintroduction of stale data (from files that should be deleted but have not yet been deleted) or omission of new data only present in WALs.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1c516193342acfa838df25bc880e3c594a659282 in accumulo's branch refs/heads/master from Josh Elser
        [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1c51619 ]

        ACCUMULO-1218 Overview on how to recover an instance from failed zookeepers

        Ample warning given to the reintroduction of stale data (from files
        that should be deleted but have not yet been deleted) or omission
        of new data only present in WALs.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1c516193342acfa838df25bc880e3c594a659282 in accumulo's branch refs/heads/master from Josh Elser [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1c51619 ] ACCUMULO-1218 Overview on how to recover an instance from failed zookeepers Ample warning given to the reintroduction of stale data (from files that should be deleted but have not yet been deleted) or omission of new data only present in WALs.
        Hide
        elserj Josh Elser added a comment -

        Added some more clarification from Eric's scenario over the attached patch.

        Show
        elserj Josh Elser added a comment - Added some more clarification from Eric's scenario over the attached patch.

          People

          • Assignee:
            elserj Josh Elser
            Reporter:
            ecn Eric Newton
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development