Uploaded image for project: 'Accumulo'
  1. Accumulo
  2. ACCUMULO-4496

Upgrade fails due to outstanding Fate operations, refers to nonexistant explaination of how to handle

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.7.2
    • Fix Version/s: 1.7.4, 1.8.2, 2.0.0
    • Component/s: None
    • Labels:
      None

      Description

      2016-10-11 15:24:30,023 [server.Accumulo] FATAL: Problem verifying Fate readiness
      org.apache.accumulo.core.client.AccumuloException: Aborting upgrade because there are outstanding FATE transactions from a previous Accumulo version. Please see the README document for instructions on w
      hat to do under your previous version.
              at org.apache.accumulo.server.Accumulo.abortIfFateTransactions(Accumulo.java:317)
              at org.apache.accumulo.master.Master.upgradeZookeeper(Master.java:343)
              at org.apache.accumulo.master.Master.setMasterState(Master.java:283)
              at org.apache.accumulo.master.Master.getMasterLock(Master.java:1372)
              at org.apache.accumulo.master.Master.run(Master.java:1113)
              at com.sqrrl.analytics.server.admin.SqrrlMasterStart$MasterShim.run(SqrrlMasterStart.java:99)
              at java.lang.Thread.run(Thread.java:745)

      However, there is no mention of this situation in README.md, nor in the UPGRADING.md. Not sure what the appropriate response is to this situation, but we should document it.

        Issue Links

          Activity

          Hide
          elserj Josh Elser added a comment -

          Hah! Funny, I think I was just talking to Sean Busbey about this today IRL.

          He noticed that the completed, but not reaped, FATE transactions could cause an upgrade to fail. Instead of trying to parse the FATE op and ignore COMPLETE operations, we just fail if any are found. He suggested that we could attempt to parse the FATE op and ignore those which are completed, failing only when there is an unparse-able or non-completed one.

          This sounded like a good idea to me.

          FYI Keith Turner

          Show
          elserj Josh Elser added a comment - Hah! Funny, I think I was just talking to Sean Busbey about this today IRL. He noticed that the completed, but not reaped, FATE transactions could cause an upgrade to fail. Instead of trying to parse the FATE op and ignore COMPLETE operations, we just fail if any are found. He suggested that we could attempt to parse the FATE op and ignore those which are completed, failing only when there is an unparse-able or non-completed one. This sounded like a good idea to me. FYI Keith Turner
          Hide
          busbey Sean Busbey added a comment -

          Yep. For those who stumble here and need to work around:

          • Start tservers
          • Start shell
          • Run fate print to list all
          • If completed, just delete with fate delete
          • Start masters once there are no more fate operations

          If any of the operations are not complete, you should rollback the upgrade and troubleshoot completing them with your prior version.

          Show
          busbey Sean Busbey added a comment - Yep. For those who stumble here and need to work around: Start tservers Start shell Run fate print to list all If completed, just delete with fate delete Start masters once there are no more fate operations If any of the operations are not complete, you should rollback the upgrade and troubleshoot completing them with your prior version.

            People

            • Assignee:
              Unassigned
              Reporter:
              vines John Vines
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development