Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: None
    • Labels:
      None

      Description

      Need to verify that 1.6.0 release candidate can upgrade from 1.5.0

      1. ACCUMULO-2148.1.patch.txt
        3 kB
        Sean Busbey
      2. ACCUMULO-2148.2.patch.txt
        3 kB
        Sean Busbey

        Issue Links

          Activity

          Hide
          Sean Busbey added a comment -

          Safe to assume that our upgrade path for 1.4.x is to first do 1.4.x -> 1.5.x and then 1.5.x -> 1.6.0 (and not direct from 1.4.x -> 1.6.x)?

          Show
          Sean Busbey added a comment - Safe to assume that our upgrade path for 1.4.x is to first do 1.4.x -> 1.5.x and then 1.5.x -> 1.6.0 (and not direct from 1.4.x -> 1.6.x)?
          Hide
          Keith Turner added a comment -

          Upgrading directly from 1.4.x to 1.6.x is not supported. We should probably verify that 1.6 does nothing if given a 1.4 instance.

          Show
          Keith Turner added a comment - Upgrading directly from 1.4.x to 1.6.x is not supported. We should probably verify that 1.6 does nothing if given a 1.4 instance.
          Hide
          Sean Busbey added a comment -

          I can verify that given a 1.4 instance, 1.6.0-SNAPSHOT properly says "Nooooooope."

          The error message is the generic "This version of Accumulo doesn't work with the version from your data" from server/base/src/main/java/org/apache/accumulo/server/Accumulo.java

              if (dataVersion != ServerConstants.DATA_VERSION && dataVersion != ServerConstants.PREV_DATA_VERSION) {
                throw new RuntimeException("This version of accumulo (" + codeVersion + ") is not compatible with files stored using data version " + dataVersion);
              }
          

          Does this upgrade just need to be manual? Would an upgrade from 1.5.1-rc2 be sufficient?

          Show
          Sean Busbey added a comment - I can verify that given a 1.4 instance, 1.6.0-SNAPSHOT properly says "Nooooooope." The error message is the generic "This version of Accumulo doesn't work with the version from your data" from server/base/src/main/java/org/apache/accumulo/server/Accumulo.java if (dataVersion != ServerConstants.DATA_VERSION && dataVersion != ServerConstants.PREV_DATA_VERSION) { throw new RuntimeException( "This version of accumulo (" + codeVersion + ") is not compatible with files stored using data version " + dataVersion); } Does this upgrade just need to be manual? Would an upgrade from 1.5.1-rc2 be sufficient?
          Hide
          Christopher Tubbs added a comment -

          An upgrade from any 1.5.x to any 1.6.x should work.

          Show
          Christopher Tubbs added a comment - An upgrade from any 1.5.x to any 1.6.x should work.
          Hide
          Sean Busbey added a comment -

          In flight patch for ACCUMULO-2451 (as seen on the reviewboard for ACCUMULO-2061) changes the upgrade process, so this ticket needs to wait for that to land.

          Show
          Sean Busbey added a comment - In flight patch for ACCUMULO-2451 (as seen on the reviewboard for ACCUMULO-2061 ) changes the upgrade process, so this ticket needs to wait for that to land.
          Hide
          Sean Busbey added a comment -

          As a part of this, I plan to update the README's section on upgrading. Right now it still gives the 1.4 -> 1.5 instructions.

          Besides the instructions about Fate that will be added as a part of ACCUMULO-2519, are there any other special considerations?

          Show
          Sean Busbey added a comment - As a part of this, I plan to update the README's section on upgrading. Right now it still gives the 1.4 -> 1.5 instructions. Besides the instructions about Fate that will be added as a part of ACCUMULO-2519 , are there any other special considerations?
          Hide
          Josh Elser added a comment -

          I think how to upgrade with the final intent of configuring multiple volumes should be captured somewhere.

          Show
          Josh Elser added a comment - I think how to upgrade with the final intent of configuring multiple volumes should be captured somewhere.
          Hide
          Sean Busbey added a comment -

          Josh Elser, that sounds like something that should be covered in the user manual chapter on multi volume set ups.

          For one thing, I would expect to do such an upgrade in two steps

          1. Upgrade 1.5.x -> 1.6.x and verify
          2. move from single volume to multi volume and verify

          This looks, to me, like a distinct step unrelated to upgrades.

          Furthermore, as the user manual chapter points out, multivolume configurations are an advanced option only needed at a particular scale. It's not something we should be calling out in the README for common upgrades.

          Can you please file a follow on ticket?

          Show
          Sean Busbey added a comment - Josh Elser , that sounds like something that should be covered in the user manual chapter on multi volume set ups . For one thing, I would expect to do such an upgrade in two steps Upgrade 1.5.x -> 1.6.x and verify move from single volume to multi volume and verify This looks, to me, like a distinct step unrelated to upgrades. Furthermore, as the user manual chapter points out, multivolume configurations are an advanced option only needed at a particular scale. It's not something we should be calling out in the README for common upgrades. Can you please file a follow on ticket?
          Hide
          Keith Turner added a comment -

          The two step process is what I would do. Could make the upgrade abort if
          instance.volumes is configured w/ a informative message.

          Show
          Keith Turner added a comment - The two step process is what I would do. Could make the upgrade abort if instance.volumes is configured w/ a informative message.
          Hide
          Josh Elser added a comment -

          Didn't realize that documentation was already added to the user manual for it (kudos to those involved). Configuration of volumes before the upgrade process is complete is exactly what I was thinking about (mostly, because I did it myself at least once).

          Show
          Josh Elser added a comment - Didn't realize that documentation was already added to the user manual for it (kudos to those involved). Configuration of volumes before the upgrade process is complete is exactly what I was thinking about (mostly, because I did it myself at least once).
          Hide
          Sean Busbey added a comment -

          Proposed updated text.

          Show
          Sean Busbey added a comment - Proposed updated text.
          Hide
          Josh Elser added a comment -

          Two minor changes, if you could:

          1. Clarify the "if your instance previously upgraded from 1.4 to 1.5" statement to only apply to recent upgrades (e.g. the 1.4->1.6 path). That may be confusing for people who have been on 1.5 for some time. Maybe something like "For those attempting to upgrade to 1.6 from 1.4 through 1.5, ..."? That seems a bit wordy too.
          2. In the last paragraph, quote "metadata" to be clear that we're referring to the table named "metadata" and not "table metadata".

          Otherwise, looks good. Very thorough.

          Show
          Josh Elser added a comment - Two minor changes, if you could: 1. Clarify the "if your instance previously upgraded from 1.4 to 1.5" statement to only apply to recent upgrades (e.g. the 1.4->1.6 path). That may be confusing for people who have been on 1.5 for some time. Maybe something like "For those attempting to upgrade to 1.6 from 1.4 through 1.5, ..."? That seems a bit wordy too. 2. In the last paragraph, quote "metadata" to be clear that we're referring to the table named "metadata" and not "table metadata". Otherwise, looks good. Very thorough.
          Hide
          Sean Busbey added a comment -

          Clarify the "if your instance previously upgraded from 1.4 to 1.5" statement to only apply to recent upgrades (e.g. the 1.4->1.6 path). That may be confusing for people who have been on 1.5 for some time. Maybe something like "For those attempting to upgrade to 1.6 from 1.4 through 1.5, ..."? That seems a bit wordy too.

          If they upgraded from 1.4 to 1.5 with an offline table that has walog entries, that all happened to be the only things on tablet server that was off (or that ran into problems attempting to do the copy, it won't matter how long ago it was. Presuming they ever bring the table back online.

          I could maybe explain that the 1.4 to 1.5 upgrade process asynchronously handled migrating 1.4 write ahead logs and that 1.6 can no longer make retries?

          In the last paragraph, quote "metadata" to be clear that we're referring to the table named "metadata" and not "table metadata".

          I did mean "table metadata". We write into multiple tables as a part of the upgrade, not just into the reserved metadata table. (the new reserved root, for example)

          Show
          Sean Busbey added a comment - Clarify the "if your instance previously upgraded from 1.4 to 1.5" statement to only apply to recent upgrades (e.g. the 1.4->1.6 path). That may be confusing for people who have been on 1.5 for some time. Maybe something like "For those attempting to upgrade to 1.6 from 1.4 through 1.5, ..."? That seems a bit wordy too. If they upgraded from 1.4 to 1.5 with an offline table that has walog entries, that all happened to be the only things on tablet server that was off (or that ran into problems attempting to do the copy, it won't matter how long ago it was. Presuming they ever bring the table back online. I could maybe explain that the 1.4 to 1.5 upgrade process asynchronously handled migrating 1.4 write ahead logs and that 1.6 can no longer make retries? In the last paragraph, quote "metadata" to be clear that we're referring to the table named "metadata" and not "table metadata". I did mean "table metadata". We write into multiple tables as a part of the upgrade, not just into the reserved metadata table. (the new reserved root, for example)
          Hide
          Sean Busbey added a comment -

          updated patch based on feedback from Bill Havanki

          Show
          Sean Busbey added a comment - updated patch based on feedback from Bill Havanki
          Hide
          Josh Elser added a comment -

          I could maybe explain that the 1.4 to 1.5 upgrade process asynchronously handled migrating 1.4 write ahead logs and that 1.6 can no longer make retries?

          I think that would be a good idea. Perhaps a disclaimer that those successfully running 1.5 upgraded from 1.4 don't need to perform the verification of no local WALs, too.

          I did mean "table metadata". We write into multiple tables as a part of the upgrade, not just into the reserved metadata table. (the new reserved root, for example)

          Oh I see. What about ".. in both ZooKeeper and the Accumulo system tables." instead? That would match nomenclature better I believe.

          Show
          Josh Elser added a comment - I could maybe explain that the 1.4 to 1.5 upgrade process asynchronously handled migrating 1.4 write ahead logs and that 1.6 can no longer make retries? I think that would be a good idea. Perhaps a disclaimer that those successfully running 1.5 upgraded from 1.4 don't need to perform the verification of no local WALs, too. I did mean "table metadata". We write into multiple tables as a part of the upgrade, not just into the reserved metadata table. (the new reserved root, for example) Oh I see. What about ".. in both ZooKeeper and the Accumulo system tables." instead? That would match nomenclature better I believe.
          Hide
          ASF subversion and git services added a comment -

          Commit f5a94f041ebe6b2b14624e4a6d25625bda6b5df9 in accumulo's branch refs/heads/1.6.0-SNAPSHOT from Sean Busbey
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=f5a94f0 ]

          ACCUMULO-2148 change upgrade docs to cover 1.5 -> 1.6

          Show
          ASF subversion and git services added a comment - Commit f5a94f041ebe6b2b14624e4a6d25625bda6b5df9 in accumulo's branch refs/heads/1.6.0-SNAPSHOT from Sean Busbey [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=f5a94f0 ] ACCUMULO-2148 change upgrade docs to cover 1.5 -> 1.6
          Hide
          ASF subversion and git services added a comment -

          Commit f5a94f041ebe6b2b14624e4a6d25625bda6b5df9 in accumulo's branch refs/heads/master from Sean Busbey
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=f5a94f0 ]

          ACCUMULO-2148 change upgrade docs to cover 1.5 -> 1.6

          Show
          ASF subversion and git services added a comment - Commit f5a94f041ebe6b2b14624e4a6d25625bda6b5df9 in accumulo's branch refs/heads/master from Sean Busbey [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=f5a94f0 ] ACCUMULO-2148 change upgrade docs to cover 1.5 -> 1.6
          Hide
          Sean Busbey added a comment -

          Verified upgrade works from 1.5.0, 1.5.1, 1.5.2-SNAPSHOT.

          (1.5.2-SNAPSHOT test included starting with data written in 1.4.x in addition to data written in 1.5.2-SNAP).

          Show
          Sean Busbey added a comment - Verified upgrade works from 1.5.0, 1.5.1, 1.5.2-SNAPSHOT. (1.5.2-SNAPSHOT test included starting with data written in 1.4.x in addition to data written in 1.5.2-SNAP).

            People

            • Assignee:
              Sean Busbey
              Reporter:
              Keith Turner
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development