Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: None
    • Labels:

      Description

      This is a proposed feature for 1.6.

      There's two ways we could do this:

      1. Continue to support both hadoop1 and hadoop2, but switch the profiles, so hadoop2 is the default, and this is what we use to build binary artifacts for release voting.
      2. Make hadoop2 required and get rid of the dependency profiles.

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          165d 7h 22m 1 Josh Elser 28/Oct/13 04:55
          Josh Elser made changes -
          Link This issue relates to ACCUMULO-1820 [ ACCUMULO-1820 ]
          Hide
          Josh Elser added a comment -

          Ah, that explains it. I didn't have the code up in front of me. I was just doing everything from the commandline+vim.

          Show
          Josh Elser added a comment - Ah, that explains it. I didn't have the code up in front of me. I was just doing everything from the commandline+vim.
          Hide
          Christopher Tubbs added a comment -

          The default compiler warnings, especially the use of deprecated methods are now abundant. However, I'm not sure we can switch without losing 1.x compatibility, so you might just need to suppress them. If they are suppressed, please suppress them on the variable, not the method... we don't want to suppress more than necessary, and end up suppressing things we do want to switch away from.

          Show
          Christopher Tubbs added a comment - The default compiler warnings, especially the use of deprecated methods are now abundant. However, I'm not sure we can switch without losing 1.x compatibility, so you might just need to suppress them. If they are suppressed, please suppress them on the variable, not the method... we don't want to suppress more than necessary, and end up suppressing things we do want to switch away from.
          Hide
          Josh Elser added a comment -

          Can you point me to said warnings? I don't know what you're talking about.

          Show
          Josh Elser added a comment - Can you point me to said warnings? I don't know what you're talking about.
          Hide
          John Vines added a comment -

          Going to do anything about all the warnings that introduced?

          Show
          John Vines added a comment - Going to do anything about all the warnings that introduced?
          Hide
          Josh Elser added a comment - - edited

          Did you update the jenkins jobs?

          Nope, I didn't. Good call.

          Edit: Done

          Show
          Josh Elser added a comment - - edited Did you update the jenkins jobs? Nope, I didn't. Good call. Edit: Done
          Hide
          John Vines added a comment -

          Did you update the jenkins jobs?

          Show
          John Vines added a comment - Did you update the jenkins jobs?
          Josh Elser made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Josh Elser added a comment -

          Updated the maven profiles, the example confs and the README.

          Of note, I changed the profile names (activation values) from 1.0 and 2.0 to 1.2 and 2.2 respectively to match the actual dependency (e.g. we're not depending on 2.0.5-alpha but 2.2.0)

          Show
          Josh Elser added a comment - Updated the maven profiles, the example confs and the README. Of note, I changed the profile names (activation values) from 1.0 and 2.0 to 1.2 and 2.2 respectively to match the actual dependency (e.g. we're not depending on 2.0.5-alpha but 2.2.0)
          Hide
          ASF subversion and git services added a comment -

          Commit cdc1dc9f08beccac7817365f2c08f9f4ceda6923 in branch refs/heads/master from Josh Elser
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=cdc1dc9 ]

          ACCUMULO-1419 Update README, confs and poms to switch default hadoop
          dependency to 2.2.0.

          Document the change in default Apache Hadoop dependency. Update
          configurations to account for new HADOOP_CONF_DIR and hadoop-2 classpath
          entries. Rename profiles from hadoop-1.0 to hadoop-1.2 and hadoop-2.0 to
          hadoop-2.2, along with updated dependencies for the hadoop-2.2 profile.

          Show
          ASF subversion and git services added a comment - Commit cdc1dc9f08beccac7817365f2c08f9f4ceda6923 in branch refs/heads/master from Josh Elser [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=cdc1dc9 ] ACCUMULO-1419 Update README, confs and poms to switch default hadoop dependency to 2.2.0. Document the change in default Apache Hadoop dependency. Update configurations to account for new HADOOP_CONF_DIR and hadoop-2 classpath entries. Rename profiles from hadoop-1.0 to hadoop-1.2 and hadoop-2.0 to hadoop-2.2, along with updated dependencies for the hadoop-2.2 profile.
          Josh Elser made changes -
          Assignee Josh Elser [ elserj ]
          Hide
          Josh Elser added a comment -

          I'm going to open up a vote for this. I don't think that we'll have adequate discussion from all parts of the community by just having it on the issue.

          Show
          Josh Elser added a comment - I'm going to open up a vote for this. I don't think that we'll have adequate discussion from all parts of the community by just having it on the issue.
          Hide
          Christopher Tubbs added a comment -

          +1 for defaulting to 2.2.0.
          This doesn't mean we drop support for versions 1.x and later, though... so I don't share John Vines' reservations.

          Show
          Christopher Tubbs added a comment - +1 for defaulting to 2.2.0. This doesn't mean we drop support for versions 1.x and later, though... so I don't share John Vines ' reservations.
          Hide
          Josh Elser added a comment -

          I'm concerned about making that jump after what happened with 0.21

          So, worst case, if something like that would happen again (which I think is unlikely given the amount of effort that has gone in 2.0 and 2.1 releases), we cut a 1.6.x that changes the default dependency back again. No?

          I'm of the mindset that if the community is confidently advertising a newer version as stable, we should also try to follow such guidance unless we have real problems in doing so.

          Show
          Josh Elser added a comment - I'm concerned about making that jump after what happened with 0.21 So, worst case, if something like that would happen again (which I think is unlikely given the amount of effort that has gone in 2.0 and 2.1 releases), we cut a 1.6.x that changes the default dependency back again. No? I'm of the mindset that if the community is confidently advertising a newer version as stable, we should also try to follow such guidance unless we have real problems in doing so.
          Hide
          John Vines added a comment -

          I'm concerned about making that jump after what happened with 0.21

          Show
          John Vines added a comment - I'm concerned about making that jump after what happened with 0.21
          Hide
          Josh Elser added a comment - - edited

          Now that Hadoop 2.2.0 is out, I'd like to discuss making it the default version for 1.6.0.

          +1 for defaulting to hadoop-2.2.0 as the default for 1.6.0. The release of 2.2.0 as GA is a sign that the Hadoop community has finalized APIs and thoroughly tested the components; they call it "stable". As such, I feel like we should follow suit in specifying our dependencies.

          Show
          Josh Elser added a comment - - edited Now that Hadoop 2.2.0 is out, I'd like to discuss making it the default version for 1.6.0. +1 for defaulting to hadoop-2.2.0 as the default for 1.6.0. The release of 2.2.0 as GA is a sign that the Hadoop community has finalized APIs and thoroughly tested the components; they call it "stable". As such, I feel like we should follow suit in specifying our dependencies.
          Hide
          Billie Rinaldi added a comment -

          Now that Hadoop 2.2.0 is out, I'd like to discuss making it the default version for 1.6.0.

          Show
          Billie Rinaldi added a comment - Now that Hadoop 2.2.0 is out, I'd like to discuss making it the default version for 1.6.0.
          Hide
          Christopher Tubbs added a comment -

          FYI, Hadoop 2.0.5 is currently in Fedora rawhide, getting ready for f20 release. Whatever we do with this ticket, let's keep that in mind, as I don't want to cut off a large potential user base by implementing changes that cause Accumulo to work only with 2.1.x and newer and not 2.0.5 (at least until there's a Fedora update from 2.0.5 to something newer).

          Show
          Christopher Tubbs added a comment - FYI, Hadoop 2.0.5 is currently in Fedora rawhide, getting ready for f20 release. Whatever we do with this ticket, let's keep that in mind, as I don't want to cut off a large potential user base by implementing changes that cause Accumulo to work only with 2.1.x and newer and not 2.0.5 (at least until there's a Fedora update from 2.0.5 to something newer).
          Hide
          Billie Rinaldi added a comment -

          I've installed Accumulo on it and performed simple operations successfully. Haven't run a test suite, though.

          Show
          Billie Rinaldi added a comment - I've installed Accumulo on it and performed simple operations successfully. Haven't run a test suite, though.
          Hide
          Mike Drob added a comment -

          Hadoop has a 2.1.0-beta release, has anybody on our side done due diligence w.r.t API and code stability?

          Show
          Mike Drob added a comment - Hadoop has a 2.1.0-beta release, has anybody on our side done due diligence w.r.t API and code stability?
          Christopher Tubbs made changes -
          Link This issue is related to ACCUMULO-1643 [ ACCUMULO-1643 ]
          Hide
          Christopher Tubbs added a comment - - edited

          We can leave the ticket open and see how things develop.

          I probably should have mentioned in the original ticket description that this was the intention.
          I should have also mentioned that, although I created this ticket, I'm with Josh on possibly making it conditional on a stable release of hadoop2 (depending on how things go), before I sign off on it.

          An additional point I'd like to clarify: if implemented, this would simply be the default hadoop version in the dependencies in the POM... the goal, of course, would be to maintain both source and binary compatibility with both, as much as possible, for the foreseeable future. And, since hadoop dependencies are marked as "provided", anyway, this ticket doesn't really propose much of a change at all.

          Show
          Christopher Tubbs added a comment - - edited We can leave the ticket open and see how things develop. I probably should have mentioned in the original ticket description that this was the intention. I should have also mentioned that, although I created this ticket, I'm with Josh on possibly making it conditional on a stable release of hadoop2 (depending on how things go), before I sign off on it. An additional point I'd like to clarify: if implemented, this would simply be the default hadoop version in the dependencies in the POM... the goal, of course, would be to maintain both source and binary compatibility with both, as much as possible, for the foreseeable future. And, since hadoop dependencies are marked as "provided", anyway, this ticket doesn't really propose much of a change at all.
          Hide
          Josh Elser added a comment -

          Sounds good to me.

          Show
          Josh Elser added a comment - Sounds good to me.
          Hide
          Billie Rinaldi added a comment -

          That's fine, I'm not saying we need to make the change right now. We can leave the ticket open and see how things develop.

          Show
          Billie Rinaldi added a comment - That's fine, I'm not saying we need to make the change right now. We can leave the ticket open and see how things develop.
          Hide
          Josh Elser added a comment -

          I understand that. My point was that I feel that we shouldn't change our default until there is a 2.0 release which is classified by the community as more than "alpha". Assuming beta releases happen, that's fine. Just voicing my concern that until such a hadoop release happen, I don't think we should change our default.

          As for the data loss point, there is also the possibility that hadoop2 will cause data loss that hadoop1 did not, but that's just speculation. It's always possible that hadoop2 introduces new issues too.

          It's perfectly fine to hope for making hadoop2 the default in 1.6.0; I just don't want us to jump the gun.

          Show
          Josh Elser added a comment - I understand that. My point was that I feel that we shouldn't change our default until there is a 2.0 release which is classified by the community as more than "alpha". Assuming beta releases happen, that's fine. Just voicing my concern that until such a hadoop release happen, I don't think we should change our default. As for the data loss point, there is also the possibility that hadoop2 will cause data loss that hadoop1 did not, but that's just speculation. It's always possible that hadoop2 introduces new issues too. It's perfectly fine to hope for making hadoop2 the default in 1.6.0; I just don't want us to jump the gun.
          Hide
          Billie Rinaldi added a comment -

          This suggestion is for 1.6.0. Assuming that is six months away, there should be at least a couple of beta releases of hadoop 2 out by then (if not better), with a stabilized API. Also, keep in mind that we have found conditions under which the stable version loses data and hadoop 2 does not.

          Show
          Billie Rinaldi added a comment - This suggestion is for 1.6.0. Assuming that is six months away, there should be at least a couple of beta releases of hadoop 2 out by then (if not better), with a stabilized API. Also, keep in mind that we have found conditions under which the stable version loses data and hadoop 2 does not.
          Hide
          Josh Elser added a comment -

          Personally, I'm a little leery to change our default to Hadoop's alpha release. I would assume we'd primarily want to support what Hadoop says is stable (maybe, beta).

          Show
          Josh Elser added a comment - Personally, I'm a little leery to change our default to Hadoop's alpha release . I would assume we'd primarily want to support what Hadoop says is stable (maybe, beta).
          Hide
          Corey J. Nolet added a comment -

          Sorry guys, I should have paid more attention to the "proposal" tag on this ticket. I'm trying to expand my experience with different parts of the codebase.

          Show
          Corey J. Nolet added a comment - Sorry guys, I should have paid more attention to the "proposal" tag on this ticket. I'm trying to expand my experience with different parts of the codebase.
          Corey J. Nolet made changes -
          Assignee Corey J. Nolet [ sonixbp ]
          Corey J. Nolet made changes -
          Comment [ Agreed- I figured a patch would get some discussion started :-)



          ]
          Hide
          Christopher Tubbs added a comment -

          Agreed that a vote would be needed before any change is applied here. That vote shouldn't happen until after 1.5 release, though, because anything sooner would be premature (since there may yet be changes needed to pass a release candidate vote for 1.5 on this subject).

          Show
          Christopher Tubbs added a comment - Agreed that a vote would be needed before any change is applied here. That vote shouldn't happen until after 1.5 release, though, because anything sooner would be premature (since there may yet be changes needed to pass a release candidate vote for 1.5 on this subject).
          Hide
          Corey J. Nolet added a comment - - edited

          Agreed- I figured a patch would get some discussion started Nevermind my first comment- I won't commit unless an agreement is made to go with #1

          Show
          Corey J. Nolet added a comment - - edited Agreed- I figured a patch would get some discussion started Nevermind my first comment- I won't commit unless an agreement is made to go with #1
          Hide
          John Vines added a comment -

          This is something that will need a lot of discussion before we go through,
          we should probably ultimately vote on this before any code changes are
          made.

          Sent from my phone, please pardon the typos and brevity.

          Show
          John Vines added a comment - This is something that will need a lot of discussion before we go through, we should probably ultimately vote on this before any code changes are made. Sent from my phone, please pardon the typos and brevity.
          Corey J. Nolet made changes -
          Attachment ACCUMULO-1419.1.patch [ 12583417 ]
          Hide
          Corey J. Nolet added a comment -

          I've got a patch for option #1. I'll post it here and if I nobody objects, I'll go ahead and commit it.

          Show
          Corey J. Nolet added a comment - I've got a patch for option #1. I'll post it here and if I nobody objects, I'll go ahead and commit it.
          Corey J. Nolet made changes -
          Assignee Corey J. Nolet [ sonixbp ]
          Hide
          Corey J. Nolet added a comment - - edited

          Nevermind about the 1.5 comment, I just noticed this is proposed for 1.6. I'm still for keeping the 1.0 support and documenting any cautions.

          Show
          Corey J. Nolet added a comment - - edited Nevermind about the 1.5 comment, I just noticed this is proposed for 1.6. I'm still for keeping the 1.0 support and documenting any cautions.
          Hide
          Corey J. Nolet added a comment -

          +1 for #1. #2 seems to imply no more support for hadoop1. If this is the case, 1.5 release would be leaving our 1.0 users behind. For those users, ACCUMULO-1364 should be well-documented and cautioned.

          Show
          Corey J. Nolet added a comment - +1 for #1. #2 seems to imply no more support for hadoop1. If this is the case, 1.5 release would be leaving our 1.0 users behind. For those users, ACCUMULO-1364 should be well-documented and cautioned.
          Christopher Tubbs made changes -
          Link This issue is depended upon by ACCUMULO-1364 [ ACCUMULO-1364 ]
          Christopher Tubbs made changes -
          Link This issue supercedes ACCUMULO-1364 [ ACCUMULO-1364 ]
          Christopher Tubbs made changes -
          Link This issue supercedes ACCUMULO-1364 [ ACCUMULO-1364 ]
          Christopher Tubbs made changes -
          Field Original Value New Value
          Link This issue supercedes ACCUMULO-1402 [ ACCUMULO-1402 ]
          Christopher Tubbs created issue -

            People

            • Assignee:
              Josh Elser
              Reporter:
              Christopher Tubbs
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development