Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.24.0
    • Fix Version/s: 0.23.0, 0.24.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Avro 1.5.3 depends on Snappy-Java 1.5.3 which enables the use of its SO file from the java.library.path

      1. HADOOP-7595v1.patch
        2 kB
        Alejandro Abdelnur

        Activity

        Hide
        Aaron T. Myers added a comment -

        Hey Arun, I'm not so sure the failure in MAPREDUCE-2926 is related to this JIRA. See HADOOP-7606 and this build, which failed with the same error before this JIRA was committed: https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Common-trunk/150/

        Show
        Aaron T. Myers added a comment - Hey Arun, I'm not so sure the failure in MAPREDUCE-2926 is related to this JIRA. See HADOOP-7606 and this build, which failed with the same error before this JIRA was committed: https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Common-trunk/150/
        Hide
        Arun C Murthy added a comment -

        This caused MR issues: MAPREDUCE-2926. In future, can we please be more careful with version bumps? Thanks.

        Show
        Arun C Murthy added a comment - This caused MR issues: MAPREDUCE-2926 . In future, can we please be more careful with version bumps? Thanks.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #804 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/804/)
        HADOOP-7595. Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm)

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #804 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/804/ ) HADOOP-7595 . Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #780 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/780/)
        HADOOP-7595. Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm)

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #780 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/780/ ) HADOOP-7595 . Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #829 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/829/)
        HADOOP-7595. Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm)

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #829 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/829/ ) HADOOP-7595 . Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Aaron T. Myers added a comment -

        I've just committed this to trunk and branch 0.23. Thanks a lot, Alejandro!

        Show
        Aaron T. Myers added a comment - I've just committed this to trunk and branch 0.23. Thanks a lot, Alejandro!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #896 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/896/)
        HADOOP-7595. Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm)

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #896 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/896/ ) HADOOP-7595 . Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #819 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/819/)
        HADOOP-7595. Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm)

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml
        • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
        • /hadoop/common/trunk/hadoop-project/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #819 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/819/ ) HADOOP-7595 . Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1164174 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml
        Hide
        Owen O'Malley added a comment -

        Sigh You can go ahead and upgrade Avro, I just find that the stated justification of upgrading Avro specifically to acquire a dependence on Snappy to be backwards. I'd rather just add the dependence on Snappy.

        Show
        Owen O'Malley added a comment - Sigh You can go ahead and upgrade Avro, I just find that the stated justification of upgrading Avro specifically to acquire a dependence on Snappy to be backwards. I'd rather just add the dependence on Snappy.
        Hide
        Aaron T. Myers added a comment -

        +1, the patch looks good to me.

        Owen, are you satisfied with Alejandro's answer? If so, I'll go ahead and commit this.

        Show
        Aaron T. Myers added a comment - +1, the patch looks good to me. Owen, are you satisfied with Alejandro's answer? If so, I'll go ahead and commit this.
        Hide
        Alejandro Abdelnur added a comment -

        Hadoop is not using Snappy-Java directly, as you say it is a transitive dependency via Avro. It is common practice that you should declare your direct dependencies. Declaring transitive dependencies should avoided if possible.

        Show
        Alejandro Abdelnur added a comment - Hadoop is not using Snappy-Java directly, as you say it is a transitive dependency via Avro. It is common practice that you should declare your direct dependencies. Declaring transitive dependencies should avoided if possible.
        Hide
        Owen O'Malley added a comment -

        I'd rather that you just add the dependency on Snappy directly into common. Depending on the transitive closure of Avro's dependencies seems strange.

        Show
        Owen O'Malley added a comment - I'd rather that you just add the dependency on Snappy directly into common. Depending on the transitive closure of Avro's dependencies seems strange.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12492390/HADOOP-7595v1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/105//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12492390/HADOOP-7595v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/105//console This message is automatically generated.

          People

          • Assignee:
            Alejandro Abdelnur
            Reporter:
            Alejandro Abdelnur
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development