Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-beta1
    • Fix Version/s: 3.0.0-beta1
    • Component/s: build, site
    • Labels:
      None

      Description

      Running mvn install site site:stage -DskipTests -Pdist,src -Preleasedocs,docs results in a stack trace when run on a fresh .m2 directory. It appears to be coming from the jdiff doclets in the annotations code.

      1. HADOOP-14835.001.patch
        7 kB
        Andrew Wang
      2. HADOOP-14835.002.patch
        8 kB
        Andrew Wang
      3. HADOOP-14835.003.patch
        8 kB
        Sean Mackrory

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          Stack trace:

          
          

          [javadoc] Constructing Javadoc information...
          [javadoc] IncludePublicAnnotationsJDiffDoclet
          [javadoc] JDiff: doclet started ...
          [javadoc] org.xml.sax.SAXException: SAX2 driver class org.apache.xerces.parsers.SAXParser not found
          [javadoc] JDiff: reading the old API in from file '/Users/aw/shared-vmware/hadoop/hadoop-common-project/hadoop-common/dev-support/jdiff/Apache_Hadoop_Common_2.8.0.xml'...SAXException: org.xml.sax.SAXException: SAX2 driver class org.apache.xerces.parsers.SAXParser not found
          [javadoc] java.lang.ClassNotFoundException: org.apache.xerces.parsers.SAXParser
          [javadoc] java.lang.ClassNotFoundException: org.apache.xerces.parsers.SAXParser
          [javadoc] at org.xml.sax.helpers.XMLReaderFactory.loadClass(XMLReaderFactory.java:230)
          [javadoc] at org.xml.sax.helpers.XMLReaderFactory.createXMLReader(XMLReaderFactory.java:221)
          [javadoc] at jdiff.XMLToAPI.readFile(XMLToAPI.java:51)
          [javadoc] at jdiff.JDiff.startGeneration(JDiff.java:83)
          [javadoc] at jdiff.JDiff.start(JDiff.java:29)
          [javadoc] at org.apache.hadoop.classification.tools.IncludePublicAnnotationsJDiffDoclet.start(IncludePublicAnnotationsJDiffDoclet.java:47)
          [javadoc] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          [javadoc] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          [javadoc] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          [javadoc] at java.lang.reflect.Method.invoke(Method.java:498)
          [javadoc] at com.sun.tools.javadoc.DocletInvoker.invoke(DocletInvoker.java:310)
          [javadoc] at com.sun.tools.javadoc.DocletInvoker.start(DocletInvoker.java:189)
          [javadoc] at com.sun.tools.javadoc.Start.parseAndExecute(Start.java:366)
          [javadoc] at com.sun.tools.javadoc.Start.begin(Start.java:219)
          [javadoc] at com.sun.tools.javadoc.Start.begin(Start.java:205)
          [javadoc] at com.sun.tools.javadoc.Main.execute(Main.java:64)
          [javadoc] at com.sun.tools.javadoc.Main.main(Main.java:54)
          [javadoc] Caused by: java.lang.ClassNotFoundException: org.apache.xerces.parsers.SAXParser
          [javadoc] at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
          [javadoc] at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
          [javadoc] at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
          [javadoc] at org.xml.sax.helpers.NewInstance.newInstance(NewInstance.java:82)
          [javadoc] at org.xml.sax.helpers.XMLReaderFactory.loadClass(XMLReaderFactory.java:228)
          [javadoc] ... 16 more

          
          
          Show
          aw Allen Wittenauer added a comment - Stack trace: [javadoc] Constructing Javadoc information... [javadoc] IncludePublicAnnotationsJDiffDoclet [javadoc] JDiff: doclet started ... [javadoc] org.xml.sax.SAXException: SAX2 driver class org.apache.xerces.parsers.SAXParser not found [javadoc] JDiff: reading the old API in from file '/Users/aw/shared-vmware/hadoop/hadoop-common-project/hadoop-common/dev-support/jdiff/Apache_Hadoop_Common_2.8.0.xml'...SAXException: org.xml.sax.SAXException: SAX2 driver class org.apache.xerces.parsers.SAXParser not found [javadoc] java.lang.ClassNotFoundException: org.apache.xerces.parsers.SAXParser [javadoc] java.lang.ClassNotFoundException: org.apache.xerces.parsers.SAXParser [javadoc] at org.xml.sax.helpers.XMLReaderFactory.loadClass(XMLReaderFactory.java:230) [javadoc] at org.xml.sax.helpers.XMLReaderFactory.createXMLReader(XMLReaderFactory.java:221) [javadoc] at jdiff.XMLToAPI.readFile(XMLToAPI.java:51) [javadoc] at jdiff.JDiff.startGeneration(JDiff.java:83) [javadoc] at jdiff.JDiff.start(JDiff.java:29) [javadoc] at org.apache.hadoop.classification.tools.IncludePublicAnnotationsJDiffDoclet.start(IncludePublicAnnotationsJDiffDoclet.java:47) [javadoc] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [javadoc] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [javadoc] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [javadoc] at java.lang.reflect.Method.invoke(Method.java:498) [javadoc] at com.sun.tools.javadoc.DocletInvoker.invoke(DocletInvoker.java:310) [javadoc] at com.sun.tools.javadoc.DocletInvoker.start(DocletInvoker.java:189) [javadoc] at com.sun.tools.javadoc.Start.parseAndExecute(Start.java:366) [javadoc] at com.sun.tools.javadoc.Start.begin(Start.java:219) [javadoc] at com.sun.tools.javadoc.Start.begin(Start.java:205) [javadoc] at com.sun.tools.javadoc.Main.execute(Main.java:64) [javadoc] at com.sun.tools.javadoc.Main.main(Main.java:54) [javadoc] Caused by: java.lang.ClassNotFoundException: org.apache.xerces.parsers.SAXParser [javadoc] at java.net.URLClassLoader.findClass(URLClassLoader.java:381) [javadoc] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) [javadoc] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) [javadoc] at org.xml.sax.helpers.NewInstance.newInstance(NewInstance.java:82) [javadoc] at org.xml.sax.helpers.XMLReaderFactory.loadClass(XMLReaderFactory.java:228) [javadoc] ... 16 more
          Hide
          andrew.wang Andrew Wang added a comment -

          This is possibly because we purged xerces in HDFS-12221.

          Show
          andrew.wang Andrew Wang added a comment - This is possibly because we purged xerces in HDFS-12221 .
          Hide
          aw Allen Wittenauer added a comment -

          That would make sense. I think I'm more surprised that it's not a fatal error though. I'm guessing that's a bug in the doclet?

          Show
          aw Allen Wittenauer added a comment - That would make sense. I think I'm more surprised that it's not a fatal error though. I'm guessing that's a bug in the doclet?
          Hide
          andrew.wang Andrew Wang added a comment -

          Here's a patch which pulls in jdiff's xerces when the docs profile is activated.

          My build gets farther but fails with MBs of javadoc error output, I didn't look deeply enough to know what's actually wrong. Appreciate anyone's help if they have any quick ideas.

          Show
          andrew.wang Andrew Wang added a comment - Here's a patch which pulls in jdiff's xerces when the docs profile is activated. My build gets farther but fails with MBs of javadoc error output, I didn't look deeply enough to know what's actually wrong. Appreciate anyone's help if they have any quick ideas.
          Hide
          aw Allen Wittenauer added a comment -

          Oh, you're likely bombing out due to YARN-6877 being committed despite it clearly failing the build. Revert 91cc070d67533ebb3325b982eba2135e0d175a82 and javadoc will work.

          Show
          aw Allen Wittenauer added a comment - Oh, you're likely bombing out due to YARN-6877 being committed despite it clearly failing the build. Revert 91cc070d67533ebb3325b982eba2135e0d175a82 and javadoc will work.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          0 mvndep 0m 43s Maven dependency ordering for branch
          +1 mvninstall 13m 38s trunk passed
          +1 compile 15m 21s trunk passed
          +1 mvnsite 5m 14s trunk passed
          -1 javadoc 0m 46s hadoop-yarn in trunk failed.
                Patch Compile Tests
          0 mvndep 0m 18s Maven dependency ordering for patch
          +1 mvninstall 5m 11s the patch passed
          +1 compile 11m 46s the patch passed
          +1 javac 11m 46s the patch passed
          +1 mvnsite 5m 28s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 7s The patch has no ill-formed XML file.
          -1 javadoc 0m 52s hadoop-yarn in the patch failed.
                Other Tests
          +1 unit 0m 9s hadoop-project in the patch passed.
          +1 unit 0m 10s hadoop-project-dist in the patch passed.
          -1 unit 73m 15s hadoop-yarn in the patch failed.
          -1 unit 24m 38s hadoop-mapreduce-client in the patch failed.
          +1 asflicense 0m 32s The patch does not generate ASF License warnings.
          162m 1s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            hadoop.mapreduce.v2.hs.webapp.TestHSWebApp



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HADOOP-14835
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885492/HADOOP-14835.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux f1bdf7dbc7ab 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d4035d4
          Default Java 1.8.0_144
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/testReport/
          modules C: hadoop-project hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests 0 mvndep 0m 43s Maven dependency ordering for branch +1 mvninstall 13m 38s trunk passed +1 compile 15m 21s trunk passed +1 mvnsite 5m 14s trunk passed -1 javadoc 0m 46s hadoop-yarn in trunk failed.       Patch Compile Tests 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 5m 11s the patch passed +1 compile 11m 46s the patch passed +1 javac 11m 46s the patch passed +1 mvnsite 5m 28s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 7s The patch has no ill-formed XML file. -1 javadoc 0m 52s hadoop-yarn in the patch failed.       Other Tests +1 unit 0m 9s hadoop-project in the patch passed. +1 unit 0m 10s hadoop-project-dist in the patch passed. -1 unit 73m 15s hadoop-yarn in the patch failed. -1 unit 24m 38s hadoop-mapreduce-client in the patch failed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 162m 1s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.mapreduce.v2.hs.webapp.TestHSWebApp Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HADOOP-14835 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885492/HADOOP-14835.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux f1bdf7dbc7ab 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d4035d4 Default Java 1.8.0_144 javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/testReport/ modules C: hadoop-project hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13173/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment - - edited

          Ripping out YARN-6877, javadoc appears to be working but I'm failing in hadoop-client-check-test-invariants with

          WARNING] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
          Duplicate classes found:
          
            Found in:
              org.apache.hadoop:hadoop-client-runtime:jar:3.1.0-SNAPSHOT:compile
              org.apache.hadoop:hadoop-client-minicluster:jar:3.1.0-SNAPSHOT:compile
            Duplicate classes:
              org/apache/hadoop/shaded/org/apache/xerces/impl/dv/dtd/NMTOKENDatatypeValidator.class
          ....
          
          Show
          aw Allen Wittenauer added a comment - - edited Ripping out YARN-6877 , javadoc appears to be working but I'm failing in hadoop-client-check-test-invariants with WARNING] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message: Duplicate classes found: Found in: org.apache.hadoop:hadoop-client-runtime:jar:3.1.0-SNAPSHOT:compile org.apache.hadoop:hadoop-client-minicluster:jar:3.1.0-SNAPSHOT:compile Duplicate classes: org/apache/hadoop/shaded/org/apache/xerces/impl/dv/dtd/NMTOKENDatatypeValidator.class ....
          Hide
          andrew.wang Andrew Wang added a comment -

          The issue is that I added the xerces dependency under the "docs" profile, which means it gets pulled into the hadoop artifacts whenever the docs profile is on. We don't want this. I don't know how to do this in Maven without splitting jdiff generation into a new module.

          Anyway, we already do the release build in two passes: artifacts, then site. The simplest fix is to not build the shaded artifacts when docs is enabled. Maven again makes it difficult to do this automatically since you can't chain profile activations; the quick fix is to pass "-DskipShade" when we do a docs/site build.

          This also requires a BUILDING.txt update to make it clear that the site needs to be built in a second pass, as ugly as that is.

          Alternatively, we cut our losses and get rid of JDiff and provide a JACC report instead.

          Show
          andrew.wang Andrew Wang added a comment - The issue is that I added the xerces dependency under the "docs" profile, which means it gets pulled into the hadoop artifacts whenever the docs profile is on. We don't want this. I don't know how to do this in Maven without splitting jdiff generation into a new module. Anyway, we already do the release build in two passes: artifacts, then site. The simplest fix is to not build the shaded artifacts when docs is enabled. Maven again makes it difficult to do this automatically since you can't chain profile activations; the quick fix is to pass "-DskipShade" when we do a docs/site build. This also requires a BUILDING.txt update to make it clear that the site needs to be built in a second pass, as ugly as that is. Alternatively, we cut our losses and get rid of JDiff and provide a JACC report instead.
          Hide
          aw Allen Wittenauer added a comment -

          Alternatively, we cut our losses and get rid of JDiff and provide a JACC report instead.

          Funny you mention that. When I first looked at this issue, I wondered when jdiff had last been updated. Then I started looking at alternatives. japicmp (http://siom79.github.io/japicmp/) looks like it might be a better replacement. The fact it ships with a maven plugin, has filters, etc, is a HUGE win. But I didn't spend a lot of time playing with it.

          Show
          aw Allen Wittenauer added a comment - Alternatively, we cut our losses and get rid of JDiff and provide a JACC report instead. Funny you mention that. When I first looked at this issue, I wondered when jdiff had last been updated. Then I started looking at alternatives. japicmp ( http://siom79.github.io/japicmp/ ) looks like it might be a better replacement. The fact it ships with a maven plugin, has filters, etc, is a HUGE win. But I didn't spend a lot of time playing with it.
          Hide
          andrew.wang Andrew Wang added a comment -

          I spent a little time playing with japicmp, seems nice. Filed a WIP patch over at HADOOP-14848.

          I don't know if I have time to get this done before beta1 though. Attaching a little rev that adds -DskipShade to create-release.

          Allen, are you okay with this as an intermediate fix to unblock the release? I think the remaining japicmp integration isn't hard, but untangling the YARN poms will be a pain.

          Show
          andrew.wang Andrew Wang added a comment - I spent a little time playing with japicmp, seems nice. Filed a WIP patch over at HADOOP-14848 . I don't know if I have time to get this done before beta1 though. Attaching a little rev that adds -DskipShade to create-release. Allen, are you okay with this as an intermediate fix to unblock the release? I think the remaining japicmp integration isn't hard, but untangling the YARN poms will be a pain.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          0 mvndep 0m 21s Maven dependency ordering for branch
          +1 mvninstall 15m 23s trunk passed
          +1 compile 16m 51s trunk passed
          +1 mvnsite 6m 21s trunk passed
          +1 javadoc 4m 5s trunk passed
                Patch Compile Tests
          0 mvndep 0m 20s Maven dependency ordering for patch
          -1 mvninstall 1m 23s hadoop-mapreduce-client in the patch failed.
          +1 compile 12m 49s the patch passed
          +1 javac 12m 49s the patch passed
          +1 mvnsite 6m 35s the patch passed
          +1 shellcheck 0m 2s There were no new shellcheck issues.
          +1 shelldocs 0m 12s There were no new shelldocs issues.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 6s The patch has no ill-formed XML file.
          +1 javadoc 3m 59s the patch passed
                Other Tests
          +1 unit 0m 13s hadoop-project in the patch passed.
          +1 unit 0m 12s hadoop-project-dist in the patch passed.
          -1 unit 74m 49s hadoop-yarn in the patch failed.
          -1 unit 24m 0s hadoop-mapreduce-client in the patch failed.
          +1 asflicense 0m 36s The patch does not generate ASF License warnings.
          173m 22s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            hadoop.mapreduce.v2.hs.webapp.TestHSWebApp



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HADOOP-14835
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885952/HADOOP-14835.002.patch
          Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 93ef802cf2d8 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b0b535d
          Default Java 1.8.0_144
          shellcheck v0.4.6
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/testReport/
          modules C: hadoop-project hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests 0 mvndep 0m 21s Maven dependency ordering for branch +1 mvninstall 15m 23s trunk passed +1 compile 16m 51s trunk passed +1 mvnsite 6m 21s trunk passed +1 javadoc 4m 5s trunk passed       Patch Compile Tests 0 mvndep 0m 20s Maven dependency ordering for patch -1 mvninstall 1m 23s hadoop-mapreduce-client in the patch failed. +1 compile 12m 49s the patch passed +1 javac 12m 49s the patch passed +1 mvnsite 6m 35s the patch passed +1 shellcheck 0m 2s There were no new shellcheck issues. +1 shelldocs 0m 12s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 6s The patch has no ill-formed XML file. +1 javadoc 3m 59s the patch passed       Other Tests +1 unit 0m 13s hadoop-project in the patch passed. +1 unit 0m 12s hadoop-project-dist in the patch passed. -1 unit 74m 49s hadoop-yarn in the patch failed. -1 unit 24m 0s hadoop-mapreduce-client in the patch failed. +1 asflicense 0m 36s The patch does not generate ASF License warnings. 173m 22s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.mapreduce.v2.hs.webapp.TestHSWebApp Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HADOOP-14835 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885952/HADOOP-14835.002.patch Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit xml uname Linux 93ef802cf2d8 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b0b535d Default Java 1.8.0_144 shellcheck v0.4.6 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/testReport/ modules C: hadoop-project hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13200/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          Allen Wittenauer, are you comfortable doing a review? Otherwise I can ask someone else.

          Show
          andrew.wang Andrew Wang added a comment - Allen Wittenauer , are you comfortable doing a review? Otherwise I can ask someone else.
          Hide
          mackrorysd Sean Mackrory added a comment -

          +1 - but attaching a v3 patch that also adds a note to BUILDING.txt as you suggested about the site having to be done in a second pass, which I think is a good idea.

          Show
          mackrorysd Sean Mackrory added a comment - +1 - but attaching a v3 patch that also adds a note to BUILDING.txt as you suggested about the site having to be done in a second pass, which I think is a good idea.
          Hide
          aw Allen Wittenauer added a comment -

          Sorry, I've been super behind on everything. I'm a little hesitant to disable shading though, especially if the goal is to push users towards it. I'd be more inclined to disable javadoc honestly.

          Show
          aw Allen Wittenauer added a comment - Sorry, I've been super behind on everything. I'm a little hesitant to disable shading though, especially if the goal is to push users towards it. I'd be more inclined to disable javadoc honestly.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 31s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          0 mvndep 1m 54s Maven dependency ordering for branch
          +1 mvninstall 17m 59s trunk passed
          +1 compile 17m 5s trunk passed
          +1 mvnsite 10m 11s trunk passed
          +1 javadoc 5m 58s trunk passed
                Patch Compile Tests
          0 mvndep 0m 20s Maven dependency ordering for patch
          -1 mvninstall 1m 49s hadoop-yarn in the patch failed.
          -1 mvninstall 0m 29s hadoop-mapreduce-client in the patch failed.
          +1 compile 13m 12s the patch passed
          +1 javac 13m 12s the patch passed
          +1 mvnsite 11m 5s the patch passed
          +1 shellcheck 0m 0s There were no new shellcheck issues.
          +1 shelldocs 0m 10s There were no new shelldocs issues.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 5s The patch has no ill-formed XML file.
          +1 javadoc 5m 8s the patch passed
                Other Tests
          -1 unit 184m 16s root in the patch failed.
          +1 asflicense 0m 42s The patch does not generate ASF License warnings.
          288m 3s



          Reason Tests
          Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManager
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.TestLeaseRecoveryStriped
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.TestReconstructStripedFile
            hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.web.TestHttpsFileSystem
            hadoop.hdfs.TestEncryptedTransfer
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
            hadoop.hdfs.TestReplaceDatanodeOnFailure
          Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HADOOP-14835
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886979/HADOOP-14835.003.patch
          Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 2f773b481c6d 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / bb34ae9
          Default Java 1.8.0_144
          shellcheck v0.4.6
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn.txt
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/testReport/
          modules C: hadoop-project hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 31s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests 0 mvndep 1m 54s Maven dependency ordering for branch +1 mvninstall 17m 59s trunk passed +1 compile 17m 5s trunk passed +1 mvnsite 10m 11s trunk passed +1 javadoc 5m 58s trunk passed       Patch Compile Tests 0 mvndep 0m 20s Maven dependency ordering for patch -1 mvninstall 1m 49s hadoop-yarn in the patch failed. -1 mvninstall 0m 29s hadoop-mapreduce-client in the patch failed. +1 compile 13m 12s the patch passed +1 javac 13m 12s the patch passed +1 mvnsite 11m 5s the patch passed +1 shellcheck 0m 0s There were no new shellcheck issues. +1 shelldocs 0m 10s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 5s The patch has no ill-formed XML file. +1 javadoc 5m 8s the patch passed       Other Tests -1 unit 184m 16s root in the patch failed. +1 asflicense 0m 42s The patch does not generate ASF License warnings. 288m 3s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManager   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestReconstructStripedFile   hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.web.TestHttpsFileSystem   hadoop.hdfs.TestEncryptedTransfer   hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   hadoop.hdfs.TestReplaceDatanodeOnFailure Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HADOOP-14835 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886979/HADOOP-14835.003.patch Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit xml uname Linux 2f773b481c6d 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bb34ae9 Default Java 1.8.0_144 shellcheck v0.4.6 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn.txt mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/testReport/ modules C: hadoop-project hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13280/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          This only turns off shading for the site tarball (which is built after the binary tarball), so I think this is pretty safe. I'd like to commit this tomorrow to unblock the release. The japicmp JIRA doesn't seem that hard, I can play with it more before GA at which point we can undo this one.

          The mvn install seems to have failed due to a timestamped snapshot dependency convergence error, which I've never been able to figure out. I'll retrigger the build to be sure.

          Show
          andrew.wang Andrew Wang added a comment - This only turns off shading for the site tarball (which is built after the binary tarball), so I think this is pretty safe. I'd like to commit this tomorrow to unblock the release. The japicmp JIRA doesn't seem that hard, I can play with it more before GA at which point we can undo this one. The mvn install seems to have failed due to a timestamped snapshot dependency convergence error, which I've never been able to figure out. I'll retrigger the build to be sure.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          0 mvndep 0m 20s Maven dependency ordering for branch
          +1 mvninstall 14m 23s trunk passed
          +1 compile 14m 14s trunk passed
          +1 mvnsite 9m 18s trunk passed
          +1 javadoc 4m 45s trunk passed
                Patch Compile Tests
          0 mvndep 0m 19s Maven dependency ordering for patch
          +1 mvninstall 18m 3s the patch passed
          +1 compile 10m 59s the patch passed
          +1 javac 10m 59s the patch passed
          +1 mvnsite 9m 11s the patch passed
          +1 shellcheck 0m 1s There were no new shellcheck issues.
          +1 shelldocs 0m 9s There were no new shelldocs issues.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 4s The patch has no ill-formed XML file.
          +1 javadoc 4m 43s the patch passed
                Other Tests
          -1 unit 13m 41s root in the patch failed.
          +1 asflicense 0m 33s The patch does not generate ASF License warnings.
          102m 7s



          Reason Tests
          Failed junit tests hadoop.security.TestKDiag



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HADOOP-14835
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886979/HADOOP-14835.003.patch
          Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 6e4a12188e2f 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e12f3e8
          Default Java 1.8.0_144
          shellcheck v0.4.6
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13288/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13288/testReport/
          modules C: hadoop-project hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13288/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 14m 23s trunk passed +1 compile 14m 14s trunk passed +1 mvnsite 9m 18s trunk passed +1 javadoc 4m 45s trunk passed       Patch Compile Tests 0 mvndep 0m 19s Maven dependency ordering for patch +1 mvninstall 18m 3s the patch passed +1 compile 10m 59s the patch passed +1 javac 10m 59s the patch passed +1 mvnsite 9m 11s the patch passed +1 shellcheck 0m 1s There were no new shellcheck issues. +1 shelldocs 0m 9s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 4s The patch has no ill-formed XML file. +1 javadoc 4m 43s the patch passed       Other Tests -1 unit 13m 41s root in the patch failed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 102m 7s Reason Tests Failed junit tests hadoop.security.TestKDiag Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HADOOP-14835 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886979/HADOOP-14835.003.patch Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit xml uname Linux 6e4a12188e2f 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e12f3e8 Default Java 1.8.0_144 shellcheck v0.4.6 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13288/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13288/testReport/ modules C: hadoop-project hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13288/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          Second try at precommit worked, will commit the v3 patch.

          Show
          andrew.wang Andrew Wang added a comment - Second try at precommit worked, will commit the v3 patch.
          Hide
          andrew.wang Andrew Wang added a comment -

          Pushed to trunk and branch-3.0. Let's handle the jdiff->japicmp switch in the linked JIRA, and undo this post beta1.

          Show
          andrew.wang Andrew Wang added a comment - Pushed to trunk and branch-3.0. Let's handle the jdiff->japicmp switch in the linked JIRA, and undo this post beta1.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #12904 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12904/)
          HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew (wang: rev 3cf3540f19b5fd1a174690db9f1b7be2977d96ba)

          • (edit) dev-support/bin/create-release
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
          • (edit) hadoop-project/pom.xml
          • (edit) hadoop-project-dist/pom.xml
          • (edit) BUILDING.txt
          • (edit) hadoop-yarn-project/hadoop-yarn/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #12904 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12904/ ) HADOOP-14835 . mvn site build throws SAX errors. Contributed by Andrew (wang: rev 3cf3540f19b5fd1a174690db9f1b7be2977d96ba) (edit) dev-support/bin/create-release (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml (edit) hadoop-project/pom.xml (edit) hadoop-project-dist/pom.xml (edit) BUILDING.txt (edit) hadoop-yarn-project/hadoop-yarn/pom.xml

            People

            • Assignee:
              andrew.wang Andrew Wang
              Reporter:
              aw Allen Wittenauer
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development