Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14414

Calling maven-site-plugin directly for docs profile is unnecessary

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-beta1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      maven-site-plugin is no longer called directly at package phase.

      Description

      For a few modules:

      • hadoop-auth
      • hadoop-kms
      • hadoop-hdfs-httpfs
      • hadoop-sls

      we call mave-site-plugin directly when docs profile is active.
      In main pom we use excludeDefaults in reporting section and allow only javadoc and dependency-plugin for the report. Since javadoc plugin is set to inherited false it won't be called on individual child modules. So actually maven-dependency-plugin:analyze-report is the only additional goal which will run.
      I debugged the process with mvn clean package -DskipTests -Dmaven.javadoc.skip=true -DskipShade -Pdocs -X command and in all the 4 affected modules I found the following configuration for site plugin:

      <reportPlugins>
          <reportPlugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <version>2.10</version>
            <reportSets>
              <reportSet>
                <id>default</id>
                <reports>
                  <report>analyze-report</report>
                </reports>
              </reportSet>
            </reportSets>
          </reportPlugin>
        </reportPlugins>

      At this point I do not see the purpose of calling mave-site-plugin for docs profile. It does not contain useful information. Or if it does why don't we call for other modules? It's inconsistent.
      Considering to remove.

      1. HADOOP-14414.01.patch
        4 kB
        Andras Bokor
      2. HADOOP-14414.02.patch
        4 kB
        Andras Bokor
      3. missingFiles
        9 kB
        Andras Bokor

        Issue Links

          Activity

          Hide
          boky01 Andras Bokor added a comment - - edited

          Attaching patch 01.
          It does not affect the site generation:

          mvn clean site:site -Preleasedocs; mvn site:stage -DstagingDirectory=/tmp/hadoop-site > /dev/null; ls /tmp/hadoop-site/hadoop-project/hadoop-hdfs-httpfs/
          ServerSetup.html	UsingHttpTools.html	css	dependency-analysis.html	images	index.html	project-reports.html
          Show
          boky01 Andras Bokor added a comment - - edited Attaching patch 01. It does not affect the site generation: mvn clean site:site -Preleasedocs; mvn site:stage -DstagingDirectory=/tmp/hadoop-site > /dev/ null ; ls /tmp/hadoop-site/hadoop-project/hadoop-hdfs-httpfs/ ServerSetup.html UsingHttpTools.html css dependency-analysis.html images index.html project-reports.html
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 16s Maven dependency ordering for branch
          +1 mvninstall 15m 4s trunk passed
          +1 compile 17m 2s trunk passed
          +1 mvnsite 12m 1s trunk passed
          +1 mvneclipse 2m 24s trunk passed
          +1 javadoc 6m 3s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 18m 56s the patch passed
          +1 compile 17m 42s the patch passed
          +1 javac 17m 42s the patch passed
          +1 mvnsite 12m 52s the patch passed
          +1 mvneclipse 1m 56s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 8s The patch has no ill-formed XML file.
          +1 javadoc 6m 27s the patch passed
          -1 unit 122m 37s root in the patch failed.
          +1 asflicense 0m 39s The patch does not generate ASF License warnings.
          235m 51s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.tools.TestDFSZKFailoverController
            hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14414
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867635/HADOOP-14414.01.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 80970a1db56e 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 09f28da
          Default Java 1.8.0_121
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12296/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12296/testReport/
          modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-tools/hadoop-sls . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12296/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 15m 4s trunk passed +1 compile 17m 2s trunk passed +1 mvnsite 12m 1s trunk passed +1 mvneclipse 2m 24s trunk passed +1 javadoc 6m 3s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 18m 56s the patch passed +1 compile 17m 42s the patch passed +1 javac 17m 42s the patch passed +1 mvnsite 12m 52s the patch passed +1 mvneclipse 1m 56s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 8s The patch has no ill-formed XML file. +1 javadoc 6m 27s the patch passed -1 unit 122m 37s root in the patch failed. +1 asflicense 0m 39s The patch does not generate ASF License warnings. 235m 51s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.tools.TestDFSZKFailoverController   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14414 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867635/HADOOP-14414.01.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 80970a1db56e 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 09f28da Default Java 1.8.0_121 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12296/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12296/testReport/ modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-tools/hadoop-sls . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12296/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          boky01 Andras Bokor added a comment -

          After HADOOP-14401 my patch does not apply to trunk. Rebasing with HADOOP-14414.02.patch.

          Show
          boky01 Andras Bokor added a comment - After HADOOP-14401 my patch does not apply to trunk. Rebasing with HADOOP-14414.02.patch .
          Hide
          boky01 Andras Bokor added a comment -

          Reattach patch 02 to kick Hadoop QA.

          Show
          boky01 Andras Bokor added a comment - Reattach patch 02 to kick Hadoop QA .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 1m 50s Maven dependency ordering for branch
          +1 mvninstall 14m 26s trunk passed
          +1 compile 17m 49s trunk passed
          +1 mvnsite 2m 42s trunk passed
          +1 mvneclipse 1m 30s trunk passed
          +1 javadoc 1m 24s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 9s the patch passed
          +1 compile 16m 49s the patch passed
          +1 javac 16m 49s the patch passed
          +1 mvnsite 2m 37s the patch passed
          +1 mvneclipse 1m 29s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 5s The patch has no ill-formed XML file.
          +1 javadoc 1m 19s the patch passed
          +1 unit 3m 13s hadoop-auth in the patch passed.
          +1 unit 3m 6s hadoop-kms in the patch passed.
          +1 unit 3m 35s hadoop-hdfs-httpfs in the patch passed.
          -1 unit 8m 1s hadoop-sls in the patch failed.
          +1 asflicense 0m 35s The patch does not generate ASF License warnings.
          84m 4s



          Reason Tests
          Failed junit tests hadoop.yarn.sls.TestSLSRunner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14414
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867946/HADOOP-14414.02.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 3d244d828a75 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6600abb
          Default Java 1.8.0_131
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12304/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12304/testReport/
          modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-tools/hadoop-sls U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12304/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 1m 50s Maven dependency ordering for branch +1 mvninstall 14m 26s trunk passed +1 compile 17m 49s trunk passed +1 mvnsite 2m 42s trunk passed +1 mvneclipse 1m 30s trunk passed +1 javadoc 1m 24s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 9s the patch passed +1 compile 16m 49s the patch passed +1 javac 16m 49s the patch passed +1 mvnsite 2m 37s the patch passed +1 mvneclipse 1m 29s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 5s The patch has no ill-formed XML file. +1 javadoc 1m 19s the patch passed +1 unit 3m 13s hadoop-auth in the patch passed. +1 unit 3m 6s hadoop-kms in the patch passed. +1 unit 3m 35s hadoop-hdfs-httpfs in the patch passed. -1 unit 8m 1s hadoop-sls in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 84m 4s Reason Tests Failed junit tests hadoop.yarn.sls.TestSLSRunner Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14414 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867946/HADOOP-14414.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 3d244d828a75 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6600abb Default Java 1.8.0_131 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12304/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12304/testReport/ modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-tools/hadoop-sls U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12304/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          boky01 Andras Bokor added a comment -

          Test failure is not related.

          Show
          boky01 Andras Bokor added a comment - Test failure is not related.
          Hide
          boky01 Andras Bokor added a comment -

          Uploading missingFiles. It's a diff between pre- and post-patch state of files.
          It shows only dependency-analysis.html, index.html, project-reports.html, and the html files that generated from md files are affected.
          These files are not necessary during docs generation. They are also generated at site phrase and part of the web site.

          Sidenote: is dependency-analysis.html used by anyone at all?

          Show
          boky01 Andras Bokor added a comment - Uploading missingFiles . It's a diff between pre- and post-patch state of files. It shows only dependency-analysis.html, index.html, project-reports.html, and the html files that generated from md files are affected. These files are not necessary during docs generation. They are also generated at site phrase and part of the web site. Sidenote: is dependency-analysis.html used by anyone at all?
          Hide
          aw Allen Wittenauer added a comment -

          +1 committing to trunk.

          Thanks for the contribution!

          Show
          aw Allen Wittenauer added a comment - +1 committing to trunk. Thanks for the contribution!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12299 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12299/)
          HADOOP-14414. Calling maven-site-plugin directly for docs profile is (aw: rev a3fee475f72c8548aafd8574da1d3f1745f0471d)

          • (edit) hadoop-tools/hadoop-sls/pom.xml
          • (edit) hadoop-common-project/hadoop-auth/pom.xml
          • (edit) hadoop-common-project/hadoop-kms/pom.xml
          • (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12299 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12299/ ) HADOOP-14414 . Calling maven-site-plugin directly for docs profile is (aw: rev a3fee475f72c8548aafd8574da1d3f1745f0471d) (edit) hadoop-tools/hadoop-sls/pom.xml (edit) hadoop-common-project/hadoop-auth/pom.xml (edit) hadoop-common-project/hadoop-kms/pom.xml (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml

            People

            • Assignee:
              boky01 Andras Bokor
              Reporter:
              boky01 Andras Bokor
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development