Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9651

All web UIs should include a robots.txt file

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Similar to HDFS-330. So that public UIs don't get crawled.

      I can provide a patch that includes a simple robots.txt. Another alternative is probably a Filter that provides one automatically for all UIs but I don't have time to do that.

      If anyone wants to take over please go ahead.

      1. HDFS-9651.2.patch
        2 kB
        Lars Francke
      2. HDFS-9651.1.patch
        0.9 kB
        Lars Francke

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 asflicense 0m 25s Patch generated 3 ASF License warnings.
          0m 48s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782301/HDFS-9651.1.patch
          JIRA Issue HDFS-9651
          Optional Tests asflicense
          uname Linux b4e57446552a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9d04f26
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14123/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 29MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14123/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 whitespace 0m 0s Patch has no whitespace issues. -1 asflicense 0m 25s Patch generated 3 ASF License warnings. 0m 48s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782301/HDFS-9651.1.patch JIRA Issue HDFS-9651 Optional Tests asflicense uname Linux b4e57446552a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9d04f26 asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14123/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 29MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14123/console This message was automatically generated.
          Hide
          lars_francke Lars Francke added a comment -

          Do these files need a license header? If so I'll add one.

          Show
          lars_francke Lars Francke added a comment - Do these files need a license header? If so I'll add one.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Do these files need a license header? If so I'll add one.

          I think you can add them to apache-rat-plugin's exclude list. Similar to datanode/robot.txt is added in hadoop-hdfs/pom.xml

          Show
          vinayrpet Vinayakumar B added a comment - Do these files need a license header? If so I'll add one. I think you can add them to apache-rat-plugin's exclude list. Similar to datanode/robot.txt is added in hadoop-hdfs/pom.xml
          Hide
          lars_francke Lars Francke added a comment -

          Excellent, thank you Vinayakumar B! Didn't know about that one.

          I've attached a new patch that includes the exclusions.

          Show
          lars_francke Lars Francke added a comment - Excellent, thank you Vinayakumar B ! Didn't know about that one. I've attached a new patch that includes the exclusions.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 11m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 17s Maven dependency ordering for branch
          +1 mvninstall 9m 1s trunk passed
          +1 compile 1m 1s trunk passed with JDK v1.8.0_72
          +1 compile 0m 50s trunk passed with JDK v1.7.0_95
          +1 mvnsite 1m 2s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 javadoc 1m 32s trunk passed with JDK v1.8.0_72
          +1 javadoc 2m 14s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 0m 57s the patch passed
          +1 compile 1m 6s the patch passed with JDK v1.8.0_72
          +1 javac 1m 6s the patch passed
          +1 compile 0m 53s the patch passed with JDK v1.7.0_95
          +1 javac 0m 53s the patch passed
          +1 mvnsite 1m 10s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 1m 36s the patch passed with JDK v1.8.0_72
          +1 javadoc 2m 12s the patch passed with JDK v1.7.0_95
          -1 unit 90m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_72.
          -1 unit 72m 34s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          201m 55s



          Reason Tests
          JDK v1.8.0_72 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.server.datanode.TestBlockScanner
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.server.datanode.TestBlockScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786766/HDFS-9651.2.patch
          JIRA Issue HDFS-9651
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 1640dc1e45a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / aea31ee
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14425/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 77MB
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14425/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 9m 1s trunk passed +1 compile 1m 1s trunk passed with JDK v1.8.0_72 +1 compile 0m 50s trunk passed with JDK v1.7.0_95 +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 13s trunk passed +1 javadoc 1m 32s trunk passed with JDK v1.8.0_72 +1 javadoc 2m 14s trunk passed with JDK v1.7.0_95 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 1m 6s the patch passed with JDK v1.8.0_72 +1 javac 1m 6s the patch passed +1 compile 0m 53s the patch passed with JDK v1.7.0_95 +1 javac 0m 53s the patch passed +1 mvnsite 1m 10s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 1m 36s the patch passed with JDK v1.8.0_72 +1 javadoc 2m 12s the patch passed with JDK v1.7.0_95 -1 unit 90m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_72. -1 unit 72m 34s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 201m 55s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.TestFileAppend   hadoop.hdfs.server.datanode.TestBlockScanner JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.datanode.TestBlockScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786766/HDFS-9651.2.patch JIRA Issue HDFS-9651 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 1640dc1e45a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / aea31ee Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14425/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 77MB Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14425/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          lars_francke Lars Francke added a comment -

          Test failures must be unrelated. I think this is good to go.

          Show
          lars_francke Lars Francke added a comment - Test failures must be unrelated. I think this is good to go.
          Hide
          djp Junping Du added a comment -

          Patch LGTM. Will commit it in next couple of days if no objections.

          Show
          djp Junping Du added a comment - Patch LGTM. Will commit it in next couple of days if no objections.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 13m 47s trunk passed
          +1 compile 0m 52s trunk passed
          +1 mvnsite 0m 57s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 javadoc 0m 42s trunk passed
          +1 mvninstall 0m 50s the patch passed
          +1 compile 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 javadoc 0m 37s the patch passed
          -1 unit 81m 23s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          102m 57s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-9651
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786766/HDFS-9651.2.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 650ab4d1d6f7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 01aca54
          Default Java 1.8.0_121
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18876/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18876/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18876/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 47s trunk passed +1 compile 0m 52s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 13s trunk passed +1 javadoc 0m 42s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 javadoc 0m 37s the patch passed -1 unit 81m 23s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 102m 57s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-9651 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786766/HDFS-9651.2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 650ab4d1d6f7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 01aca54 Default Java 1.8.0_121 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18876/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18876/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18876/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          lars_francke Lars Francke added a comment -

          Thanks for taking a look!

          Show
          lars_francke Lars Francke added a comment - Thanks for taking a look!
          Hide
          lars_francke Lars Francke added a comment -

          Junping Du It doesn't look like there are any objections. If you have a few minutes I would very much appreciate the commit.

          Show
          lars_francke Lars Francke added a comment - Junping Du It doesn't look like there are any objections. If you have a few minutes I would very much appreciate the commit.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1, I've verified the robots.txt can be accessed via http://host:port/robots.txt. Checking this in.

          Show
          ajisakaa Akira Ajisaka added a comment - +1, I've verified the robots.txt can be accessed via http://host:port/robots.txt . Checking this in.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to trunk and branch-2. Thanks Lars Francke for the contribution and thanks Junping Du for the review.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to trunk and branch-2. Thanks Lars Francke for the contribution and thanks Junping Du for the review.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I forgot to add robots.txt files in the first commit, so robots.txt files were added in the second commit. That way there are two commits for HDFS-9651. Sorry for that.

          Show
          ajisakaa Akira Ajisaka added a comment - I forgot to add robots.txt files in the first commit, so robots.txt files were added in the second commit. That way there are two commits for HDFS-9651 . Sorry for that.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11525 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11525/)
          HDFS-9651. All web UIs should include a robots.txt file. Contributed by (aajisaka: rev 09e6caec3dbba257f11fe7246d4c25ee8fc95d8b)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11525 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11525/ ) HDFS-9651 . All web UIs should include a robots.txt file. Contributed by (aajisaka: rev 09e6caec3dbba257f11fe7246d4c25ee8fc95d8b) (edit) hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11526 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11526/)
          HDFS-9651. Add missing robots.txt files. (aajisaka: rev 8cb73255274bd46ea8aedbb28f25fa13b38723ad)

          • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/robots.txt
          • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/robots.txt
          • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/robots.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11526 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11526/ ) HDFS-9651 . Add missing robots.txt files. (aajisaka: rev 8cb73255274bd46ea8aedbb28f25fa13b38723ad) (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/robots.txt (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/robots.txt (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/robots.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/)
          HDFS-9651. All web UIs should include a robots.txt file. Contributed by (aajisaka: rev 09e6caec3dbba257f11fe7246d4c25ee8fc95d8b)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/pom.xml
            HDFS-9651. Add missing robots.txt files. (aajisaka: rev 8cb73255274bd46ea8aedbb28f25fa13b38723ad)
          • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/robots.txt
          • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/robots.txt
          • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/robots.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/ ) HDFS-9651 . All web UIs should include a robots.txt file. Contributed by (aajisaka: rev 09e6caec3dbba257f11fe7246d4c25ee8fc95d8b) (edit) hadoop-hdfs-project/hadoop-hdfs/pom.xml HDFS-9651 . Add missing robots.txt files. (aajisaka: rev 8cb73255274bd46ea8aedbb28f25fa13b38723ad) (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/robots.txt (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/robots.txt (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/robots.txt

            People

            • Assignee:
              lars_francke Lars Francke
              Reporter:
              lars_francke Lars Francke
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development