Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9651

All web UIs should include a robots.txt file

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Similar to HDFS-330. So that public UIs don't get crawled.

      I can provide a patch that includes a simple robots.txt. Another alternative is probably a Filter that provides one automatically for all UIs but I don't have time to do that.

      If anyone wants to take over please go ahead.

      1. HDFS-9651.1.patch
        0.9 kB
        Lars Francke
      2. HDFS-9651.2.patch
        2 kB
        Lars Francke

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 whitespace 0m 0s Patch has no whitespace issues.
        -1 asflicense 0m 25s Patch generated 3 ASF License warnings.
        0m 48s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782301/HDFS-9651.1.patch
        JIRA Issue HDFS-9651
        Optional Tests asflicense
        uname Linux b4e57446552a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 9d04f26
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14123/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 29MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14123/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 whitespace 0m 0s Patch has no whitespace issues. -1 asflicense 0m 25s Patch generated 3 ASF License warnings. 0m 48s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782301/HDFS-9651.1.patch JIRA Issue HDFS-9651 Optional Tests asflicense uname Linux b4e57446552a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9d04f26 asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14123/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 29MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14123/console This message was automatically generated.
        Hide
        lars_francke Lars Francke added a comment -

        Do these files need a license header? If so I'll add one.

        Show
        lars_francke Lars Francke added a comment - Do these files need a license header? If so I'll add one.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Do these files need a license header? If so I'll add one.

        I think you can add them to apache-rat-plugin's exclude list. Similar to datanode/robot.txt is added in hadoop-hdfs/pom.xml

        Show
        vinayrpet Vinayakumar B added a comment - Do these files need a license header? If so I'll add one. I think you can add them to apache-rat-plugin's exclude list. Similar to datanode/robot.txt is added in hadoop-hdfs/pom.xml
        Hide
        lars_francke Lars Francke added a comment -

        Excellent, thank you Vinayakumar B! Didn't know about that one.

        I've attached a new patch that includes the exclusions.

        Show
        lars_francke Lars Francke added a comment - Excellent, thank you Vinayakumar B ! Didn't know about that one. I've attached a new patch that includes the exclusions.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 11m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 17s Maven dependency ordering for branch
        +1 mvninstall 9m 1s trunk passed
        +1 compile 1m 1s trunk passed with JDK v1.8.0_72
        +1 compile 0m 50s trunk passed with JDK v1.7.0_95
        +1 mvnsite 1m 2s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 javadoc 1m 32s trunk passed with JDK v1.8.0_72
        +1 javadoc 2m 14s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 0m 57s the patch passed
        +1 compile 1m 6s the patch passed with JDK v1.8.0_72
        +1 javac 1m 6s the patch passed
        +1 compile 0m 53s the patch passed with JDK v1.7.0_95
        +1 javac 0m 53s the patch passed
        +1 mvnsite 1m 10s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 0s The patch has no ill-formed XML file.
        +1 javadoc 1m 36s the patch passed with JDK v1.8.0_72
        +1 javadoc 2m 12s the patch passed with JDK v1.7.0_95
        -1 unit 90m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_72.
        -1 unit 72m 34s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        201m 55s



        Reason Tests
        JDK v1.8.0_72 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.namenode.TestDecommissioningStatus
          hadoop.hdfs.server.namenode.ha.TestEditLogTailer
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
          hadoop.hdfs.TestFileAppend
          hadoop.hdfs.server.datanode.TestBlockScanner
        JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.hdfs.TestFileAppend
          hadoop.hdfs.TestDFSUpgradeFromImage
          hadoop.hdfs.server.datanode.TestBlockScanner



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786766/HDFS-9651.2.patch
        JIRA Issue HDFS-9651
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
        uname Linux 1640dc1e45a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / aea31ee
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14425/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 77MB
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14425/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 9m 1s trunk passed +1 compile 1m 1s trunk passed with JDK v1.8.0_72 +1 compile 0m 50s trunk passed with JDK v1.7.0_95 +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 13s trunk passed +1 javadoc 1m 32s trunk passed with JDK v1.8.0_72 +1 javadoc 2m 14s trunk passed with JDK v1.7.0_95 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 1m 6s the patch passed with JDK v1.8.0_72 +1 javac 1m 6s the patch passed +1 compile 0m 53s the patch passed with JDK v1.7.0_95 +1 javac 0m 53s the patch passed +1 mvnsite 1m 10s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 1m 36s the patch passed with JDK v1.8.0_72 +1 javadoc 2m 12s the patch passed with JDK v1.7.0_95 -1 unit 90m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_72. -1 unit 72m 34s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 201m 55s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.TestFileAppend   hadoop.hdfs.server.datanode.TestBlockScanner JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.datanode.TestBlockScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786766/HDFS-9651.2.patch JIRA Issue HDFS-9651 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 1640dc1e45a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / aea31ee Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14425/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14425/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 77MB Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14425/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        lars_francke Lars Francke added a comment -

        Test failures must be unrelated. I think this is good to go.

        Show
        lars_francke Lars Francke added a comment - Test failures must be unrelated. I think this is good to go.
        Hide
        djp Junping Du added a comment -

        Patch LGTM. Will commit it in next couple of days if no objections.

        Show
        djp Junping Du added a comment - Patch LGTM. Will commit it in next couple of days if no objections.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 13m 47s trunk passed
        +1 compile 0m 52s trunk passed
        +1 mvnsite 0m 57s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 javadoc 0m 42s trunk passed
        +1 mvninstall 0m 50s the patch passed
        +1 compile 0m 45s the patch passed
        +1 javac 0m 45s the patch passed
        +1 mvnsite 0m 50s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 javadoc 0m 37s the patch passed
        -1 unit 81m 23s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        102m 57s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-9651
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786766/HDFS-9651.2.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
        uname Linux 650ab4d1d6f7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 01aca54
        Default Java 1.8.0_121
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18876/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18876/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18876/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 47s trunk passed +1 compile 0m 52s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 13s trunk passed +1 javadoc 0m 42s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 javadoc 0m 37s the patch passed -1 unit 81m 23s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 102m 57s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-9651 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786766/HDFS-9651.2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 650ab4d1d6f7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 01aca54 Default Java 1.8.0_121 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18876/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18876/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18876/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        lars_francke Lars Francke added a comment -

        Thanks for taking a look!

        Show
        lars_francke Lars Francke added a comment - Thanks for taking a look!
        Hide
        lars_francke Lars Francke added a comment -

        Junping Du It doesn't look like there are any objections. If you have a few minutes I would very much appreciate the commit.

        Show
        lars_francke Lars Francke added a comment - Junping Du It doesn't look like there are any objections. If you have a few minutes I would very much appreciate the commit.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        +1, I've verified the robots.txt can be accessed via http://host:port/robots.txt. Checking this in.

        Show
        ajisakaa Akira Ajisaka added a comment - +1, I've verified the robots.txt can be accessed via http://host:port/robots.txt . Checking this in.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Committed this to trunk and branch-2. Thanks Lars Francke for the contribution and thanks Junping Du for the review.

        Show
        ajisakaa Akira Ajisaka added a comment - Committed this to trunk and branch-2. Thanks Lars Francke for the contribution and thanks Junping Du for the review.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        I forgot to add robots.txt files in the first commit, so robots.txt files were added in the second commit. That way there are two commits for HDFS-9651. Sorry for that.

        Show
        ajisakaa Akira Ajisaka added a comment - I forgot to add robots.txt files in the first commit, so robots.txt files were added in the second commit. That way there are two commits for HDFS-9651 . Sorry for that.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11525 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11525/)
        HDFS-9651. All web UIs should include a robots.txt file. Contributed by (aajisaka: rev 09e6caec3dbba257f11fe7246d4c25ee8fc95d8b)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/pom.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11525 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11525/ ) HDFS-9651 . All web UIs should include a robots.txt file. Contributed by (aajisaka: rev 09e6caec3dbba257f11fe7246d4c25ee8fc95d8b) (edit) hadoop-hdfs-project/hadoop-hdfs/pom.xml
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11526 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11526/)
        HDFS-9651. Add missing robots.txt files. (aajisaka: rev 8cb73255274bd46ea8aedbb28f25fa13b38723ad)

        • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/robots.txt
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/robots.txt
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/robots.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11526 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11526/ ) HDFS-9651 . Add missing robots.txt files. (aajisaka: rev 8cb73255274bd46ea8aedbb28f25fa13b38723ad) (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/robots.txt (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/robots.txt (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/robots.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/)
        HDFS-9651. All web UIs should include a robots.txt file. Contributed by (aajisaka: rev 09e6caec3dbba257f11fe7246d4c25ee8fc95d8b)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/pom.xml
          HDFS-9651. Add missing robots.txt files. (aajisaka: rev 8cb73255274bd46ea8aedbb28f25fa13b38723ad)
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/robots.txt
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/robots.txt
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/robots.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/ ) HDFS-9651 . All web UIs should include a robots.txt file. Contributed by (aajisaka: rev 09e6caec3dbba257f11fe7246d4c25ee8fc95d8b) (edit) hadoop-hdfs-project/hadoop-hdfs/pom.xml HDFS-9651 . Add missing robots.txt files. (aajisaka: rev 8cb73255274bd46ea8aedbb28f25fa13b38723ad) (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/robots.txt (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/robots.txt (add) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/robots.txt

          People

          • Assignee:
            lars_francke Lars Francke
            Reporter:
            lars_francke Lars Francke
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development