Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9274

Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec should be consistent

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Always see following error log while running:

      ERROR datanode.DirectoryScanner (DirectoryScanner.java:<init>(430)) - dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
      
      <property>
        <name>dfs.datanode.directoryscan.throttle.limit.ms.per.sec</name>
        <value>0</value>
      ...
      

      The default value should be 1000 and consistent with DFS_DATANODE_DIRECTORYSCAN_THROTTLE_LIMIT_MS_PER_SEC_DEFAULT

        Activity

        Hide
        templedf Daniel Templeton added a comment -

        Thanks, Yi Liu. Not sure how that slipped past me. +1 (non-binding)

        Show
        templedf Daniel Templeton added a comment - Thanks, Yi Liu . Not sure how that slipped past me. +1 (non-binding)
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 15m 58s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 57s There were no new javac warning messages.
        +1 javadoc 10m 41s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 native 3m 14s Pre-build of native portion
        +1 hdfs tests 49m 32s Tests passed in hadoop-hdfs.
            89m 55s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12767719/HDFS-9274.001.patch
        Optional Tests javadoc javac unit
        git revision trunk / 0c4af0f
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13104/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13104/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13104/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 58s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 57s There were no new javac warning messages. +1 javadoc 10m 41s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 native 3m 14s Pre-build of native portion +1 hdfs tests 49m 32s Tests passed in hadoop-hdfs.     89m 55s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767719/HDFS-9274.001.patch Optional Tests javadoc javac unit git revision trunk / 0c4af0f hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13104/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13104/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13104/console This message was automatically generated.
        Hide
        zhz Zhe Zhang added a comment -

        Good catch Yi! +1 on the patch. I just committed to both trunk and branch-2.

        Also thanks Daniel for the review.

        Show
        zhz Zhe Zhang added a comment - Good catch Yi! +1 on the patch. I just committed to both trunk and branch-2. Also thanks Daniel for the review.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1300 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1300/)
        HDFS-9274. Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1300 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1300/ ) HDFS-9274 . Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8681 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8681/)
        HDFS-9274. Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8681 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8681/ ) HDFS-9274 . Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #579 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/579/)
        HDFS-9274. Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #579 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/579/ ) HDFS-9274 . Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2512 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2512/)
        HDFS-9274. Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2512 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2512/ ) HDFS-9274 . Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #565 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/565/)
        HDFS-9274. Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #565 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/565/ ) HDFS-9274 . Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #523 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/523/)
        HDFS-9274. Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #523 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/523/ ) HDFS-9274 . Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2460 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2460/)
        HDFS-9274. Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2460 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2460/ ) HDFS-9274 . Default value of (zhz: rev 59ce780d53fda1d2823e5da23d03ca68b15a4bf9) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

          People

          • Assignee:
            hitliuyi Yi Liu
            Reporter:
            hitliuyi Yi Liu
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development