Hadoop Common
  1. Hadoop Common
  2. HADOOP-3683

Hadoop dfs metric FilesListed shows number of files listed instead of operations

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.17.1
    • Fix Version/s: 0.18.0
    • Component/s: metrics
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Change FileListed to getNumGetListingOps and add CreateFileOps, DeleteFileOps and AddBlockOps metrics.

      Description

      Hadoop dfs metric FilesListed shows number of files listed, it should be number of fileListing operations instead.

      1. HADOOP-Metricsfix-3-19.patch
        6 kB
        Lohit Vijayarenu
      2. HADOOP-Metricsfix-3.patch
        6 kB
        Lohit Vijayarenu
      3. HADOOP-Metricsfix-2.patch
        6 kB
        Lohit Vijayarenu
      4. HADOOP-Metricsfix.patch
        6 kB
        Lohit Vijayarenu
      5. HADOOP-Metricsfix.patch
        6 kB
        Lohit Vijayarenu

        Activity

        Lohit Vijayarenu created issue -
        Hide
        Lohit Vijayarenu added a comment -

        Attached patch fixes this

        • changed FilesListed to FilesListedOps
        • added FilesCreatedOps, FilesDeletedOps and AddBlockOps
        Show
        Lohit Vijayarenu added a comment - Attached patch fixes this changed FilesListed to FilesListedOps added FilesCreatedOps, FilesDeletedOps and AddBlockOps
        Lohit Vijayarenu made changes -
        Field Original Value New Value
        Attachment HADOOP-Metricsfix.patch [ 12385151 ]
        Hide
        Lohit Vijayarenu added a comment -

        Updated patch with changed names as suggested by Raghu.

        Show
        Lohit Vijayarenu added a comment - Updated patch with changed names as suggested by Raghu.
        Lohit Vijayarenu made changes -
        Attachment HADOOP-Metricsfix.patch [ 12385155 ]
        Hide
        Lohit Vijayarenu added a comment -

        I had renamed a public API. This attached patch deprecates it following the release process.

        Show
        Lohit Vijayarenu added a comment - I had renamed a public API. This attached patch deprecates it following the release process.
        Lohit Vijayarenu made changes -
        Attachment HADOOP-Metricsfix-2.patch [ 12385157 ]
        Lohit Vijayarenu made changes -
        Release Note Change FileListed to getNumGetListingOps and add CreateFileOps, DeleteFileOps and AddBlockOps metrics.
        Affects Version/s 0.18.0 [ 12312972 ]
        Assignee Lohit Vijayarenu [ lohit ]
        Fix Version/s 0.18.0 [ 12312972 ]
        Hadoop Flags [Incompatible change]
        Affects Version/s 0.17.1 [ 12313190 ]
        Lohit Vijayarenu made changes -
        Attachment HADOOP-Metricsfix-3.patch [ 12385158 ]
        Hide
        Raghu Angadi added a comment -

        +1.

        Show
        Raghu Angadi added a comment - +1.
        Raghu Angadi made changes -
        Hadoop Flags [Incompatible change] [Incompatible change, Reviewed]
        Lohit Vijayarenu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags [Reviewed, Incompatible change] [Incompatible change, Reviewed]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12385158/HADOOP-Metricsfix-3.patch
        against trunk revision 673517.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2787/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2787/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2787/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2787/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12385158/HADOOP-Metricsfix-3.patch against trunk revision 673517. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2787/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2787/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2787/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2787/console This message is automatically generated.
        Hide
        Lohit Vijayarenu added a comment -

        Same patch against for 0.19

        Show
        Lohit Vijayarenu added a comment - Same patch against for 0.19
        Lohit Vijayarenu made changes -
        Attachment HADOOP-Metricsfix-3-19.patch [ 12385245 ]
        Hide
        Lohit Vijayarenu added a comment -

        [exec] -1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
        [exec] Please justify why no tests are needed for this patch.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.

        Tested test-patch locally for the patch against 0.19

        Show
        Lohit Vijayarenu added a comment - [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. Tested test-patch locally for the patch against 0.19
        Hide
        Chris Douglas added a comment -

        I just committed this. Thanks, Lohit

        Show
        Chris Douglas added a comment - I just committed this. Thanks, Lohit
        Chris Douglas made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed, Incompatible change] [Incompatible change, Reviewed]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #581 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/ )
        Nigel Daley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Owen O'Malley made changes -
        Component/s dfs [ 12310710 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        3h 20m 1 Lohit Vijayarenu 03/Jul/08 01:13
        Patch Available Patch Available Resolved Resolved
        1d 2h 2m 1 Chris Douglas 04/Jul/08 03:15
        Resolved Resolved Closed Closed
        49d 17h 34m 1 Nigel Daley 22/Aug/08 20:50

          People

          • Assignee:
            Lohit Vijayarenu
            Reporter:
            Lohit Vijayarenu
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development