Hadoop Common
  1. Hadoop Common
  2. HADOOP-3703

[HOD] logcondense needs to use the new pattern of output in hadoop dfs -lsr

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.18.0
    • Fix Version/s: 0.18.0
    • Component/s: contrib/hod
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Modified logcondense.py to use the new format of hadoop dfs -lsr output. This version of logcondense would not work with previous versions of Hadoop and hence is incompatible.

      Description

      In Hadoop 0.18, the format of hadoop dfs -ls and hadoop dfs -lsr commands has changed. It is a Unix like output now, with the filename being printed as the last column of the file listing as opposed to the first. logcondense.py needs to use this new format.

      1. HADOOP-3703
        0.6 kB
        Vinod Kumar Vavilapalli

        Activity

        Hide
        Vinod Kumar Vavilapalli added a comment -

        Attaching (minor) patch/ one-line fix.

        Show
        Vinod Kumar Vavilapalli added a comment - Attaching (minor) patch/ one-line fix.
        Hide
        Hemanth Yamijala added a comment -

        Looks good. +1.

        Show
        Hemanth Yamijala added a comment - Looks good. +1.
        Hide
        Hemanth Yamijala added a comment -

        Running through hudson.

        Show
        Hemanth Yamijala added a comment - Running through hudson.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12385391/HADOOP-3703
        against trunk revision 674671.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2810/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2810/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2810/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2810/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12385391/HADOOP-3703 against trunk revision 674671. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2810/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2810/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2810/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2810/console This message is automatically generated.
        Hide
        Hemanth Yamijala added a comment -

        The script depends on an external HDFS which requires some stubbing to be able to write unit test cases. This is skipped for now.
        The failure in contrib tests seems unrelated. The console output actually says contrib tests have run successfully.

        Show
        Hemanth Yamijala added a comment - The script depends on an external HDFS which requires some stubbing to be able to write unit test cases. This is skipped for now. The failure in contrib tests seems unrelated. The console output actually says contrib tests have run successfully.
        Hide
        Hemanth Yamijala added a comment -

        I just committed this to trunk. Thanks, Vinod

        Show
        Hemanth Yamijala added a comment - I just committed this to trunk. Thanks, Vinod
        Hide
        Devaraj Das added a comment -

        I committed this to the 0.18 branch.

        Show
        Devaraj Das added a comment - I committed this to the 0.18 branch.
        Hide
        Karam Singh added a comment -

        Verified that logcondense.py deletes datanod and tasktrackers in hadoop 0.18.0. it works with bothh -dynamic true/false

        Show
        Karam Singh added a comment - Verified that logcondense.py deletes datanod and tasktrackers in hadoop 0.18.0. it works with bothh -dynamic true/false
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #581 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/ )

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Hemanth Yamijala
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development