Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-658

NPE in distcp if source path does not exist

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: distcp
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      distcp throws NullPointerException if the source path does not exist. It should emit a proper exception with meaningful error message.

      1. d_better_error_report.patch
        1 kB
        Ravi Gummadi
      2. d_better_error_report_v1.patch
        1 kB
        Ravi Gummadi

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        1d 22h 25m 1 Ravi Gummadi 26/Jun/09 09:52
        Patch Available Patch Available Resolved Resolved
        2d 22h 11m 1 Chris Douglas 29/Jun/09 08:04
        Resolved Resolved Closed Closed
        421d 14h 10m 1 Tom White 24/Aug/10 22:14
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #15 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/15/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #15 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/15/ )
        Chris Douglas made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Chris Douglas added a comment -

        I committed this. Thanks, Ravi!

        Show
        Chris Douglas added a comment - I committed this. Thanks, Ravi!
        Hide
        Chris Douglas added a comment -
             [exec] -1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
             [exec]                         Please justify why no new tests are needed for this patch.
             [exec]                         Also please list what manual steps were performed to verify this patch.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        
        Show
        Chris Douglas added a comment - [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        Ravi Gummadi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags [Reviewed]
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
        Ravi Gummadi made changes -
        Attachment d_better_error_report_v1.patch [ 12411746 ]
        Hide
        Ravi Gummadi added a comment -

        Attaching new patch with suggested changes.

        Show
        Ravi Gummadi added a comment - Attaching new patch with suggested changes.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I suggest change the message "Either the dfs is ... " to "Either the file system is ... " since the file system may not be a dfs. It may even be better to obtain the fs info from the src fs or the FNFE object.

        Show
        Tsz Wo Nicholas Sze added a comment - I suggest change the message "Either the dfs is ... " to "Either the file system is ... " since the file system may not be a dfs. It may even be better to obtain the fs info from the src fs or the FNFE object.
        Ravi Gummadi made changes -
        Field Original Value New Value
        Attachment d_better_error_report.patch [ 12411631 ]
        Hide
        Ravi Gummadi added a comment -

        Attaching patch that makes distcp to emit proper exception with meaningful error message in the case of 'source path does not exist'.
        Also emits a meaningful error message in map if any of the source files is not accessible(dfs is not accessible or the file is deleted).

        Please review and provide your comments.

        Show
        Ravi Gummadi added a comment - Attaching patch that makes distcp to emit proper exception with meaningful error message in the case of 'source path does not exist'. Also emits a meaningful error message in map if any of the source files is not accessible(dfs is not accessible or the file is deleted). Please review and provide your comments.
        Ravi Gummadi created issue -

          People

          • Assignee:
            Ravi Gummadi
            Reporter:
            Ravi Gummadi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development