Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-658

NPE in distcp if source path does not exist

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: distcp
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      distcp throws NullPointerException if the source path does not exist. It should emit a proper exception with meaningful error message.

      1. d_better_error_report.patch
        1 kB
        Ravi Gummadi
      2. d_better_error_report_v1.patch
        1 kB
        Ravi Gummadi

        Activity

        Hide
        Ravi Gummadi added a comment -

        Attaching patch that makes distcp to emit proper exception with meaningful error message in the case of 'source path does not exist'.
        Also emits a meaningful error message in map if any of the source files is not accessible(dfs is not accessible or the file is deleted).

        Please review and provide your comments.

        Show
        Ravi Gummadi added a comment - Attaching patch that makes distcp to emit proper exception with meaningful error message in the case of 'source path does not exist'. Also emits a meaningful error message in map if any of the source files is not accessible(dfs is not accessible or the file is deleted). Please review and provide your comments.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I suggest change the message "Either the dfs is ... " to "Either the file system is ... " since the file system may not be a dfs. It may even be better to obtain the fs info from the src fs or the FNFE object.

        Show
        Tsz Wo Nicholas Sze added a comment - I suggest change the message "Either the dfs is ... " to "Either the file system is ... " since the file system may not be a dfs. It may even be better to obtain the fs info from the src fs or the FNFE object.
        Hide
        Ravi Gummadi added a comment -

        Attaching new patch with suggested changes.

        Show
        Ravi Gummadi added a comment - Attaching new patch with suggested changes.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
        Hide
        Chris Douglas added a comment -
             [exec] -1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
             [exec]                         Please justify why no new tests are needed for this patch.
             [exec]                         Also please list what manual steps were performed to verify this patch.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        
        Show
        Chris Douglas added a comment - [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        Hide
        Chris Douglas added a comment -

        I committed this. Thanks, Ravi!

        Show
        Chris Douglas added a comment - I committed this. Thanks, Ravi!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #15 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/15/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #15 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/15/ )

          People

          • Assignee:
            Ravi Gummadi
            Reporter:
            Ravi Gummadi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development