Hadoop Common
  1. Hadoop Common
  2. HADOOP-10775

Shell operations to fail with meaningful errors on windows if winutils.exe not found

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: trunk-win, 2.7.1
    • Fix Version/s: 2.8.0
    • Component/s: util
    • Labels:
      None
    • Environment:

      windows

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      If winutils.exe can't be found HADOOP_HOME wrong/unset or other causes, then an error is logged -but when any of the Shell operations are used, an NPE is raised rather than something meaningful.

      The error message at setup time should be preserved and then raised before any attempt to invoke a winutils-driven process made

      1. HADOOP-10775.patch
        22 kB
        Steve Loughran
      2. HADOOP-10775-002.patch
        22 kB
        Steve Loughran
      3. HADOOP-10775-003.patch
        34 kB
        Steve Loughran
      4. HADOOP-10775-004.patch
        67 kB
        Steve Loughran
      5. HADOOP-10775-005.patch
        100 kB
        Steve Loughran
      6. HADOOP-10775-006.patch
        100 kB
        Steve Loughran
      7. HADOOP-10775-007.patch
        100 kB
        Steve Loughran
      8. HADOOP-10775-008.patch
        100 kB
        Steve Loughran
      9. HADOOP-10775-009.patch
        99 kB
        Steve Loughran

        Issue Links

          Activity

          Steve Loughran made changes -
          Link This issue breaks HADOOP-12478 [ HADOOP-12478 ]
          Steve Loughran made changes -
          Link This issue is duplicated by HADOOP-11003 [ HADOOP-11003 ]
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 2.8.0 [ 12329058 ]
          Resolution Fixed [ 1 ]
          Chris Nauroth made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775-009.patch [ 12765651 ]
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775-008.patch [ 12765243 ]
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775-007.patch [ 12765240 ]
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775-006.patch [ 12765229 ]
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775-005.patch [ 12764762 ]
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775-004.patch [ 12764270 ]
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Chris Nauroth made changes -
          Assignee Steve Loughran [ stevel@apache.org ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 2.8.0 [ 12329058 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775-003.patch [ 12755798 ]
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Environment Apache jenkins windows1 server windows
          Affects Version/s 2.7.1 [ 12331977 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775-002.patch [ 12755731 ]
          Steve Loughran made changes -
          Attachment HADOOP-10775.patch [ 12755728 ]
          Steve Loughran made changes -
          Link This issue is related to SPARK-6961 [ SPARK-6961 ]
          Steve Loughran made changes -
          Link This issue relates to HADOOP-11003 [ HADOOP-11003 ]
          Steve Loughran made changes -
          Link This issue relates to SPARK-2356 [ SPARK-2356 ]
          Steve Loughran made changes -
          Link This issue relates to SLIDER-201 [ SLIDER-201 ]
          Steve Loughran made changes -
          Field Original Value New Value
          Summary Shell operations could fail with meaningful errors on windows if winutils.exe not found Shell operations to fail with meaningful errors on windows if winutils.exe not found
          Steve Loughran created issue -

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development