Hive
  1. Hive
  2. HIVE-1893

ConditionalResolverSkewJoin.getTasks may throw NullPointerException

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.4.0, 0.4.1, 0.5.0, 0.6.0
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None
    • Environment:

      RedHat 2.6.18-164.11.1.el5, JDK 1.6.0_06

      Description

      In ConditionalResolverSkewJoin.getTasks(HiveConf, Object), FileStatus[] fstatus = inpFs.listStatus(dirPath) and then fstatus.length is accessed. However, DistributedFileSystem.listStatus(Path p) could return null, thus fstatus may be null, and then fstatus.length will throw NullPointerException.

        Issue Links

          Activity

          Hide
          Carl Steinbach added a comment -

          Implementations of FileSystem that don't adhere to the spec have been a
          recurring problem – see the linked issues.

          Show
          Carl Steinbach added a comment - Implementations of FileSystem that don't adhere to the spec have been a recurring problem – see the linked issues.
          Hide
          Carl Steinbach added a comment -

          I grepped through the Hive source code and found this problem in many places.

          Instead of modifying all of these files to add null checks I recommend instead that we
          create a HiveFileSystem wrapper class and put the checks there, and then modify
          Hive code to access FileSystem objects through the wrapper. This will allow Hive
          to rely on the FileSystem spec even when implementations of the FileSystem
          spec don't adhere to it.

          Show
          Carl Steinbach added a comment - I grepped through the Hive source code and found this problem in many places. Instead of modifying all of these files to add null checks I recommend instead that we create a HiveFileSystem wrapper class and put the checks there, and then modify Hive code to access FileSystem objects through the wrapper. This will allow Hive to rely on the FileSystem spec even when implementations of the FileSystem spec don't adhere to it.

            People

            • Assignee:
              Unassigned
              Reporter:
              Zhai Yantang
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development