Hive
  1. Hive
  2. HIVE-4935

Potential NPE in MetadataOnlyOptimizer

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      In MetadataOnlyOptimizer.TableScanProcessor.process, it is possible that we consider a TableScanOperator as "MayBeMetadataOnly" when this TS does not have a conf. In MetadataOnlyOptimizer.MetadataOnlyTaskDispatcher.dispatch(Node, Stack<Node>, Object...), when we convert this TS, we want to get the alias from its conf....

      1. HIVE-4935.1.patch
        1 kB
        Yin Huai
      2. HIVE-4935.1.patch
        1 kB
        Yin Huai

        Activity

        Hide
        Ashutosh Chauhan added a comment -

        This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Yin!

        Show
        Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Yin!
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12594321/HIVE-4935.1.patch

        SUCCESS: +1 2652 tests passed

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/191/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/191/console

        Messages:

        Executing org.apache.hive.ptest.execution.CleanupPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12594321/HIVE-4935.1.patch SUCCESS: +1 2652 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/191/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/191/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
        Hide
        Yin Huai added a comment -

        btw, this test passed on my laptop

        Show
        Yin Huai added a comment - btw, this test passed on my laptop
        Hide
        Yin Huai added a comment -

        Trigger the precommit tests again.

        Show
        Yin Huai added a comment - Trigger the precommit tests again.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12594224/HIVE-4935.1.patch

        ERROR: -1 due to 1 failed/errored test(s), 2651 tests executed
        Failed tests:

        org.apache.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/188/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/188/console

        Messages:

        Executing org.apache.hive.ptest.execution.CleanupPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 1 tests failed
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12594224/HIVE-4935.1.patch ERROR: -1 due to 1 failed/errored test(s), 2651 tests executed Failed tests: org.apache.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/188/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/188/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 1 tests failed This message is automatically generated.
        Hide
        Ashutosh Chauhan added a comment -

        +1

        Show
        Ashutosh Chauhan added a comment - +1
        Hide
        Yin Huai added a comment -

        right. If that tso does not have any needed column and it does not have a conf, MetadataOnlyOptimizer will consider it as a possible TS.

        Show
        Yin Huai added a comment - right. If that tso does not have any needed column and it does not have a conf, MetadataOnlyOptimizer will consider it as a possible TS.
        Hide
        Ashutosh Chauhan added a comment -

        Which line could result in NPE? Is it

        • LOG.info("Metadata only table scan for " + tso.getConf().getAlias()); ?
        Show
        Ashutosh Chauhan added a comment - Which line could result in NPE? Is it LOG.info("Metadata only table scan for " + tso.getConf().getAlias()); ?

          People

          • Assignee:
            Yin Huai
            Reporter:
            Yin Huai
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development