Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4790

MapredLocalTask task does not make virtual columns

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      From mailing list, http://www.mail-archive.com/user@hive.apache.org/msg08264.html

                      SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON b.rownumber = a.number;
      fails with this error:
       
          > SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON b.rownumber = a.number;
      Automatically selecting local only mode for query
      Total MapReduce jobs = 1
      setting HADOOP_USER_NAME        pmarron
      13/06/25 10:52:56 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
      Execution log at: /tmp/pmarron/.log
      2013-06-25 10:52:56     Starting to launch local task to process map join;      maximum memory = 932118528
      java.lang.RuntimeException: cannot find field block__offset__inside__file from [0:rownumber, 1:offset]
              at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366)
              at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldRef(LazySimpleStructObjectInspector.java:168)
              at org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldRef(DelegatedStructObjectInspector.java:74)
              at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
              at org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:68)
              at org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.initializeOp(HashTableSinkOperator.java:222)
              at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
              at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
              at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
              at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
              at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
              at org.apache.hadoop.hive.ql.exec.MapredLocalTask.initializeOperators(MapredLocalTask.java:394)
              at org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:277)
              at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:676)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
      Execution failed with exit status: 2
      

        Attachments

        1. HIVE-4790.14.patch.txt
          39 kB
          Navis
        2. HIVE-4790.13.patch.txt
          37 kB
          Navis
        3. HIVE-4790.12.patch.txt
          39 kB
          Navis
        4. HIVE-4790.11.patch.txt
          37 kB
          Navis
        5. HIVE-4790.10.patch.txt
          38 kB
          Navis
        6. HIVE-4790.9.patch.txt
          40 kB
          Navis
        7. HIVE-4790.8.patch.txt
          40 kB
          Navis
        8. HIVE-4790.7.patch.txt
          37 kB
          Navis
        9. HIVE-4790.6.patch.txt
          34 kB
          Navis
        10. HIVE-4790.5.patch.txt
          32 kB
          Navis
        11. D11511.4.patch
          30 kB
          Phabricator
        12. D11511.3.patch
          22 kB
          Phabricator
        13. HIVE-4790.D11511.2.patch
          27 kB
          Phabricator
        14. HIVE-4790.D11511.1.patch
          27 kB
          Phabricator

          Issue Links

            Activity

              People

              • Assignee:
                navis Navis
                Reporter:
                navis Navis
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: