Hive
  1. Hive
  2. HIVE-2154

add exception handling to hive's record reader

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. HIVE-2154.1.patch
      24 kB
      He Yongqiang
    2. HIVE-2154.2.patch
      25 kB
      He Yongqiang
    3. HIVE-2154.3.patch
      25 kB
      He Yongqiang

      Activity

      Carl Steinbach made changes -
      Status Resolved [ 5 ] Closed [ 6 ]
      Hide
      Hudson added a comment -

      Integrated in Hive-trunk-h0.21 #756 (See https://builds.apache.org/hudson/job/Hive-trunk-h0.21/756/)
      HIVE-2154. add exception handling to hive's record reader (Yongqiang He via Ning Zhang)

      nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1130427
      Files :

      • /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java
      • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java
      • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java
      • /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java
      • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java
      • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io
      • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java
      • /hive/trunk/conf/hive-default.xml
      Show
      Hudson added a comment - Integrated in Hive-trunk-h0.21 #756 (See https://builds.apache.org/hudson/job/Hive-trunk-h0.21/756/ ) HIVE-2154 . add exception handling to hive's record reader (Yongqiang He via Ning Zhang) nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1130427 Files : /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java /hive/trunk/conf/hive-default.xml
      Ning Zhang made changes -
      Status Patch Available [ 10002 ] Resolved [ 5 ]
      Hadoop Flags [Reviewed]
      Fix Version/s 0.8.0 [ 12316178 ]
      Resolution Fixed [ 1 ]
      Hide
      Ning Zhang added a comment -

      Committed. Thanks Yongqiang!

      Show
      Ning Zhang added a comment - Committed. Thanks Yongqiang!
      Hide
      Ning Zhang added a comment -

      +1

      Show
      Ning Zhang added a comment - +1
      He Yongqiang made changes -
      Attachment HIVE-2154.3.patch [ 12481146 ]
      Hide
      jiraposter@reviews.apache.org added a comment -

      On 2011-06-01 06:48:14, Ning Zhang wrote:

      > trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java, line 57

      > <https://reviews.apache.org/r/812/diff/2/?file=20084#file20084line57>

      >

      > Do you need to throw exception when all handler return false?

      ioExpectionHandlerChain will throw exception if all handler returns false.

      • Yongqiang

      -----------------------------------------------------------
      This is an automatically generated e-mail. To reply, visit:
      https://reviews.apache.org/r/812/#review741
      -----------------------------------------------------------

      On 2011-06-01 00:38:42, Yongqiang He wrote:

      -----------------------------------------------------------

      This is an automatically generated e-mail. To reply, visit:

      https://reviews.apache.org/r/812/

      -----------------------------------------------------------

      (Updated 2011-06-01 00:38:42)

      Review request for hive.

      Summary

      -------

      add exception handling to hive's record reader

      This addresses bug HIVE-2154.

      https://issues.apache.org/jira/browse/HIVE-2154

      Diffs

      -----

      trunk/conf/hive-default.xml 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1129953

      trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1129953

      trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1129953

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java PRE-CREATION

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION

      Diff: https://reviews.apache.org/r/812/diff

      Testing

      -------

      Thanks,

      Yongqiang

      Show
      jiraposter@reviews.apache.org added a comment - On 2011-06-01 06:48:14, Ning Zhang wrote: > trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java, line 57 > < https://reviews.apache.org/r/812/diff/2/?file=20084#file20084line57 > > > Do you need to throw exception when all handler return false? ioExpectionHandlerChain will throw exception if all handler returns false. Yongqiang ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/812/#review741 ----------------------------------------------------------- On 2011-06-01 00:38:42, Yongqiang He wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/812/ ----------------------------------------------------------- (Updated 2011-06-01 00:38:42) Review request for hive. Summary ------- add exception handling to hive's record reader This addresses bug HIVE-2154 . https://issues.apache.org/jira/browse/HIVE-2154 Diffs ----- trunk/conf/hive-default.xml 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1129953 trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1129953 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1129953 trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION Diff: https://reviews.apache.org/r/812/diff Testing ------- Thanks, Yongqiang
      Hide
      jiraposter@reviews.apache.org added a comment -

      -----------------------------------------------------------
      This is an automatically generated e-mail. To reply, visit:
      https://reviews.apache.org/r/812/#review741
      -----------------------------------------------------------

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java
      <https://reviews.apache.org/r/812/#comment1516>

      This function is not intuitive. Maybe you need more comments about the why it is designed this way.

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java
      <https://reviews.apache.org/r/812/#comment1514>

      Do you need to throw exception when all handler return false?

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java
      <https://reviews.apache.org/r/812/#comment1515>

      same as above

      • Ning

      On 2011-06-01 00:38:42, Yongqiang He wrote:

      -----------------------------------------------------------

      This is an automatically generated e-mail. To reply, visit:

      https://reviews.apache.org/r/812/

      -----------------------------------------------------------

      (Updated 2011-06-01 00:38:42)

      Review request for hive.

      Summary

      -------

      add exception handling to hive's record reader

      This addresses bug HIVE-2154.

      https://issues.apache.org/jira/browse/HIVE-2154

      Diffs

      -----

      trunk/conf/hive-default.xml 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1129953

      trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1129953

      trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1129953

      trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1129953

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java PRE-CREATION

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION

      Diff: https://reviews.apache.org/r/812/diff

      Testing

      -------

      Thanks,

      Yongqiang

      Show
      jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/812/#review741 ----------------------------------------------------------- trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java < https://reviews.apache.org/r/812/#comment1516 > This function is not intuitive. Maybe you need more comments about the why it is designed this way. trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java < https://reviews.apache.org/r/812/#comment1514 > Do you need to throw exception when all handler return false? trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java < https://reviews.apache.org/r/812/#comment1515 > same as above Ning On 2011-06-01 00:38:42, Yongqiang He wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/812/ ----------------------------------------------------------- (Updated 2011-06-01 00:38:42) Review request for hive. Summary ------- add exception handling to hive's record reader This addresses bug HIVE-2154 . https://issues.apache.org/jira/browse/HIVE-2154 Diffs ----- trunk/conf/hive-default.xml 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1129953 trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1129953 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1129953 trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION Diff: https://reviews.apache.org/r/812/diff Testing ------- Thanks, Yongqiang
      Hide
      jiraposter@reviews.apache.org added a comment -

      -----------------------------------------------------------
      This is an automatically generated e-mail. To reply, visit:
      https://reviews.apache.org/r/812/
      -----------------------------------------------------------

      (Updated 2011-06-01 00:38:42.863958)

      Review request for hive.

      Changes
      -------

      address Ning's comments to move some code to hadoopshims

      Summary
      -------

      add exception handling to hive's record reader

      This addresses bug HIVE-2154.
      https://issues.apache.org/jira/browse/HIVE-2154

      Diffs (updated)


      trunk/conf/hive-default.xml 1129953
      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1129953
      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1129953
      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1129953
      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1129953
      trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1129953
      trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1129953
      trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1129953
      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION
      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION
      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java PRE-CREATION
      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION

      Diff: https://reviews.apache.org/r/812/diff

      Testing
      -------

      Thanks,

      Yongqiang

      Show
      jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/812/ ----------------------------------------------------------- (Updated 2011-06-01 00:38:42.863958) Review request for hive. Changes ------- address Ning's comments to move some code to hadoopshims Summary ------- add exception handling to hive's record reader This addresses bug HIVE-2154 . https://issues.apache.org/jira/browse/HIVE-2154 Diffs (updated) trunk/conf/hive-default.xml 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1129953 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1129953 trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1129953 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1129953 trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerUtil.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION Diff: https://reviews.apache.org/r/812/diff Testing ------- Thanks, Yongqiang
      He Yongqiang made changes -
      Attachment HIVE-2154.2.patch [ 12481020 ]
      Hide
      jiraposter@reviews.apache.org added a comment -

      -----------------------------------------------------------
      This is an automatically generated e-mail. To reply, visit:
      https://reviews.apache.org/r/812/#review740
      -----------------------------------------------------------

      trunk/conf/hive-default.xml
      <https://reviews.apache.org/r/812/#comment1493>

      comment should be more specific: class names of the exception handlers.

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java
      <https://reviews.apache.org/r/812/#comment1494>

      rename the variable to indicate it is a chain rather than just 1 handler.

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java
      <https://reviews.apache.org/r/812/#comment1497>

      this function throws IOException but the original code throws RuntimeException. Should check if this is OK.

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java
      <https://reviews.apache.org/r/812/#comment1496>

      same as below

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java
      <https://reviews.apache.org/r/812/#comment1495>

      check if the function return false and throw exception

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
      <https://reviews.apache.org/r/812/#comment1498>

      indentation

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
      <https://reviews.apache.org/r/812/#comment1499>

      comment on what the innerReader is when there is an exception in the first call of getRecordReader().

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
      <https://reviews.apache.org/r/812/#comment1500>

      also need to throw exception when all handlers return false. Should try to refactor the code

      • Ning

      On 2011-05-31 20:31:23, Yongqiang He wrote:

      -----------------------------------------------------------

      This is an automatically generated e-mail. To reply, visit:

      https://reviews.apache.org/r/812/

      -----------------------------------------------------------

      (Updated 2011-05-31 20:31:23)

      Review request for hive.

      Summary

      -------

      add exception handling to hive's record reader

      This addresses bug HIVE-2154.

      https://issues.apache.org/jira/browse/HIVE-2154

      Diffs

      -----

      trunk/conf/hive-default.xml 1127229

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1101261

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1101261

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1101261

      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1101261

      trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1101261

      trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1101261

      trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1101261

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION

      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION

      Diff: https://reviews.apache.org/r/812/diff

      Testing

      -------

      Thanks,

      Yongqiang

      Show
      jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/812/#review740 ----------------------------------------------------------- trunk/conf/hive-default.xml < https://reviews.apache.org/r/812/#comment1493 > comment should be more specific: class names of the exception handlers. trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java < https://reviews.apache.org/r/812/#comment1494 > rename the variable to indicate it is a chain rather than just 1 handler. trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java < https://reviews.apache.org/r/812/#comment1497 > this function throws IOException but the original code throws RuntimeException. Should check if this is OK. trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java < https://reviews.apache.org/r/812/#comment1496 > same as below trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java < https://reviews.apache.org/r/812/#comment1495 > check if the function return false and throw exception trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java < https://reviews.apache.org/r/812/#comment1498 > indentation trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java < https://reviews.apache.org/r/812/#comment1499 > comment on what the innerReader is when there is an exception in the first call of getRecordReader(). trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java < https://reviews.apache.org/r/812/#comment1500 > also need to throw exception when all handlers return false. Should try to refactor the code Ning On 2011-05-31 20:31:23, Yongqiang He wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/812/ ----------------------------------------------------------- (Updated 2011-05-31 20:31:23) Review request for hive. Summary ------- add exception handling to hive's record reader This addresses bug HIVE-2154 . https://issues.apache.org/jira/browse/HIVE-2154 Diffs ----- trunk/conf/hive-default.xml 1127229 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1101261 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1101261 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1101261 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1101261 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1101261 trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1101261 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1101261 trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION Diff: https://reviews.apache.org/r/812/diff Testing ------- Thanks, Yongqiang
      Hide
      jiraposter@reviews.apache.org added a comment -

      -----------------------------------------------------------
      This is an automatically generated e-mail. To reply, visit:
      https://reviews.apache.org/r/812/
      -----------------------------------------------------------

      (Updated 2011-05-31 20:31:23.581581)

      Review request for hive.

      Changes
      -------

      add exception handling to hive's record reader

      Summary
      -------

      add exception handling to hive's record reader

      This addresses bug HIVE-2154.
      https://issues.apache.org/jira/browse/HIVE-2154

      Diffs


      trunk/conf/hive-default.xml 1127229
      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1101261
      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1101261
      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1101261
      trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1101261
      trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1101261
      trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1101261
      trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1101261
      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION
      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION
      trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION

      Diff: https://reviews.apache.org/r/812/diff

      Testing
      -------

      Thanks,

      Yongqiang

      Show
      jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/812/ ----------------------------------------------------------- (Updated 2011-05-31 20:31:23.581581) Review request for hive. Changes ------- add exception handling to hive's record reader Summary ------- add exception handling to hive's record reader This addresses bug HIVE-2154 . https://issues.apache.org/jira/browse/HIVE-2154 Diffs trunk/conf/hive-default.xml 1127229 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveRecordReader.java 1101261 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 1101261 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveRecordReader.java 1101261 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java 1101261 trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java 1101261 trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 1101261 trunk/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 1101261 trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandler.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionHandlerChain.java PRE-CREATION trunk/shims/src/common/java/org/apache/hadoop/hive/io/HiveIOExceptionNextHandleResult.java PRE-CREATION Diff: https://reviews.apache.org/r/812/diff Testing ------- Thanks, Yongqiang
      Show
      He Yongqiang added a comment - https://reviews.apache.org/r/812/
      Hide
      Ning Zhang added a comment -

      Yongqiang, can you post it to review board? It's much easier that way.

      Show
      Ning Zhang added a comment - Yongqiang, can you post it to review board? It's much easier that way.
      He Yongqiang made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      He Yongqiang made changes -
      Field Original Value New Value
      Attachment HIVE-2154.1.patch [ 12480981 ]
      He Yongqiang created issue -

        People

        • Assignee:
          He Yongqiang
          Reporter:
          He Yongqiang
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development