[HIVE-11376] CombineHiveInputFormat is falling back to HiveInputFormat in case codecs are found for one of the input files

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Assignee: Rajat Khandelwal
• Reporter: Rajat Khandelwal
• Created: 27/Jul/15 11:39
• Updated: 09/Jul/17 04:51
• Resolved: 10/Aug/15 12:22

Description

https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java#L379

This is the exact code snippet:

/ Since there is no easy way of knowing whether MAPREDUCE-1597 is present in the tree or not,
      // we use a configuration variable for the same
      if (this.mrwork != null && !this.mrwork.getHadoopSupportsSplittable()) {
        // The following code should be removed, once
        // https://issues.apache.org/jira/browse/MAPREDUCE-1597 is fixed.
        // Hadoop does not handle non-splittable files correctly for CombineFileInputFormat,
        // so don't use CombineFileInputFormat for non-splittable files

        //ie, dont't combine if inputformat is a TextInputFormat and has compression turned on

Issue Links

• blocks: LENS-518 Move Hive dependency to Apache Hive
• is blocked by: MAPREDUCE-1597 combinefileinputformat does not work with non-splittable files
• relates to: HIVE-1280 Add option to CombineHiveInputFormat for non-splittable inputs.