Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-85

separate compression options for different output types

    XMLWordPrintableJSON

    Details

      Description

      currently hive uses mapred.output.compress to determine compression for all output files. however not all files are final output. at least three different kinds of output files are generated:
      1. intermediate output files for next map-reduce job
      2. files targeted for result hdfs directories or hive tables/partitions (which are just hdfs dirs)
      3. files written to user local directories (downloading results)

      the plan is to provide three separate options for controlling 1,2,3 separately. we may want to split (2) in case compression is determined by table metadata (and not session options).

        Attachments

        1. hive-85.patch.5
          139 kB
          Joydeep Sen Sarma
        2. hive-85.patch.4
          138 kB
          Joydeep Sen Sarma
        3. hive-85.patch.3
          137 kB
          Joydeep Sen Sarma
        4. lt100.txt.deflate
          0.3 kB
          Joydeep Sen Sarma
        5. hive-85.patch.2
          136 kB
          Joydeep Sen Sarma
        6. hive-85.patch.1
          135 kB
          Joydeep Sen Sarma

          Activity

            People

            • Assignee:
              jsensarma Joydeep Sen Sarma
              Reporter:
              jsensarma Joydeep Sen Sarma
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: