HBase
  1. HBase
  2. HBASE-3474

HFileOutputFormat to use column family's compression algorithm

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.92.0
    • Fix Version/s: 0.92.0
    • Component/s: mapreduce
    • Labels:
      None
    • Environment:

      All

    • Hadoop Flags:
      Reviewed
    • Release Note:
      HFileOutputFormat to use column family's compression algorithm instead of a blanket all column family wide

      Description

      HFileOutputFormat currently creates HFile writer's using a compression algorithm set as configuration "hbase.hregion.max.filesize" with default as no compression. The code does not take into account the compression algorithm configured for the table's column family. As a result bulk uploaded tables are not compressed until a major compaction is run on them. This could be fixed by using the column family descriptors while creating HFile writers.

      1. hbase-3474.txt
        16 kB
        Ashish Shinde
      2. hbase-3474.txt
        17 kB
        Todd Lipcon
      3. patch3474.txt
        16 kB
        Ashish Shinde
      4. patch3474.txt
        4 kB
        Ashish Shinde
      5. patch3474.txt
        4 kB
        Ashish Shinde

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Ashish Shinde
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 48h
              48h
              Remaining:
              Remaining Estimate - 48h
              48h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development