Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18649

Deprecate KV Usage in MR to move to Cells in 3.0

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha-2
    • Fix Version/s: 2.0.0-beta-1, 2.0.0
    • Component/s: API, mapreduce
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      All the mappers and reducers output type will be now of MapReduceCell type. No more KeyValue type. How ever in branch-2 for compatibility we have allowed the older interfaces/classes that work with KeyValue to stay in the code base but they have been marked as deprecated.
      The following interfaces/classes have been deprecated in branch-2
      Import#KeyValueWritableComparablePartitioner
      Import#KeyValueWritableComparator
      Import#KeyValueWritableComparable
      Import#KeyValueReducer
      Import#KeyValueSortImporter
      Import#KeyValueImporter
      KeyValueSortReducer
      KeyValueSerialization
      WALPlayer#WALKeyValueMapper

      So any existing MR jobs that is using the above public interfaces/classes will continue to work in branch-2 and the expected output value type of those mappers and reducers can continue to be KeyValue type.

      In branch-3 the mappers and reducers output will only expect MapReduceCell as the type and will no longer work with KeyValue type.
      The new public classes/interfaces added for branch-3 and in branch-2 are
      CellSerialization
      CellSortReducer
      Import#CellWritableComparablePartitioner
      Import#CellWritableComparable
      Import#CellWritableComparator
      Import#CellReducer
      Import#CellSortImporter
      Import#CellImporter
      WALPlayer#WALCellMapper
      Show
      All the mappers and reducers output type will be now of MapReduceCell type. No more KeyValue type. How ever in branch-2 for compatibility we have allowed the older interfaces/classes that work with KeyValue to stay in the code base but they have been marked as deprecated. The following interfaces/classes have been deprecated in branch-2 Import#KeyValueWritableComparablePartitioner Import#KeyValueWritableComparator Import#KeyValueWritableComparable Import#KeyValueReducer Import#KeyValueSortImporter Import#KeyValueImporter KeyValueSortReducer KeyValueSerialization WALPlayer#WALKeyValueMapper So any existing MR jobs that is using the above public interfaces/classes will continue to work in branch-2 and the expected output value type of those mappers and reducers can continue to be KeyValue type. In branch-3 the mappers and reducers output will only expect MapReduceCell as the type and will no longer work with KeyValue type. The new public classes/interfaces added for branch-3 and in branch-2 are CellSerialization CellSortReducer Import#CellWritableComparablePartitioner Import#CellWritableComparable Import#CellWritableComparator Import#CellReducer Import#CellSortImporter Import#CellImporter WALPlayer#WALCellMapper

      Attachments

      1. HBASE-18649-branch-2.patch
        75 kB
        ramkrishna.s.vasudevan
      2. HBASE-18649-branch-2_2.patch
        172 kB
        ramkrishna.s.vasudevan
      3. HBASE-18649-branch-2_1.patch
        75 kB
        ramkrishna.s.vasudevan
      4. HBASE-18649_master.patch
        28 kB
        ramkrishna.s.vasudevan
      5. HBASE-18649_master_6.patch
        69 kB
        ramkrishna.s.vasudevan
      6. HBASE-18649_master_6.patch
        69 kB
        ramkrishna.s.vasudevan
      7. HBASE-18649_master_5.patch
        69 kB
        ramkrishna.s.vasudevan
      8. HBASE-18649_master_3.patch
        61 kB
        ramkrishna.s.vasudevan
      9. HBASE-18649_master_2.patch
        40 kB
        ramkrishna.s.vasudevan
      10. HBASE-18649_branch-2.patch
        6 kB
        ramkrishna.s.vasudevan

        Issue Links

          Activity

            People

            • Assignee:
              ram_krish ramkrishna.s.vasudevan
              Reporter:
              ram_krish ramkrishna.s.vasudevan
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: