Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18649

Deprecate KV Usage in MR to move to Cells in 3.0

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0-alpha-2
    • 2.0.0-beta-1, 2.0.0
    • API, mapreduce
    • None
    • Incompatible change, Reviewed
    • Hide
      All the mappers and reducers output type will be now of MapReduceCell type. No more KeyValue type. How ever in branch-2 for compatibility we have allowed the older interfaces/classes that work with KeyValue to stay in the code base but they have been marked as deprecated.
      The following interfaces/classes have been deprecated in branch-2
      Import#KeyValueWritableComparablePartitioner
      Import#KeyValueWritableComparator
      Import#KeyValueWritableComparable
      Import#KeyValueReducer
      Import#KeyValueSortImporter
      Import#KeyValueImporter
      KeyValueSortReducer
      KeyValueSerialization
      WALPlayer#WALKeyValueMapper

      So any existing MR jobs that is using the above public interfaces/classes will continue to work in branch-2 and the expected output value type of those mappers and reducers can continue to be KeyValue type.

      In branch-3 the mappers and reducers output will only expect MapReduceCell as the type and will no longer work with KeyValue type.
      The new public classes/interfaces added for branch-3 and in branch-2 are
      CellSerialization
      CellSortReducer
      Import#CellWritableComparablePartitioner
      Import#CellWritableComparable
      Import#CellWritableComparator
      Import#CellReducer
      Import#CellSortImporter
      Import#CellImporter
      WALPlayer#WALCellMapper
      Show
      All the mappers and reducers output type will be now of MapReduceCell type. No more KeyValue type. How ever in branch-2 for compatibility we have allowed the older interfaces/classes that work with KeyValue to stay in the code base but they have been marked as deprecated. The following interfaces/classes have been deprecated in branch-2 Import#KeyValueWritableComparablePartitioner Import#KeyValueWritableComparator Import#KeyValueWritableComparable Import#KeyValueReducer Import#KeyValueSortImporter Import#KeyValueImporter KeyValueSortReducer KeyValueSerialization WALPlayer#WALKeyValueMapper So any existing MR jobs that is using the above public interfaces/classes will continue to work in branch-2 and the expected output value type of those mappers and reducers can continue to be KeyValue type. In branch-3 the mappers and reducers output will only expect MapReduceCell as the type and will no longer work with KeyValue type. The new public classes/interfaces added for branch-3 and in branch-2 are CellSerialization CellSortReducer Import#CellWritableComparablePartitioner Import#CellWritableComparable Import#CellWritableComparator Import#CellReducer Import#CellSortImporter Import#CellImporter WALPlayer#WALCellMapper

    Attachments

      1. HBASE-18649_master.patch
        28 kB
        ramkrishna.s.vasudevan
      2. HBASE-18649_master_2.patch
        40 kB
        ramkrishna.s.vasudevan
      3. HBASE-18649_branch-2.patch
        6 kB
        ramkrishna.s.vasudevan
      4. HBASE-18649_master_3.patch
        61 kB
        ramkrishna.s.vasudevan
      5. HBASE-18649_master_5.patch
        69 kB
        ramkrishna.s.vasudevan
      6. HBASE-18649_master_6.patch
        69 kB
        ramkrishna.s.vasudevan
      7. HBASE-18649-branch-2.patch
        75 kB
        ramkrishna.s.vasudevan
      8. HBASE-18649-branch-2_1.patch
        75 kB
        ramkrishna.s.vasudevan
      9. HBASE-18649-branch-2_2.patch
        172 kB
        ramkrishna.s.vasudevan
      10. HBASE-18649_master_6.patch
        69 kB
        ramkrishna.s.vasudevan

      Issue Links

        Activity

          People

            ram_krish ramkrishna.s.vasudevan
            ram_krish ramkrishna.s.vasudevan
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: