Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18640

Move mapreduce out of hbase-server into separate hbase-mapreduce module

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0-alpha-3, 2.0.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      - Moves all org.apache.hadoop.hbase.mapreduce.* (except LoadIncrementalHFiles) and org.apache.hadoop.hbase.mapred.* classes from hbase-server module to new hbase-mapreduce module.
      - Also moves following tools from hbase-server module to hbase-mapreduce module: CompactionTool, ExportSnapshot, PerformanceEvaluation, LoadTestTool
      - Very minor breakages in LoadTestTool(LimitedPrivate HBaseInterfaceAudience.TOOLS)
      Show
      - Moves all org.apache.hadoop.hbase.mapreduce.* (except LoadIncrementalHFiles) and org.apache.hadoop.hbase.mapred.* classes from hbase-server module to new hbase-mapreduce module. - Also moves following tools from hbase-server module to hbase-mapreduce module: CompactionTool, ExportSnapshot, PerformanceEvaluation, LoadTestTool - Very minor breakages in LoadTestTool(LimitedPrivate HBaseInterfaceAudience.TOOLS)

      Description

      (Couldn't find another dedicated jira, so creating new one).
      Uploaded patch which is moving ~60 files to the new module. Few notes:

      • The classes remaining in hbase-server are the ones which are intensively coupled with visibility labels/wal/filesystem/hfile. These can not be migrated to new module until corresponding subcomponents are untangled out of hbase-server into their own separate modules.
      • Almost all mapreduce tests uses HBaseTestingUtil, so they can't be moved to hbase-mapreduce module. Given these dependency constraints, one way would be having a separate module for tests:
        hbase-mapreduce <---- hbase-server <------- hbase-mapreduce-tests
        Imo, this makes sense and looks fine.
        The only issue is - yetus' pre-commit. It won't run tests in hbase-mapreduce-tests module if something changed in just hbase-mapreduce. However, yetus' limitation shouldn't warrant against the idea.
        So i'd say that we should go that way, unless there are better suggestions.

        Attachments

        1. HBASE-18640.branch-2.001.patch
          254 kB
          Appy
        2. HBASE-18640.master.008.patch
          254 kB
          Appy
        3. HBASE-18640.master.007.patch
          237 kB
          Appy
        4. HBASE-18640.master.006.patch
          232 kB
          Appy
        5. HBASE-18640.master.005.patch
          295 kB
          Appy
        6. HBASE-18640.master.004.patch
          278 kB
          stack
        7. HBASE-18640.master.003.patch
          239 kB
          stack
        8. HBASE-18640.master.004.patch
          278 kB
          Appy
        9. HBASE-18640.master.003.patch
          239 kB
          Appy
        10. HBASE-18640.master.002.patch
          172 kB
          Appy
        11. HBASE-18640.master.001.patch
          182 kB
          Appy

          Issue Links

            Activity

              People

              • Assignee:
                appy Appy
                Reporter:
                appy Appy
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: