Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3600

Core MapReduce classes don't provide location info

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.8.0
    • 4.10.0
    • None
    • None

    Description

      The core MapReduce classes org.apache.phoenix.mapreduce.PhoenixInputSplit and org.apache.phoenix.mapreduce.PhoenixInputFormat don't provide region size or location information, leaving the execution engine (MR, Spark, etc.) to randomly assign splits to nodes.

      Interestingly, the phoenix-hive module has reimplemented these classes, including the node-aware functionality. We should port a subset of those changes back to the core code so that other engines can make use of them.

      Attachments

        1. PHOENIX-3600.patch
          10 kB
          Josh Mahonin
        2. PHOENIX-3600_v2.patch
          10 kB
          Josh Mahonin

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-4489 HBase Connection leak in Phoenix MR Jobs

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              jmahonin Josh Mahonin
              jmahonin Josh Mahonin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: