Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3600

Core MapReduce classes don't provide location info

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.8.0
    • Fix Version/s: 4.10.0
    • Labels:
      None

      Description

      The core MapReduce classes org.apache.phoenix.mapreduce.PhoenixInputSplit and org.apache.phoenix.mapreduce.PhoenixInputFormat don't provide region size or location information, leaving the execution engine (MR, Spark, etc.) to randomly assign splits to nodes.

      Interestingly, the phoenix-hive module has reimplemented these classes, including the node-aware functionality. We should port a subset of those changes back to the core code so that other engines can make use of them.

        Attachments

        1. PHOENIX-3600_v2.patch
          10 kB
          Josh Mahonin
        2. PHOENIX-3600.patch
          10 kB
          Josh Mahonin

          Issue Links

            Activity

              People

              • Assignee:
                jmahonin Josh Mahonin
                Reporter:
                jmahonin Josh Mahonin
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: