Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-2938

HFile support for SparkSQL DataFrame saves

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Currently when saving a DataFrame in Spark it is persisted as upserts. Having an option to do saves natively via HFiles, as the MapReduce loader does, would be a great performance improvement for large bulk loads. The current work around to reduce the load on the regionservers would be to save to csv from Spark then load via the MapReduce loader.

        Attachments

          Activity

            People

            • Assignee:
              kalyanhadoop Kalyan
              Reporter:
              cftarnas Chris Tarnas
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: