Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-1072 Streaming Ingestion Feature
  3. CARBONDATA-1175

Streaming Ingest: Write path data conversion/transformation

    XMLWordPrintableJSON

    Details

      Description

      Streaming Ingest: Write path Data conversion/Transformation

      • input data is a byte stream in catalyst InternalRow format (row major), which needs to be converted to column format
      • column converter and corresponding iterators needs to be created before invoking carbon layer load path
      • various carbon properties needs to be set, for example SORT_SCOPE (to skip sorting), blocket size, skip global dictionary creation etc

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              aniketadnaik Aniket Adnaik
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: