Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-3617

loadDataUsingGlobalSort should based on SortColumns Instead Of Whole CarbonRow

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.6.1, 2.0.0
    • Fix Version/s: 2.1.0
    • Component/s: data-load
    • Labels:
      None

      Description

      During loading Data usesing globalsort, the sortby processing is based the whole carbon row, the overhead of gc is huge when there are many columns. Theoretically, the sortby processing can works well just based on the sort columns, which will brings less time overhead and gc overhead.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                XingjunHao Xingjun Hao
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 7h 50m
                  7h 50m