Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-2625

CarbonReader is very slow for many files

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      CarbonReader is very slow for many files， including build carbonReader and read data， it's more than 8 minutes when there are 140 files and more than 10 million rows data.

      Reason is BlockletDataMap lists files of folder while loading each segment. 

      Attachments

        Activity

          People

            xubo245 Bo Xu
            xubo245 Bo Xu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 25h 10m
                25h 10m