Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-2428

Support Flat folder structure in carbon.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.0, 1.4.1
    • Component/s: None
    • Labels:
      None

      Description

      1. Flat folder makes all carbondata files store flat under table path.
      2. It is controlled through table property `flat_folder`. By default it is false.
      3. It cannot be hybrid, so user cannot change the property once table created.
      4. Segment file is created for each loading.And segment file is created under MetaData folder under table path.
      5. Segment number is added as part of carbondata and index files.
      6. All datamap files now create directly under table path with <tablepath>/<dmname>/<segment_number>/<task_name>/dm

      IUD : It supports but list files during IUD may hit performance.
      Compaction: Supports
      Delete Segment : No impact
      Clean files : No impact
      Alter table : No impact
      Pre Agg : Property need to inherited to child, so it also supports flat folder structure.
      Partition : No Impact on this feature as it already has flat folder structure.
      Streaming : Only during handoff it supports flat folder structure. Streaming segment location is no change.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ravi.pesala Ravindra Pesala
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 11h
                  11h