Uploaded image for project: 'Apache Gobblin'
  1. Apache Gobblin
  2. GOBBLIN-568

FsDataWriter lacks support for multiple files per partition using record count or block size

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: gobblin-core
    • Labels:
      None

      Description

      Currently, FsDataWriter writer one file per partition which is not optimal when writing files likeĀ 
      Parquet where one would want to land an optimal size either based on file size or records per file.

        Attachments

          Activity

            People

            • Assignee:
              abti Abhishek Tiwari
              Reporter:
              tilakpatidar Tilak Patidar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: