Uploaded image for project: 'Apache Gobblin'
  1. Apache Gobblin
  2. GOBBLIN-568

FsDataWriter lacks support for multiple files per partition using record count or block size

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • gobblin-core
    • None

    Description

      Currently, FsDataWriter writer one file per partition which is not optimal when writing files like 
      Parquet where one would want to land an optimal size either based on file size or records per file.

      Attachments

        Activity

          People

            abti Abhishek Tiwari
            tilakpatidar Tilak Patidar
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: