Uploaded image for project: 'Tajo (Retired)'
  1. Tajo (Retired)
  2. TAJO-9

Change the default intermediate data file format for hash repartitioning

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • 0.8.0
    • Data Shuffle
    • None

    Description

      For easy debugging, the hash repartitioning have used CSV as the default intermediate data format. CSV file format may cause parsing overhead, and it may cause relatively large intermediate data to be transmitted via networks. We need to change it to RawFile or another efficient file format.

      Digging PartitionedStoredExec class is a good starting point for this issue.

      Attachments

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. TAJO-435 Improve intermediate file

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. TAJO-224 Rearrange DataType enumeration and Refactor type systems

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              hyunsik Hyunsik Choi
              hyunsik Hyunsik Choi
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: