Uploaded image for project: 'Tajo (Retired)'
  1. Tajo (Retired)
  2. TAJO-983

Worker should directly read Intermediate data stored in localhost rather than fetching

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0
    • Data Shuffle
    • None

    Description

      Currently, worker always fetches all intermediate via Fetcher and than store them in local file system even though some intermediate data already are stored in local file system. It is inefficient and causes unnecessary I/O and extra storage occupation. We should improve it.

      Attachments

        1. TAJO-983.140820.0.patch.txt
          15 kB
          Mai Hai Thanh
        2. TAJO-983.140822.patch.txt
          20 kB
          Mai Hai Thanh
        3. TAJO-983.140825.1.patch.txt
          19 kB
          Mai Hai Thanh
        4. TAJO-983.140902.patch
          20 kB
          Mai Hai Thanh
        5. TAJO-983.140904.patch
          20 kB
          Mai Hai Thanh
        6. TAJO-983.140916.patch
          20 kB
          Hyunsik Choi
        7. TAJO-983.140918.patch
          39 kB
          Mai Hai Thanh
        8. TAJO-983.140922.patch
          41 kB
          Mai Hai Thanh
        9. TAJO-983.140923.patch
          40 kB
          Mai Hai Thanh
        10. TAJO-983.140924-Hyunsik.patch
          41 kB
          Hyunsik Choi

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. TAJO-222 Eliminate local data transmission during repartition phases

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              mhthanh Mai Hai Thanh
              hyunsik Hyunsik Choi
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: