Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-983

Worker should directly read Intermediate data stored in localhost rather than fetching

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: Data Shuffle
    • Labels:
      None

      Description

      Currently, worker always fetches all intermediate via Fetcher and than store them in local file system even though some intermediate data already are stored in local file system. It is inefficient and causes unnecessary I/O and extra storage occupation. We should improve it.

        Attachments

        1. TAJO-983.140924-Hyunsik.patch
          41 kB
          Hyunsik Choi
        2. TAJO-983.140923.patch
          40 kB
          Mai Hai Thanh
        3. TAJO-983.140922.patch
          41 kB
          Mai Hai Thanh
        4. TAJO-983.140918.patch
          39 kB
          Mai Hai Thanh
        5. TAJO-983.140916.patch
          20 kB
          Hyunsik Choi
        6. TAJO-983.140904.patch
          20 kB
          Mai Hai Thanh
        7. TAJO-983.140902.patch
          20 kB
          Mai Hai Thanh
        8. TAJO-983.140825.1.patch.txt
          19 kB
          Mai Hai Thanh
        9. TAJO-983.140822.patch.txt
          20 kB
          Mai Hai Thanh
        10. TAJO-983.140820.0.patch.txt
          15 kB
          Mai Hai Thanh

          Issue Links

            Activity

              People

              • Assignee:
                mhthanh Mai Hai Thanh
                Reporter:
                hyunsik Hyunsik Choi
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: