Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1855

Avoid scanning for previously written files within Inputs / Outputs

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.7.0
    • None
    • None
    • Reviewed

    Description

      TezTaskOutput has a bunch of methods - getOutputFile, getOutputIndexFile, getSpillIndexFile - which are used within an Output to scan for files written earlier by the same Output. This should be avoided in favour of keeping track of previously written files.

      Attachments

        1. TEZ-1855.1.patch
          15 kB
          Rajesh Balamohan
        2. TEZ-1855.2.patch
          28 kB
          Rajesh Balamohan
        3. TEZ-1855.3.patch
          28 kB
          Rajesh Balamohan

        Activity

          People

            rajesh.balamohan Rajesh Balamohan
            sseth Siddharth Seth
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: