Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4233 YARN Timeline Service plugin: ATS v1.5
  3. YARN-4705

ATS 1.5 parse pipeline to consider handling open() events recoverably

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.8.0
    • Fix Version/s: None
    • Component/s: timelineserver
    • Labels:
      None

      Description

      During one of my own timeline test runs, I've been seeing a stack trace warning that the CRC check failed in Filesystem.open() file; something the FS was ignoring.

      Even though its swallowed (and probably not the cause of my test failure), looking at the code in LogInfo.parsePath() that it considers a failure to open a file as unrecoverable.

      on some filesystems, this may not be the case, i.e. if its open for writing it may not be available for reading; checksums maybe a similar issue.

      Perhaps a failure at open() should be viewed as recoverable while the app is still running?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: