Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4233 YARN Timeline Service plugin: ATS v1.5
  3. YARN-4705

ATS 1.5 parse pipeline to consider handling open() events recoverably

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 2.8.0
    • None
    • timelineserver
    • None

    Description

      During one of my own timeline test runs, I've been seeing a stack trace warning that the CRC check failed in Filesystem.open() file; something the FS was ignoring.

      Even though its swallowed (and probably not the cause of my test failure), looking at the code in LogInfo.parsePath() that it considers a failure to open a file as unrecoverable.

      on some filesystems, this may not be the case, i.e. if its open for writing it may not be available for reading; checksums maybe a similar issue.

      Perhaps a failure at open() should be viewed as recoverable while the app is still running?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: