Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5626

TaskLogServlet could not get syslog

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: 1.2.1
    • Fix Version/s: 1.2.1
    • Component/s: None
    • Labels:
    • Environment:

      Linux version 2.6.18-238.9.1.el5
      Java(TM) SE Runtime Environment (build 1.6.0_43-b01)
      hadoop-1.2.1

    • Target Version/s:
    • Release Note:
      modify TaskLogServlet haveTaskLog method

      Description

      When multiply tasks use one jvm and generated logs.
      eg.
      ./attempt_201211220735_0001_m_000000_0:
      log.index
      ./attempt_201211220735_0001_m_000001_0:
      log.index
      ./attempt_201211220735_0001_m_000002_0:
      log.index stderr stdout syslog
      get from http://xxxxxxxx:50060/tasklog?attemptid= attempt_201211220735_0001_m_000000_0
      could get stderr,stdout,but not the others,include syslog.

      see TaskLogServlet.haveTaskLog() method, not check from local && log.index, but check the original path.

      resolve:
      modify TaskLogServlet haveTaskLog method
      private boolean haveTaskLog(TaskAttemptID taskId, boolean isCleanup,
      TaskLog.LogName type) throws IOException {
      File f = TaskLog.getTaskLogFile(taskId, isCleanup, type);
      if (f.exists() && f.canRead())

      { return true; }

      else {
      File indexFile = TaskLog.getIndexFile(taskId, isCleanup);
      if (!indexFile.exists())

      { return false; }

      BufferedReader fis;
      try

      { fis = new BufferedReader(new InputStreamReader( SecureIOUtils.openForRead(indexFile, TaskLog.obtainLogDirOwner(taskId)))); }

      catch (FileNotFoundException ex) {
      LOG.warn("Index file for the log of " + taskId
      + " does not exist.");

      // Assume no task reuse is used and files exist on attemptdir
      StringBuffer input = new StringBuffer();
      input.append(LogFileDetail.LOCATION
      + TaskLog.getAttemptDir(taskId, isCleanup) + "\n");
      for (LogName logName : TaskLog.LOGS_TRACKED_BY_INDEX_FILES)

      { input.append(logName + ":0 -1\n"); }


      fis = new BufferedReader(new StringReader(input.toString()));
      }

      try {
      String str = fis.readLine();
      if (str == null)

      { // thefile doesn't have anything throw new IOException("Index file for the log of " + taskId + "is empty."); }


      String loc = str.substring(str.indexOf(LogFileDetail.LOCATION)
      + LogFileDetail.LOCATION.length());
      File tf = new File(loc, type.toString());
      return tf.exists() && tf.canRead();

      } finally

      { if (fis != null) fis.close(); }


      }

      }

      workaround:
      url add filter=SYSLOG could print syslog also.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              yangjun2@sohu.com yangjun
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 2h
                2h
                Remaining:
                Remaining Estimate - 2h
                2h
                Logged:
                Time Spent - Not Specified
                Not Specified