Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6788

[Herriot] Exception exclusion functionality is not working correctly.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Tags:
      herriot

      Description

      Exception exclusion functionality is not working correctly because of that tests are failing by not matching the error count.
      I debugged the issue and found that the problem with shell command which is generating in the getNumberOfMatchesInLogFile function.

      Currently building the shell command in the following way.

      if(list != null){
      for(int i =0; i < list.length; ++i)

      { filePattern.append(" | grep -v " + list[i] ); }

      }
      String[] cmd =
      new String[] {
      "bash",
      "-c",
      "grep -c "
      + pattern + " " + filePattern
      + " | awk -F: '

      {s+=$2} END {print s}'" };

      However, The above commnad won't work correctly because you are counting the exceptions in the file before excluding the known exceptions.
      In this case it gives the mismatch error counts everytime.The shell command should be in the following way to work correctly.

      if (list != null) {
      int index = 0;
      for (String excludeExp : list) { filePattern.append((++index < list.length)? "| grep -v " : "| grep -vc " + list[i] ); }
      }
      String[] cmd =
      new String[] {
      "bash",
      "-c",
      "grep "
      + pattern + " " + filePattern
      + " | awk -F: '{s+=$2}

      END

      {print s}

      '" };

        Attachments

        1. 6788-ydist-security.patch
          2 kB
          Vinay Kumar Thota
        2. HADOOP-6788.patch
          2 kB
          Vinay Kumar Thota
        3. 6788-ydist-security.patch
          2 kB
          Vinay Kumar Thota
        4. HADOOP-6788.patch
          2 kB
          Vinay Kumar Thota

          Issue Links

            Activity

              People

              • Assignee:
                vinaythota Vinay Kumar Thota
                Reporter:
                vinaythota Vinay Kumar Thota
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: