Hadoop Common
  1. Hadoop Common
  2. HADOOP-10015

UserGroupInformation prints out excessive ERROR warnings

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 2.4.0
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In UserGroupInformation::doAs(), it prints out a log at ERROR level whenever it catches an exception.

      However, it prints benign warnings in the following paradigm:

       try {
              ugi.doAs(new PrivilegedExceptionAction<FileStatus>() {
                @Override
                public FileStatus run() throws Exception {
                  return fs.getFileStatus(nonExist);
                }
              });
            } catch (FileNotFoundException e) {
            }
      

      For example, FileSystem#exists() follows this paradigm. Distcp uses this paradigm too. The exception is expected therefore there should be no ERROR logs printed in the namenode logs.

      Currently, the user quickly finds out that the namenode log is quickly filled by benign ERROR logs when he or she runs distcp in secure set up. This behavior confuses the operators.

      This jira proposes to move the log to DEBUG level.

      1. HADOOP-10015.002.patch
        4 kB
        Haohui Mai
      2. HADOOP-10015.001.patch
        4 kB
        Haohui Mai
      3. HADOOP-10015.000.patch
        1 kB
        Haohui Mai
      4. 10015.v6.patch
        0.9 kB
        Nicolas Liochon
      5. 10015.v5.patch
        0.9 kB
        Nicolas Liochon
      6. 10015.v4.patch
        2 kB
        Nicolas Liochon
      7. 10015.v3.patch
        2 kB
        Nicolas Liochon

        Issue Links

          Activity

          Haohui Mai created issue -
          Haohui Mai made changes -
          Field Original Value New Value
          Attachment HADOOP-10015.000.patch [ 12606608 ]
          Haohui Mai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Haohui Mai made changes -
          Attachment HADOOP-10015.001.patch [ 12606906 ]
          Brandon Li made changes -
          Description In UserGroupInformation::doAs(), it prints out a log at ERROR level whenever it catches an exception.

          However, in some cases IOExceptions are expected -- For example, the upp (e.g., calling exists() in filesystem). As a result, these error logs are benign but they are printed at the ERROR level, which confuses the operators.
          In UserGroupInformation::doAs(), it prints out a log at ERROR level whenever it catches an exception.

          However, it prints benign warnings in the following paradigms:

          {noformat}

          {noformat}

          in some cases IOExceptions are expected by the upper level users (e.g., FileSystem#exists() in filesystem). As a result, these error logs are benign but they are printed at the ERROR level, which confuses the operators.
          Brandon Li made changes -
          Description In UserGroupInformation::doAs(), it prints out a log at ERROR level whenever it catches an exception.

          However, it prints benign warnings in the following paradigms:

          {noformat}

          {noformat}

          in some cases IOExceptions are expected by the upper level users (e.g., FileSystem#exists() in filesystem). As a result, these error logs are benign but they are printed at the ERROR level, which confuses the operators.
          In UserGroupInformation::doAs(), it prints out a log at ERROR level whenever it catches an exception.

          However, it prints benign warnings in the following paradigm:

          {noformat}
           try {
                  ugi.doAs(new PrivilegedExceptionAction<FileStatus>() {
                    @Override
                    public FileStatus run() throws Exception {
                      return fs.getFileStatus(nonExist);
                    }
                  });
                } catch (FileNotFoundException e) {
                }
          {noformat}

          For example, FileSystem#exists() follows this paradigm. Distcp uses this paradigm too. The exception is expected therefore there should be no ERROR logs printed in the namenode logs.

          Currently, the user quickly finds out that the namenode log is quickly filled by _benign_ ERROR logs when he or she runs distcp in secure set up. This behavior confuses the operators.

          This jira proposes to move the log to DEBUG level.
          Haohui Mai made changes -
          Attachment HADOOP-10015.002.patch [ 12607265 ]
          Haohui Mai made changes -
          Link This issue relates to HADOOP-10274 [ HADOOP-10274 ]
          Haohui Mai made changes -
          Link This issue duplicates HADOOP-10274 [ HADOOP-10274 ]
          Haohui Mai made changes -
          Link This issue duplicates HADOOP-10274 [ HADOOP-10274 ]
          Haohui Mai made changes -
          Link This issue is duplicated by HADOOP-10274 [ HADOOP-10274 ]
          Haohui Mai made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Nicolas Liochon made changes -
          Resolution Duplicate [ 3 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Assignee Haohui Mai [ wheat9 ] Nicolas Liochon [ nkeywal ]
          Nicolas Liochon made changes -
          Attachment 10015.v3.patch [ 12634210 ]
          Nicolas Liochon made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Nicolas Liochon made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Nicolas Liochon made changes -
          Attachment 10015.v4.patch [ 12635562 ]
          Nicolas Liochon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Nicolas Liochon made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Nicolas Liochon made changes -
          Attachment 10015.v5.patch [ 12636029 ]
          Nicolas Liochon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Nicolas Liochon made changes -
          Affects Version/s 3.0.0 [ 12320357 ]
          Nicolas Liochon made changes -
          Fix Version/s 3.0.0 [ 12320357 ]
          Fix Version/s 2.4.0 [ 12326144 ]
          Fix Version/s 2.3.0 [ 12325254 ]
          Nicolas Liochon made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Nicolas Liochon made changes -
          Attachment 10015.v6.patch [ 12636253 ]
          Nicolas Liochon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Priority Major [ 3 ] Minor [ 4 ]
          Component/s security [ 12312526 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Fix Version/s 3.0.0 [ 12320357 ]
          Fix Version/s 2.3.0 [ 12325254 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Robert Joseph Evans made changes -
          Link This issue duplicates HDFS-3594 [ HDFS-3594 ]

            People

            • Assignee:
              Nicolas Liochon
              Reporter:
              Haohui Mai
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development