Details

    • Type: Improvement
    • Status: Closed
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.0
    • Component/s: fs/s3
    • Labels:
    • Target Version/s:
    • Tags:
      fs/s3

      Description

      S3AFileSystem generates INFO level logs in open and rename, which are not necessary.

      1. HADOOP-11607.000.patch
        2 kB
        Lei (Eddy) Xu
      2. HADOOP-11607.001.patch
        2 kB
        Lei (Eddy) Xu

        Issue Links

          Activity

          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Change several INFO level log messages to DEBUG.

          No test is added since the change is trivial.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Change several INFO level log messages to DEBUG. No test is added since the change is trivial.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1

          Note that as s3a uses SLF4J for its log API, it can switch to log.info("item {}", value) for terser/more efficient logging logic (we still recommend wrapping debug statements though, just to skip string creation.

          If you switch to that mode, I'd be even happier. But I'm OK with what you've done so far

          Show
          stevel@apache.org Steve Loughran added a comment - +1 Note that as s3a uses SLF4J for its log API, it can switch to log.info("item {}", value) for terser/more efficient logging logic (we still recommend wrapping debug statements though, just to skip string creation. If you switch to that mode, I'd be even happier. But I'm OK with what you've done so far
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Steve Loughran Thanks much for reviewing.

          If you switch to that mode,

          Sure, I'd love to change it. I will post a new patch tomorrow.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Steve Loughran Thanks much for reviewing. If you switch to that mode, Sure, I'd love to change it. I will post a new patch tomorrow.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Updated the patch to use SLF4J formating.

          Steve Loughran Would you mind to take another look?

          Show
          eddyxu Lei (Eddy) Xu added a comment - Updated the patch to use SLF4J formating. Steve Loughran Would you mind to take another look?
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12699514/HADOOP-11607.001.patch
          against trunk revision 2ecea5a.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-aws.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5736//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5736//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12699514/HADOOP-11607.001.patch against trunk revision 2ecea5a. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-tools/hadoop-aws. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5736//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5736//console This message is automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1

          Show
          stevel@apache.org Steve Loughran added a comment - +1
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Steve Loughran Thank you so much for the review and +1.

          Could you help us to commit this as well?

          Show
          eddyxu Lei (Eddy) Xu added a comment - Steve Loughran Thank you so much for the review and +1. Could you help us to commit this as well?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          -sorry, thought you were a committer. Done!

          Show
          stevel@apache.org Steve Loughran added a comment - -sorry, thought you were a committer. Done!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #7167 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7167/)
          HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7167 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7167/ ) HADOOP-11607 . Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Steve Loughran Thanks a lot for committing this. Hopefully I can commit sometime soon

          Show
          eddyxu Lei (Eddy) Xu added a comment - Steve Loughran Thanks a lot for committing this. Hopefully I can commit sometime soon
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/111/)
          HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/111/ ) HADOOP-11607 . Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #845 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/845/)
          HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #845 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/845/ ) HADOOP-11607 . Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2043 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2043/)
          HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2043 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2043/ ) HADOOP-11607 . Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #102 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/102/)
          HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #102 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/102/ ) HADOOP-11607 . Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #112 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/112/)
          HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #112 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/112/ ) HADOOP-11607 . Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2062 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2062/)
          HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2062 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2062/ ) HADOOP-11607 . Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) (stevel: rev aa1c437b6a806de612f030a68984c606c623f1d9) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt

            People

            • Assignee:
              eddyxu Lei (Eddy) Xu
              Reporter:
              eddyxu Lei (Eddy) Xu
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development