Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Bzip2Factory.isNativeBzip2Loaded is not protected from multiple threads calling it simultaneously. A thread can return false from this method despite logging the fact that was going to return true due to manipulations of the static boolean from another thread calling the same method.

      1. HADOOP-12191.patch
        1 kB
        Brahma Reddy Battula

        Issue Links

          Activity

          Hide
          jlowe Jason Lowe added a comment -

          Thanks for the comments, Samer! Since this is already resolved and does fix the stated problem it would be better to file a separate improvement JIRA with these suggestions rather than try to address them here.

          Show
          jlowe Jason Lowe added a comment - Thanks for the comments, Samer! Since this is already resolved and does fix the stated problem it would be better to file a separate improvement JIRA with these suggestions rather than try to address them here.
          Hide
          SamerAdra Samer Adra added a comment -

          It would perform better if this method only performed the logic the first time it was called, and cached the result for all subsequent calls, because whether or not the Bzip2 library is available should be the same throughout the lifetime of the application – no need to figure it out every time.
          Also, the catch(Throwable) in this method is kind of scary.

          Show
          SamerAdra Samer Adra added a comment - It would perform better if this method only performed the logic the first time it was called, and cached the result for all subsequent calls, because whether or not the Bzip2 library is available should be the same throughout the lifetime of the application – no need to figure it out every time. Also, the catch(Throwable) in this method is kind of scary.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #254 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/254/)
          HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #254 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/254/ ) HADOOP-12191 . Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2202 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2202/)
          HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2202 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2202/ ) HADOOP-12191 . Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #244 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/244/)
          HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #244 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/244/ ) HADOOP-12191 . Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2183 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2183/)
          HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2183 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2183/ ) HADOOP-12191 . Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #986 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/986/)
          HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #986 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/986/ ) HADOOP-12191 . Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #256 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/256/)
          HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #256 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/256/ ) HADOOP-12191 . Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8161 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8161/)
          HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8161 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8161/ ) HADOOP-12191 . Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. (ozawa: rev 4084eaf94347042d9b8cb1e86ba831036621dfdd) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Committed this to trunk, branch-2, and branch-2.7. Thanks Brahma Reddy Battula for your contribution and thanks Jason Lowe for your reporting!

          Show
          ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk, branch-2, and branch-2.7. Thanks Brahma Reddy Battula for your contribution and thanks Jason Lowe for your reporting!
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          +1, committing this shortly.

          Show
          ozawa Tsuyoshi Ozawa added a comment - +1, committing this shortly.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 29s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 3s There were no new javac warning messages.
          +1 javadoc 10m 16s There were no new javadoc warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 6s There were no new checkstyle issues.
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 57s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 43s Tests passed in hadoop-common.
              64m 6s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743760/HADOOP-12191.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fc92d3e
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7164/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7164/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7164/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7164/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 29s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 3s There were no new javac warning messages. +1 javadoc 10m 16s There were no new javadoc warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 6s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 57s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 43s Tests passed in hadoop-common.     64m 6s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743760/HADOOP-12191.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fc92d3e whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7164/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7164/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7164/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7164/console This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Jason Lowe kindly review the attached patch..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Jason Lowe kindly review the attached patch..
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Jason Lowe thanks for reporting ..will Add synchronize and update the patch soon..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Jason Lowe thanks for reporting ..will Add synchronize and update the patch soon..

            People

            • Assignee:
              brahmareddy Brahma Reddy Battula
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development