Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 4.3.0
    • 5.0.0b1, 4.3.1
    • action
    • None

    Description

      There are two issues w/ Spark action's argument parsing within SparkMain

      Driver and executor extra classpaths: equals sign used

      When the user specifies --conf spark.executor.extraClassPath=XYZ or --conf spark.driver.extraClassPath=ABC, the option --conf will be added to sparkArgs. Then when the code tries to evaluate spark.executor.extraClassPath=XYZ, it uses special logic and set addToSparkArgs = false. As a result there will be a extra --conf in the sparkArgs eventually.

      For example: --conf spark.executor.extraClassPath=XYZ --conf otherProperty=ABC will become --conf --conf otherProperty=ABC, which will cause spark job submit failure later.

      We might need to remove one prior --conf in sparkArgs if the current evaluated opt is EXECUTOR_CLASSPATH or DRIVER_CLASSPATH.

      User provided files and archives: equals sign used

      For the following workflow XML snippet:

      <spark-opts>--files=${nameNode}/home/share/hive-site.xml --num-executors 4 --executor-memory 7g --driver-memory 7g</spark-opts>
      

      the --files=${nameNode}/home/share/hive-site.xml opt will be placed into sparkArgs in previous Oozie version without any modification, because we don't have special handling for --files opt.

      If the user specifies --files=${nameNode}/home/share/hive-site.xml --num-executor 4, then SparkMain code treats --num-executor as a file path / name. That caused the issue as I described in my previous comment. We might need to change the handling logic for FILES_OPTION and ARCHIVES_OPTION to be the same to DRIVER_CLASSPATH_OPTION.

      Attachments

        1. OOZIE-2923.001.patch
          60 kB
          Andras Piros
        2. OOZIE-2923.002.patch
          60 kB
          Andras Piros
        3. OOZIE-2923.003.patch
          72 kB
          Andras Piros
        4. OOZIE-2923.004.patch
          72 kB
          Andras Piros
        5. OOZIE-2923.005.patch
          72 kB
          Andras Piros
        6. OOZIE-2923.006.patch
          72 kB
          Andras Piros

        Issue Links

          Activity

            hadoopqa Hadoop QA added a comment -

            Testing JIRA OOZIE-2923

            Cleaning local git workspace

            ----------------------------

            +1 PATCH_APPLIES
            +1 CLEAN
            +1 RAW_PATCH_ANALYSIS
            . +1 the patch does not introduce any @author tags
            . +1 the patch does not introduce any tabs
            . +1 the patch does not introduce any trailing spaces
            . +1 the patch does not introduce any line longer than 132
            . +1 the patch does adds/modifies 3 testcase(s)
            +1 RAT
            . +1 the patch does not seem to introduce new RAT warnings
            +1 JAVADOC
            . +1 the patch does not seem to introduce new Javadoc warnings
            . WARNING: the current HEAD has 6 Javadoc warning(s)
            +1 COMPILE
            . +1 HEAD compiles
            . +1 patch compiles
            . +1 the patch does not seem to introduce new javac warnings
            -1 There are [1] new bugs found below threshold in total that must be fixed.
            . +1 There are no new bugs found in [sharelib/hive].
            . +1 There are no new bugs found in [sharelib/hcatalog].
            . +1 There are no new bugs found in [sharelib/sqoop].
            . +1 There are no new bugs found in [sharelib/pig].
            . +1 There are no new bugs found in [sharelib/streaming].
            . -1 There are [1] new bugs found below threshold in [sharelib/spark] that must be fixed.
            . You can find the FindBugs diff here (look for the red and orange ones): sharelib/spark/findbugs-new.html
            . The most important FindBugs errors are:
            . Dereferenced at SparkMain.java:[line 165]: Possible null pointer dereference in org.apache.oozie.action.hadoop.SparkMain.getMatchingFile(Pattern) due to return value of called method
            . Known null at SparkMain.java:[line 165]
            . +1 There are no new bugs found in [sharelib/oozie].
            . +1 There are no new bugs found in [sharelib/distcp].
            . +1 There are no new bugs found in [sharelib/hive2].
            . +1 There are no new bugs found in [server].
            . +1 There are no new bugs found in [core].
            . +1 There are no new bugs found in [tools].
            . +1 There are no new bugs found in [docs].
            . +1 There are no new bugs found in [client].
            . +1 There are no new bugs found in [examples].
            +1 BACKWARDS_COMPATIBILITY
            . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
            . +1 the patch does not modify JPA files
            +1 TESTS
            . Tests run: 1961
            . Tests rerun: 39
            . Tests failed at first run: org.apache.oozie.action.hadoop.TestLauncherAM,org.apache.oozie.action.hadoop.TestHdfsOperations,org.apache.oozie.event.TestEventGeneration,org.apache.oozie.command.coord.TestAbandonedCoordChecker,
            +1 DISTRO
            . +1 distro tarball builds with the patch

            ----------------------------
            -1 Overall result, please check the reported -1(s)

            . There is at least one warning, please check

            The full output of the test-patch run is available at

            . https://builds.apache.org/job/oozie-trunk-precommit-build/3839/

            hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2923 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 3 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings . WARNING : the current HEAD has 6 Javadoc warning(s) +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings -1 There are [1] new bugs found below threshold in total that must be fixed. . +1 There are no new bugs found in [sharelib/hive] . . +1 There are no new bugs found in [sharelib/hcatalog] . . +1 There are no new bugs found in [sharelib/sqoop] . . +1 There are no new bugs found in [sharelib/pig] . . +1 There are no new bugs found in [sharelib/streaming] . . -1 There are [1] new bugs found below threshold in [sharelib/spark] that must be fixed. . You can find the FindBugs diff here (look for the red and orange ones): sharelib/spark/findbugs-new.html . The most important FindBugs errors are: . Dereferenced at SparkMain.java: [line 165] : Possible null pointer dereference in org.apache.oozie.action.hadoop.SparkMain.getMatchingFile(Pattern) due to return value of called method . Known null at SparkMain.java: [line 165] . +1 There are no new bugs found in [sharelib/oozie] . . +1 There are no new bugs found in [sharelib/distcp] . . +1 There are no new bugs found in [sharelib/hive2] . . +1 There are no new bugs found in [server] . . +1 There are no new bugs found in [core] . . +1 There are no new bugs found in [tools] . . +1 There are no new bugs found in [docs] . . +1 There are no new bugs found in [client] . . +1 There are no new bugs found in [examples] . +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 1961 . Tests rerun: 39 . Tests failed at first run: org.apache.oozie.action.hadoop.TestLauncherAM,org.apache.oozie.action.hadoop.TestHdfsOperations,org.apache.oozie.event.TestEventGeneration,org.apache.oozie.command.coord.TestAbandonedCoordChecker, +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) . There is at least one warning, please check The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/3839/
            andras.piros Andras Piros added a comment -

            Addressing the only FINDBUGS_DIFF error.

            andras.piros Andras Piros added a comment - Addressing the only FINDBUGS_DIFF error.
            hadoopqa Hadoop QA added a comment -

            Testing JIRA OOZIE-2923

            Cleaning local git workspace

            ----------------------------

            +1 PATCH_APPLIES
            +1 CLEAN
            +1 RAW_PATCH_ANALYSIS
            . +1 the patch does not introduce any @author tags
            . +1 the patch does not introduce any tabs
            . +1 the patch does not introduce any trailing spaces
            . +1 the patch does not introduce any line longer than 132
            . +1 the patch does adds/modifies 3 testcase(s)
            +1 RAT
            . +1 the patch does not seem to introduce new RAT warnings
            +1 JAVADOC
            . +1 the patch does not seem to introduce new Javadoc warnings
            . WARNING: the current HEAD has 6 Javadoc warning(s)
            +1 COMPILE
            . +1 HEAD compiles
            . +1 patch compiles
            . +1 the patch does not seem to introduce new javac warnings
            +1 There are no new bugs found in total.
            . +1 There are no new bugs found in [server].
            . +1 There are no new bugs found in [client].
            . +1 There are no new bugs found in [docs].
            . +1 There are no new bugs found in [sharelib/hive].
            . +1 There are no new bugs found in [sharelib/spark].
            . +1 There are no new bugs found in [sharelib/hcatalog].
            . +1 There are no new bugs found in [sharelib/hive2].
            . +1 There are no new bugs found in [sharelib/streaming].
            . +1 There are no new bugs found in [sharelib/pig].
            . +1 There are no new bugs found in [sharelib/sqoop].
            . +1 There are no new bugs found in [sharelib/distcp].
            . +1 There are no new bugs found in [sharelib/oozie].
            . +1 There are no new bugs found in [core].
            . +1 There are no new bugs found in [tools].
            . +1 There are no new bugs found in [examples].
            +1 BACKWARDS_COMPATIBILITY
            . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
            . +1 the patch does not modify JPA files
            -1 TESTS
            . Tests run: 1961
            . Tests failed: 1
            . Tests errors: 2

            . The patch failed the following testcases:

            . testNofindLogs(org.apache.oozie.util.TestTimestampedMessageParser)

            . Tests failing with errors:
            . testProcessRemainingLog(org.apache.oozie.util.TestTimestampedMessageParser)
            . testProcessRemainingCoordinatorLogForActions(org.apache.oozie.util.TestTimestampedMessageParser)

            +1 DISTRO
            . +1 distro tarball builds with the patch

            ----------------------------
            -1 Overall result, please check the reported -1(s)

            . There is at least one warning, please check

            The full output of the test-patch run is available at

            . https://builds.apache.org/job/oozie-trunk-precommit-build/3841/

            hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2923 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 3 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings . WARNING : the current HEAD has 6 Javadoc warning(s) +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. . +1 There are no new bugs found in [server] . . +1 There are no new bugs found in [client] . . +1 There are no new bugs found in [docs] . . +1 There are no new bugs found in [sharelib/hive] . . +1 There are no new bugs found in [sharelib/spark] . . +1 There are no new bugs found in [sharelib/hcatalog] . . +1 There are no new bugs found in [sharelib/hive2] . . +1 There are no new bugs found in [sharelib/streaming] . . +1 There are no new bugs found in [sharelib/pig] . . +1 There are no new bugs found in [sharelib/sqoop] . . +1 There are no new bugs found in [sharelib/distcp] . . +1 There are no new bugs found in [sharelib/oozie] . . +1 There are no new bugs found in [core] . . +1 There are no new bugs found in [tools] . . +1 There are no new bugs found in [examples] . +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 1961 . Tests failed: 1 . Tests errors: 2 . The patch failed the following testcases: . testNofindLogs(org.apache.oozie.util.TestTimestampedMessageParser) . Tests failing with errors: . testProcessRemainingLog(org.apache.oozie.util.TestTimestampedMessageParser) . testProcessRemainingCoordinatorLogForActions(org.apache.oozie.util.TestTimestampedMessageParser) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) . There is at least one warning, please check The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/3841/
            andras.piros Andras Piros added a comment -

            The tests failings are unrelated, these relate to OOZIE-2815.

            andras.piros Andras Piros added a comment - The tests failings are unrelated, these relate to OOZIE-2815 .
            gezapeti GĂ©zapeti added a comment -

            Thanks for the contribution!
            I like the idea of extracting things from SparkMain to make things more testable. Left some comments on RB

            gezapeti GĂ©zapeti added a comment - Thanks for the contribution! I like the idea of extracting things from SparkMain to make things more testable. Left some comments on RB
            andras.piros Andras Piros added a comment -

            Addressing review comments.

            andras.piros Andras Piros added a comment - Addressing review comments.
            hadoopqa Hadoop QA added a comment -

            Testing JIRA OOZIE-2923

            Cleaning local git workspace

            ----------------------------

            +1 PATCH_APPLIES
            +1 CLEAN
            +1 RAW_PATCH_ANALYSIS
            . +1 the patch does not introduce any @author tags
            . +1 the patch does not introduce any tabs
            . +1 the patch does not introduce any trailing spaces
            . +1 the patch does not introduce any line longer than 132
            . +1 the patch does adds/modifies 4 testcase(s)
            +1 RAT
            . +1 the patch does not seem to introduce new RAT warnings
            +1 JAVADOC
            . +1 the patch does not seem to introduce new Javadoc warnings
            . WARNING: the current HEAD has 6 Javadoc warning(s)
            +1 COMPILE
            . +1 HEAD compiles
            . +1 patch compiles
            . +1 the patch does not seem to introduce new javac warnings
            -1 There are [1] new bugs found below threshold in total that must be fixed.
            . +1 There are no new bugs found in [server].
            . +1 There are no new bugs found in [client].
            . +1 There are no new bugs found in [docs].
            . +1 There are no new bugs found in [sharelib/hive].
            . -1 There are [1] new bugs found below threshold in [sharelib/spark] that must be fixed.
            . You can find the FindBugs diff here (look for the red and orange ones): sharelib/spark/findbugs-new.html
            . The most important FindBugs errors are:
            . At HadoopUriFinder.java:[line 34]: org.apache.oozie.action.hadoop.HadoopUriFinder.getJarVersion(File) may fail to close stream
            . +1 There are no new bugs found in [sharelib/hcatalog].
            . +1 There are no new bugs found in [sharelib/hive2].
            . +1 There are no new bugs found in [sharelib/streaming].
            . +1 There are no new bugs found in [sharelib/pig].
            . +1 There are no new bugs found in [sharelib/sqoop].
            . +1 There are no new bugs found in [sharelib/distcp].
            . +1 There are no new bugs found in [sharelib/oozie].
            . +1 There are no new bugs found in [core].
            . +1 There are no new bugs found in [tools].
            . +1 There are no new bugs found in [examples].
            +1 BACKWARDS_COMPATIBILITY
            . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
            . +1 the patch does not modify JPA files
            -1 TESTS
            . Tests run: 1963
            . Tests failed: 1
            . Tests errors: 2

            . The patch failed the following testcases:

            . testNofindLogs(org.apache.oozie.util.TestTimestampedMessageParser)

            . Tests failing with errors:
            . testProcessRemainingLog(org.apache.oozie.util.TestTimestampedMessageParser)
            . testProcessRemainingCoordinatorLogForActions(org.apache.oozie.util.TestTimestampedMessageParser)

            +1 DISTRO
            . +1 distro tarball builds with the patch

            ----------------------------
            -1 Overall result, please check the reported -1(s)

            . There is at least one warning, please check

            The full output of the test-patch run is available at

            . https://builds.apache.org/job/oozie-trunk-precommit-build/3844/

            hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2923 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 4 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings . WARNING : the current HEAD has 6 Javadoc warning(s) +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings -1 There are [1] new bugs found below threshold in total that must be fixed. . +1 There are no new bugs found in [server] . . +1 There are no new bugs found in [client] . . +1 There are no new bugs found in [docs] . . +1 There are no new bugs found in [sharelib/hive] . . -1 There are [1] new bugs found below threshold in [sharelib/spark] that must be fixed. . You can find the FindBugs diff here (look for the red and orange ones): sharelib/spark/findbugs-new.html . The most important FindBugs errors are: . At HadoopUriFinder.java: [line 34] : org.apache.oozie.action.hadoop.HadoopUriFinder.getJarVersion(File) may fail to close stream . +1 There are no new bugs found in [sharelib/hcatalog] . . +1 There are no new bugs found in [sharelib/hive2] . . +1 There are no new bugs found in [sharelib/streaming] . . +1 There are no new bugs found in [sharelib/pig] . . +1 There are no new bugs found in [sharelib/sqoop] . . +1 There are no new bugs found in [sharelib/distcp] . . +1 There are no new bugs found in [sharelib/oozie] . . +1 There are no new bugs found in [core] . . +1 There are no new bugs found in [tools] . . +1 There are no new bugs found in [examples] . +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 1963 . Tests failed: 1 . Tests errors: 2 . The patch failed the following testcases: . testNofindLogs(org.apache.oozie.util.TestTimestampedMessageParser) . Tests failing with errors: . testProcessRemainingLog(org.apache.oozie.util.TestTimestampedMessageParser) . testProcessRemainingCoordinatorLogForActions(org.apache.oozie.util.TestTimestampedMessageParser) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) . There is at least one warning, please check The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/3844/
            andras.piros Andras Piros added a comment -

            Addressing the only FINDBUGS_DIFF issue.

            andras.piros Andras Piros added a comment - Addressing the only FINDBUGS_DIFF issue.
            hadoopqa Hadoop QA added a comment -

            Testing JIRA OOZIE-2923

            Cleaning local git workspace

            ----------------------------

            +1 PATCH_APPLIES
            +1 CLEAN
            +1 RAW_PATCH_ANALYSIS
            . +1 the patch does not introduce any @author tags
            . +1 the patch does not introduce any tabs
            . +1 the patch does not introduce any trailing spaces
            . +1 the patch does not introduce any line longer than 132
            . +1 the patch does adds/modifies 4 testcase(s)
            +1 RAT
            . +1 the patch does not seem to introduce new RAT warnings
            +1 JAVADOC
            . +1 the patch does not seem to introduce new Javadoc warnings
            . WARNING: the current HEAD has 6 Javadoc warning(s)
            +1 COMPILE
            . +1 HEAD compiles
            . +1 patch compiles
            . +1 the patch does not seem to introduce new javac warnings
            +1 There are no new bugs found in total.
            . +1 There are no new bugs found in [server].
            . +1 There are no new bugs found in [client].
            . +1 There are no new bugs found in [core].
            . +1 There are no new bugs found in [docs].
            . +1 There are no new bugs found in [tools].
            . +1 There are no new bugs found in [examples].
            . +1 There are no new bugs found in [sharelib/streaming].
            . +1 There are no new bugs found in [sharelib/sqoop].
            . +1 There are no new bugs found in [sharelib/distcp].
            . +1 There are no new bugs found in [sharelib/oozie].
            . +1 There are no new bugs found in [sharelib/hcatalog].
            . +1 There are no new bugs found in [sharelib/hive].
            . +1 There are no new bugs found in [sharelib/hive2].
            . +1 There are no new bugs found in [sharelib/pig].
            . +1 There are no new bugs found in [sharelib/spark].
            +1 BACKWARDS_COMPATIBILITY
            . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
            . +1 the patch does not modify JPA files
            +1 TESTS
            . Tests run: 1963
            . Tests rerun: 79
            . Tests failed at first run: org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.jms.TestJMSJobEventListener,org.apache.oozie.action.hadoop.TestLauncherAM,
            +1 DISTRO
            . +1 distro tarball builds with the patch

            ----------------------------
            +1 Overall result, good!, no -1s

            . There is at least one warning, please check

            The full output of the test-patch run is available at

            . https://builds.apache.org/job/oozie-trunk-precommit-build/3848/

            hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2923 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 4 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings . WARNING : the current HEAD has 6 Javadoc warning(s) +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. . +1 There are no new bugs found in [server] . . +1 There are no new bugs found in [client] . . +1 There are no new bugs found in [core] . . +1 There are no new bugs found in [docs] . . +1 There are no new bugs found in [tools] . . +1 There are no new bugs found in [examples] . . +1 There are no new bugs found in [sharelib/streaming] . . +1 There are no new bugs found in [sharelib/sqoop] . . +1 There are no new bugs found in [sharelib/distcp] . . +1 There are no new bugs found in [sharelib/oozie] . . +1 There are no new bugs found in [sharelib/hcatalog] . . +1 There are no new bugs found in [sharelib/hive] . . +1 There are no new bugs found in [sharelib/hive2] . . +1 There are no new bugs found in [sharelib/pig] . . +1 There are no new bugs found in [sharelib/spark] . +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 1963 . Tests rerun: 79 . Tests failed at first run: org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.jms.TestJMSJobEventListener,org.apache.oozie.action.hadoop.TestLauncherAM, +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- +1 Overall result, good!, no -1s . There is at least one warning, please check The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/3848/
            andras.piros Andras Piros added a comment -

            Addressing RB comment.

            andras.piros Andras Piros added a comment - Addressing RB comment.
            hadoopqa Hadoop QA added a comment -

            Testing JIRA OOZIE-2923

            Cleaning local git workspace

            ----------------------------

            +1 PATCH_APPLIES
            +1 CLEAN
            +1 RAW_PATCH_ANALYSIS
            . +1 the patch does not introduce any @author tags
            . +1 the patch does not introduce any tabs
            . +1 the patch does not introduce any trailing spaces
            . +1 the patch does not introduce any line longer than 132
            . +1 the patch does adds/modifies 4 testcase(s)
            +1 RAT
            . +1 the patch does not seem to introduce new RAT warnings
            +1 JAVADOC
            . +1 the patch does not seem to introduce new Javadoc warnings
            . WARNING: the current HEAD has 6 Javadoc warning(s)
            +1 COMPILE
            . +1 HEAD compiles
            . +1 patch compiles
            . +1 the patch does not seem to introduce new javac warnings
            +1 There are no new bugs found in total.
            . +1 There are no new bugs found in [server].
            . +1 There are no new bugs found in [client].
            . +1 There are no new bugs found in [core].
            . +1 There are no new bugs found in [docs].
            . +1 There are no new bugs found in [tools].
            . +1 There are no new bugs found in [examples].
            . +1 There are no new bugs found in [sharelib/streaming].
            . +1 There are no new bugs found in [sharelib/sqoop].
            . +1 There are no new bugs found in [sharelib/distcp].
            . +1 There are no new bugs found in [sharelib/oozie].
            . +1 There are no new bugs found in [sharelib/hcatalog].
            . +1 There are no new bugs found in [sharelib/hive].
            . +1 There are no new bugs found in [sharelib/hive2].
            . +1 There are no new bugs found in [sharelib/pig].
            . +1 There are no new bugs found in [sharelib/spark].
            +1 BACKWARDS_COMPATIBILITY
            . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
            . +1 the patch does not modify JPA files
            +1 TESTS
            . Tests run: 1963
            . Tests rerun: 79
            . Tests failed at first run: org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.jms.TestJMSJobEventListener,org.apache.oozie.action.hadoop.TestLauncherAM,
            +1 DISTRO
            . +1 distro tarball builds with the patch

            ----------------------------
            +1 Overall result, good!, no -1s

            . There is at least one warning, please check

            The full output of the test-patch run is available at

            . https://builds.apache.org/job/oozie-trunk-precommit-build/3862/

            hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2923 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 4 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings . WARNING : the current HEAD has 6 Javadoc warning(s) +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. . +1 There are no new bugs found in [server] . . +1 There are no new bugs found in [client] . . +1 There are no new bugs found in [core] . . +1 There are no new bugs found in [docs] . . +1 There are no new bugs found in [tools] . . +1 There are no new bugs found in [examples] . . +1 There are no new bugs found in [sharelib/streaming] . . +1 There are no new bugs found in [sharelib/sqoop] . . +1 There are no new bugs found in [sharelib/distcp] . . +1 There are no new bugs found in [sharelib/oozie] . . +1 There are no new bugs found in [sharelib/hcatalog] . . +1 There are no new bugs found in [sharelib/hive] . . +1 There are no new bugs found in [sharelib/hive2] . . +1 There are no new bugs found in [sharelib/pig] . . +1 There are no new bugs found in [sharelib/spark] . +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 1963 . Tests rerun: 79 . Tests failed at first run: org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.jms.TestJMSJobEventListener,org.apache.oozie.action.hadoop.TestLauncherAM, +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- +1 Overall result, good!, no -1s . There is at least one warning, please check The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/3862/
            gezapeti GĂ©zapeti added a comment -

            Thanks andras.piros!
            Some of the tests are failing for me because of some differences:

            --files, hive-site.xml,spark-log4j.properties,
            --files, spark-log4j.properties,hive-site.xml,

            I think it may be a HashMap ordering thing that changes in Java8. Please don't depend on the order or fix the order in the Extractor.

            gezapeti GĂ©zapeti added a comment - Thanks andras.piros ! Some of the tests are failing for me because of some differences: --files, hive-site.xml,spark-log4j.properties, --files, spark-log4j.properties,hive-site.xml, I think it may be a HashMap ordering thing that changes in Java8. Please don't depend on the order or fix the order in the Extractor.
            andras.piros Andras Piros added a comment -

            Addressing fixed order of parameters using LinkedHashMap for JDK7 and JDK8.

            andras.piros Andras Piros added a comment - Addressing fixed order of parameters using LinkedHashMap for JDK7 and JDK8.
            hadoopqa Hadoop QA added a comment -

            Testing JIRA OOZIE-2923

            Cleaning local git workspace

            ----------------------------

            +1 PATCH_APPLIES
            +1 CLEAN
            +1 RAW_PATCH_ANALYSIS
            . +1 the patch does not introduce any @author tags
            . +1 the patch does not introduce any tabs
            . +1 the patch does not introduce any trailing spaces
            . +1 the patch does not introduce any line longer than 132
            . +1 the patch does adds/modifies 4 testcase(s)
            +1 RAT
            . +1 the patch does not seem to introduce new RAT warnings
            +1 JAVADOC
            . +1 the patch does not seem to introduce new Javadoc warnings
            . WARNING: the current HEAD has 6 Javadoc warning(s)
            +1 COMPILE
            . +1 HEAD compiles
            . +1 patch compiles
            . +1 the patch does not seem to introduce new javac warnings
            +1 There are no new bugs found in total.
            . +1 There are no new bugs found in [server].
            . +1 There are no new bugs found in [client].
            . +1 There are no new bugs found in [core].
            . +1 There are no new bugs found in [docs].
            . +1 There are no new bugs found in [tools].
            . +1 There are no new bugs found in [examples].
            . +1 There are no new bugs found in [sharelib/streaming].
            . +1 There are no new bugs found in [sharelib/sqoop].
            . +1 There are no new bugs found in [sharelib/distcp].
            . +1 There are no new bugs found in [sharelib/oozie].
            . +1 There are no new bugs found in [sharelib/hcatalog].
            . +1 There are no new bugs found in [sharelib/hive].
            . +1 There are no new bugs found in [sharelib/hive2].
            . +1 There are no new bugs found in [sharelib/pig].
            . +1 There are no new bugs found in [sharelib/spark].
            +1 BACKWARDS_COMPATIBILITY
            . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
            . +1 the patch does not modify JPA files
            +1 TESTS
            . Tests run: 1963
            . Tests rerun: 64
            . Tests failed at first run: org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.action.hadoop.TestLauncherAM,
            +1 DISTRO
            . +1 distro tarball builds with the patch

            ----------------------------
            +1 Overall result, good!, no -1s

            . There is at least one warning, please check

            The full output of the test-patch run is available at

            . https://builds.apache.org/job/oozie-trunk-precommit-build/3866/

            hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2923 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 4 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings . WARNING : the current HEAD has 6 Javadoc warning(s) +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. . +1 There are no new bugs found in [server] . . +1 There are no new bugs found in [client] . . +1 There are no new bugs found in [core] . . +1 There are no new bugs found in [docs] . . +1 There are no new bugs found in [tools] . . +1 There are no new bugs found in [examples] . . +1 There are no new bugs found in [sharelib/streaming] . . +1 There are no new bugs found in [sharelib/sqoop] . . +1 There are no new bugs found in [sharelib/distcp] . . +1 There are no new bugs found in [sharelib/oozie] . . +1 There are no new bugs found in [sharelib/hcatalog] . . +1 There are no new bugs found in [sharelib/hive] . . +1 There are no new bugs found in [sharelib/hive2] . . +1 There are no new bugs found in [sharelib/pig] . . +1 There are no new bugs found in [sharelib/spark] . +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 1963 . Tests rerun: 64 . Tests failed at first run: org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.action.hadoop.TestLauncherAM, +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- +1 Overall result, good!, no -1s . There is at least one warning, please check The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/3866/
            gezapeti GĂ©zapeti added a comment -

            +1
            committed to master!
            Thanks for the contribution!

            gezapeti GĂ©zapeti added a comment - +1 committed to master! Thanks for the contribution!
            satishsaley Satish Saley added a comment -

            Cherry-picked from master to branch-4.3.

            satishsaley Satish Saley added a comment - Cherry-picked from master to branch-4.3.

            Closing issue; Oozie 5.0.0-beta1 is released

            asasvari Attila Sasvári added a comment - Closing issue; Oozie 5.0.0-beta1 is released

            People

              andras.piros Andras Piros
              andras.piros Andras Piros
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: