Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24950

scala DateTimeUtilsSuite daysToMillis and millisToDays fails w/java 8 181-b13

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.1.3, 2.2.2, 2.3.1, 2.4.0
    • 2.1.4, 2.2.3, 2.3.2, 2.4.0
    • Build, Tests
    • None

    Description

      during my travails to port the spark builds to run on ubuntu 16.04LTS, i have encountered a strange and apparently java version-specific failure on one specific unit test.

      the failure is here:

      https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.6-ubuntu-test/868/testReport/junit/org.apache.spark.sql.catalyst.util/DateTimeUtilsSuite/daysToMillis_and_millisToDays/

      the java version on this worker is:

      sknapp@ubuntu-testing:~$ java -version
      java version "1.8.0_181"
      Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
      Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

      however, when i run this exact build on the other ubuntu workers, it passes.  they systems are set up (for the most part) identically except for the java version:

      sknapp@amp-jenkins-staging-worker-02:~$ java -version
      java version "1.8.0_171"
      Java(TM) SE Runtime Environment (build 1.8.0_171-b11)
      Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)

      there are some minor kernel and other package differences on these ubuntu workers, but nothing that (in my opinion) would affect this test.  i am willing to help investigate this, however.

      the test also passes on the centos 6.9 workers, which have the following java version installed:

      [sknapp@amp-jenkins-worker-05 ~]$ java -version
      java version "1.8.0_60"
      Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
      Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)my guess is that either:

      sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala

      or

      sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala

      is doing something wrong.  i am not a scala expert by any means, so i'd really like some help in trying to un-block the project to port the builds to ubuntu.

      Attachments

        Issue Links

          Activity

            People

              d80tb7 Chris Martin
              shaneknapp Shane Knapp
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: