Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21093

Multiple gapply execution occasionally failed in SparkR

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.1.1, 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: SparkR
    • Labels:
      None
    • Environment:

      CentOS 7.2.1511 / R 3.4.0, CentOS 7.2.1511 / R 3.3.3

    • Target Version/s:

      Description

      On Centos 7.2.1511 with R 3.4.0/3.3.0, multiple execution of gapply looks failed as below:

       Welcome to
          ____              __
         / __/__  ___ _____/ /__
        _\ \/ _ \/ _ `/ __/  '_/
       /___/ .__/\_,_/_/ /_/\_\   version  2.3.0-SNAPSHOT
          /_/
      
      
       SparkSession available as 'spark'.
      > df <- createDataFrame(list(list(1L, 1, "1", 0.1)), c("a", "b", "c", "d"))
      > collect(gapply(df, "a", function(key, x) { x }, schema(df)))
      17/06/14 18:21:01 WARN Utils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.
        a b c   d
      1 1 1 1 0.1
      > collect(gapply(df, "a", function(key, x) { x }, schema(df)))
        a b c   d
      1 1 1 1 0.1
      > collect(gapply(df, "a", function(key, x) { x }, schema(df)))
        a b c   d
      1 1 1 1 0.1
      > collect(gapply(df, "a", function(key, x) { x }, schema(df)))
        a b c   d
      1 1 1 1 0.1
      > collect(gapply(df, "a", function(key, x) { x }, schema(df)))
        a b c   d
      1 1 1 1 0.1
      > collect(gapply(df, "a", function(key, x) { x }, schema(df)))
        a b c   d
      1 1 1 1 0.1
      > collect(gapply(df, "a", function(key, x) { x }, schema(df)))
      
      Error in handleErrors(returnStatus, conn) :
        org.apache.spark.SparkException: Job aborted due to stage failure: Task 98 in stage 14.0 failed 1 times, most recent failure: Lost task 98.0 in stage 14.0 (TID 1305, localhost, executor driver): org.apache.spark.SparkException: R computation failed with
      
              at org.apache.spark.api.r.RRunner.compute(RRunner.scala:108)
              at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$13.apply(objects.scala:432)
              at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$13.apply(objects.scala:414)
              at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
              at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
              at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
              at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
              at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
              at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
              at org.a
      
      ...
      
      *** buffer overflow detected ***: /usr/lib64/R/bin/exec/R terminated
      ======= Backtrace: =========
      /lib64/libc.so.6(__fortify_fail+0x37)[0x7fe699b3f597]
      /lib64/libc.so.6(+0x10c750)[0x7fe699b3d750]
      /lib64/libc.so.6(+0x10e507)[0x7fe699b3f507]
      /usr/lib64/R/modules//internet.so(+0x6015)[0x7fe689bb7015]
      /usr/lib64/R/modules//internet.so(+0xe81e)[0x7fe689bbf81e]
      /usr/lib64/R/lib/libR.so(+0xbd1b6)[0x7fe69c54a1b6]
      /usr/lib64/R/lib/libR.so(+0x1104d0)[0x7fe69c59d4d0]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x354)[0x7fe69c5ad2f4]
      /usr/lib64/R/lib/libR.so(+0x123f8e)[0x7fe69c5b0f8e]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x589)[0x7fe69c5ad529]
      /usr/lib64/R/lib/libR.so(+0x1254ce)[0x7fe69c5b24ce]
      /usr/lib64/R/lib/libR.so(+0x1104d0)[0x7fe69c59d4d0]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(+0x119101)[0x7fe69c5a6101]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x120a7e)[0x7fe69c5ada7e]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x817)[0x7fe69c5ad7b7]
      /usr/lib64/R/lib/libR.so(+0x1256d1)[0x7fe69c5b26d1]
      /usr/lib64/R/lib/libR.so(+0x1552e9)[0x7fe69c5e22e9]
      /usr/lib64/R/lib/libR.so(+0x11062a)[0x7fe69c59d62a]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(+0x119101)[0x7fe69c5a6101]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(+0x119101)[0x7fe69c5a6101]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x120a7e)[0x7fe69c5ada7e]
      /usr/lib64/R/lib/libR.so(+0x11ee91)[0x7fe69c5abe91]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x120a7e)[0x7fe69c5ada7e]
      /usr/lib64/R/lib/libR.so(+0x11ee91)[0x7fe69c5abe91]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x120a7e)[0x7fe69c5ada7e]
      /usr/lib64/R/lib/libR.so(+0x11ee91)[0x7fe69c5abe91]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x120a7e)[0x7fe69c5ada7e]
      /usr/lib64/R/lib/libR.so(+0x11ee91)[0x7fe69c5abe91]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x120a7e)[0x7fe69c5ada7e]
      /usr/lib64/R/lib/libR.so(+0x11ee91)[0x7fe69c5abe91]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(+0x119101)[0x7fe69c5a6101]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(+0x119101)[0x7fe69c5a6101]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(+0x119101)[0x7fe69c5a6101]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(+0x119101)[0x7fe69c5a6101]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x198)[0x7fe69c5ad138]
      /usr/lib64/R/lib/libR.so(+0x1221af)[0x7fe69c5af1af]
      /usr/lib64/R/lib/libR.so(+0x119101)[0x7fe69c5a6101]
      /usr/lib64/R/lib/libR.so(+0x12649e)[0x7fe69c5b349e]
      /usr/lib64/R/lib/libR.so(+0x12744b)[0x7fe69c5b444b]
      /usr/lib64/R/lib/libR.so(Rf_eval+0x589)[0x7fe69c5ad529]
      /usr/lib64/R/lib/libR.so(Rf_ReplIteration+0x232)[0x7fe69c5d6e12]
      ======= Memory map: ========
      00400000-00401000 r-xp 00000000 08:04 55839004975                        /usr/lib64/R/bin/exec/R
      00600000-00601000 r--p 00000000 08:04 55839004975                        /usr/lib64/R/bin/exec/R
      00601000-00602000 rw-p 00001000 08:04 55839004975                        /usr/lib64/R/bin/exec/R
      01c55000-047ce000 rw-p 00000000 00:00 0                                  [heap]
      047ce000-04852000 rw-p 00000000 00:00 0                                  [heap]
      7fe685e06000-7fe685e0b000 r-xp 00000000 08:04 2164504660                 /usr/lib64/R/library/parallel/libs/parallel.so
      7fe685e0b000-7fe68600a000 ---p 00005000 08:04 2164504660                 /usr/lib64/R/library/parallel/libs/parallel.so
      7fe68600a000-7fe68600b000 r--p 00004000 08:04 2164504660                 /usr/lib64/R/library/parallel/libs/parallel.so
      7fe68600b000-7fe68600c000 rw-p 00005000 08:04 2164504660                 /usr/lib64/R/library/parallel/libs/parallel.so
      7fe68600c000-7fe686018000 r-xp 00000000 08:04 64425907345                /usr/lib64/libnss_files-2.17.so
      7fe686018000-7fe686217000 ---p 0000c000 08:04 64425907345                /usr/lib64/libnss_files-2.17.so
      7fe686217000-7fe686218000 r--p 0000b000 08:04 64425907345                /usr/lib64/libnss_files-2.17.so
      7fe686218000-7fe686219000 rw-p 0000c000 08:04 64425907345                /usr/lib64/libnss_files-2.17.so
      7fe686219000-7fe68621f000 rw-p 00000000 00:00 0
      7fe68621f000-7fe686221000 r-xp 00000000 08:04 64424510590                /usr/lib64/libfreebl3.so
      7fe686221000-7fe686420000 ---p 00002000 08:04 64424510590                /usr/lib64/libfreebl3.so
      7fe686420000-7fe686421000 r--p 00001000 08:04 64424510590                /usr/lib64/libfreebl3.so
      7fe686421000-7fe686422000 rw-p 00002000 08:04 64424510590                /usr/lib64/libfreebl3.so
      7fe686422000-7fe68642a000 r-xp 00000000 08:04 64425907331                /usr/lib64/libcrypt-2.17.so
      7fe68642a000-7fe686629000 ---p 00008000 08:04 64425907331                /usr/lib64/libcrypt-2.17.so
      7fe686629000-7fe68662a000 r--p 00007000 08:04 64425907331                /usr/lib64/libcrypt-2.17.so
      7fe68662a000-7fe68662b000 rw-p 00008000 08:04 64425907331                /usr/lib64/libcrypt-2.17.so
      7fe68662b000-7fe686659000 rw-p 00000000 00:00 0
      7fe686659000-7fe68667d000 r-xp 00000000 08:04 64433322030                /usr/lib64/libselinux.so.1
      7fe68667d000-7fe68687c000 ---p 00024000 08:04 64433322030                /usr/lib64/libselinux.so.1
      7fe68687c000-7fe68687d000 r--p 00023000 08:04 64433322030                /usr/lib64/libselinux.so.1
      7fe68687d000-7fe68687e000 rw-p 00024000 08:04 64433322030                /usr/lib64/libselinux.so.1
      7fe68687e000-7fe686880000 rw-p 00000000 00:00 0
      7fe686880000-7fe68689c000 r-xp 00000000 08:04 64424511832                /usr/lib64/libsasl2.so.3.0.0
      7fe68689c000-7fe686a9b000 ---p 0001c000 08:04 64424511832                /usr/lib64/libsasl2.so.3.0.0
      7fe686a9b000-7fe686a9c000 r--p 0001b000 08:04 64424511832                /usr/lib64/libsasl2.so.3.0.0
      7fe686a9c000-7fe686a9d000 rw-p 0001c000 08:04 64424511832                /usr/lib64/libsasl2.so.3.0.0
      7fe686a9d000-7fe686ab3000 r-xp 00000000 08:04 64425907355                /usr/lib64/libresolv-2.17.so
      7fe686ab3000-7fe686cb3000 ---p 00016000 08:04 64425907355                /usr/lib64/libresolv-2.17.so
      7fe686cb3000-7fe686cb4000 r--p 00016000 08:04 64425907355                /usr/lib64/libresolv-2.17.so
      7fe686cb4000-7fe686cb5000 rw-p 00017000 08:04 64425907355                /usr/lib64/libresolv-2.17.so
      7fe686cb5000-7fe686cb7000 rw-p 00000000 00:00 0
      7fe686cb7000-7fe686cba000 r-xp 00000000 08:04 64424511994                /usr/lib64/libkeyutils.so.1.5
      7fe686cba000-7fe686eb9000 ---p 00003000 08:04 64424511994                /usr/lib64/libkeyutils.so.1.5
      7fe686eb9000-7fe686eba000 r--p 00002000 08:04 64424511994                /usr/lib64/libkeyutils.so.1.5
      7fe686eba000-7fe686ebb000 rw-p 00003000 08:04 64424511994                /usr/lib64/libkeyutils.so.1.5
      7fe686ebb000-7fe686ec8000 r-xp 00000000 08:04 64424692791                /usr/lib64/libkrb5support.so.0.1
      7fe686ec8000-7fe6870c8000 ---p 0000d000 08:04 64424692791                /usr/lib64/libkrb5support.so.0.1
      7fe6870c8000-7fe6870c9000 r--p 0000d000 08:04 64424692791                /usr/lib64/libkrb5support.so.0.1
      7fe6870c9000-7fe6870ca000 rw-p 0000e000 08:04 64424692791                /usr/lib64/libkrb5support.so.0.1
      7fe6870ca000-7fe687288000 r-xp 00000000 08:04 64424692795                /usr/lib64/libcrypto.so.1.0.1e
      7fe687288000-7fe687488000 ---p 001be000 08:04 64424692795                /usr/lib64/libcrypto.so.1.0.1e
      7fe687488000-7fe6874a2000 r--p 001be000 08:04 64424692795                /usr/lib64/libcrypto.so.1.0.1e
      7fe6874a2000-7fe6874ae000 rw-p 001d8000 08:04 64424692795                /usr/lib64/libcrypto.so.1.0.1e
      7fe6874ae000-7fe6874b2000 rw-p 00000000 00:00 0
      7fe6874b2000-7fe687515000 r-xp 00000000 08:04 64424692797                /usr/lib64/libssl.so.1.0.1e
      7fe687515000-7fe687714000 ---p 00063000 08:04 64424692797                /usr/lib64/libssl.so.1.0.1e
      7fe687714000-7fe687718000 r--p 00062000 08:04 64424692797                /usr/lib64/libssl.so.1.0.1e
      7fe687718000-7fe68771f000 rw-p 00066000 08:04 64424692797                /usr/lib64/libssl.so.1.0.1e
      7fe68771f000-7fe68776f000 r-xp 00000000 08:04 64425430157                /usr/lib64/libldap-2.4.so.2.10.3
      7fe68776f000-7fe68796e000 ---p 00050000 08:04 64425430157                /usr/lib64/libldap-2.4.so.2.10.3
      7fe68796e000-7fe687971000 r--p 0004f000 08:04 64425430157                /usr/lib64/libldap-2.4.so.2.10.3
      7fe687971000-7fe687972000 rw-p 00052000 08:04 64425430157                /usr/lib64/libldap-2.4.so.2.10.3
      7fe687972000-7fe687980000 r-xp 00000000 08:04 64425430152                /usr/lib64/liblber-2.4.so.2.10.3
      7fe687980000-7fe687b7f000 ---p 0000e000 08:04 64425430152                /usr/lib64/liblber-2.4.so.2.10.3
      7fe687b7f000-7fe687b80000 r--p 0000d000 08:04 64425430152                /usr/lib64/liblber-2.4.so.2.10.3
      7fe687b80000-7fe687b81000 rw-p 0000e000 08:04 64425430152                /usr/lib64/liblber-2.4.so.2.10.3
      7fe687b81000-7fe687b84000 r-xp 00000000 08:04 64424511895                /usr/lib64/libcom_err.so.2.1
      7fe687b84000-7fe687d83000 ---p 00003000 08:04 64424511895                /usr/lib64/libcom_err.so.2.1
      7fe687d83000-7fe687d84000 r--p 00002000 08:04 64424511895                /usr/lib64/libcom_err.so.2.1
      7fe687d84000-7fe687d85000 rw-p 00003000 08:04 64424511895                /usr/lib64/libcom_err.so.2.1
      7fe687d85000-7fe687db4000 r-xp 00000000 08:04 64424692789                /usr/lib64/libk5crypto.so.3.1
      7fe687db4000-7fe687fb3000 ---p 0002f000 08:04 64424692789                /usr/lib64/libk5crypto.so.3.1
      7fe687fb3000-7fe687fb5000 r--p 0002e000 08:04 64424692789                /usr/lib64/libk5crypto.so.3.1
      7fe687fb5000-7fe687fb6000 rw-p 00030000 08:04 64424692789                /usr/lib64/libk5crypto.so.3.1
      7fe687fb6000-7fe687fb7000 rw-p 00000000 00:00 0
      7fe687fb7000-7fe68808c000 r-xp 00000000 08:04 64424512023                /usr/lib64/libkrb5.so.3.3
      7fe68808c000-7fe68828c000 ---p 000d5000 08:04 64424512023                /usr/lib64/libkrb5.so.3.3
      7fe68828c000-7fe688299000 r--p 000d5000 08:04 64424512023                /usr/lib64/libkrb5.so.3.3
      7fe688299000-7fe68829c000 rw-p 000e2000 08:04 64424512023                /usr/lib64/libkrb5.so.3.3
      7fe68829c000-7fe6882e5000 r-xp 00000000 08:04 64424513555                /usr/lib64/libgssapi_krb5.so.2.2
      7fe6882e5000-7fe6884e5000 ---p 00049000 08:04 64424513555                /usr/lib64/libgssapi_krb5.so.2.2
      7fe6884e5000-7fe6884e6000 r--p 00049000 08:04 64424513555                /usr/lib64/libgssapi_krb5.so.2.2
      7fe6884e6000-7fe6884e8000 rw-p 0004a000 08:04 64424513555                /usr/lib64/libgssapi_krb5.so.2.2
      7fe6884e8000-7fe688522000 r-xp 00000000 08:04 64424511817                /usr/lib64/libnspr4.so
      7fe688522000-7fe688721000 ---p 0003a000 08:04 64424511817                /usr/lib64/libnspr4.so
      7fe688721000-7fe688722000 r--p 00039000 08:04 64424511817                /usr/lib64/libnspr4.so
      7fe688722000-7fe688724000 rw-p 0003a000 08:04 64424511817                /usr/lib64/libnspr4.so
      7fe688724000-7fe688726000 rw-p 00000000 00:00 0
      7fe688726000-7fe68872a000 r-xp 00000000 08:04 64424511819                /usr/lib64/libplc4.so
      7fe68872a000-7fe688929000 ---p 00004000 08:04 64424511819                /usr/lib64/libplc4.so
      7fe688929000-7fe68892a000 r--p 00003000 08:04 64424511819                /usr/lib64/libplc4.so
      7fe68892a000-7fe68892b000 rw-p 00004000 08:04 64424511819                /usr/lib64/libplc4.so
      7fe68892b000-7fe68892e000 r-xp 00000000 08:04 64424511821                /usr/lib64/libplds4.so
      7fe68892e000-7fe688b2d000 ---p 00003000 08:04 64424511821                /usr/lib64/libplds4.so
      7fe688b2d000-7fe688b2e000 r--p 00002000 08:04 64424511821                /usr/lib64/libplds4.so
      7fe688b2e000-7fe688b2f000 rw-p 00003000 08:04 64424511821                /usr/lib64/libplds4.so
      7fe688b2f000-7fe688b55000 r-xp 00000000 08:04 64424511822                /usr/lib64/libnssutil3.so
      7fe688b55000-7fe688d54000 ---p 00026000 08:04 64424511822                /usr/lib64/libnssutil3.so
      7fe688d54000-7fe688d5a000 r--p 00025000 08:04 64424511822                /usr/lib64/libnssutil3.so
      7fe688d5a000-7fe688d5b000 rw-p 0002b000 08:04 64424511822                /usr/lib64/libnssutil3.so
      7fe688d5b000-7fe688e79000 r-xp 00000000 08:04 64426776728                /usr/lib64/libnss3.so
      7fe688e79000-7fe689078000 ---p 0011e000 08:04 64426776728                /usr/lib64/libnss3.so
      7fe689078000-7fe68907d000 r--p 0011d000 08:04 64426776728                /usr/lib64/libnss3.so
      7fe68907d000-7fe68907f000 rw-p 00122000 08:04 64426776728                /usr/lib64/libnss3.so
      7fe68907f000-7fe689081000 rw-p 00000000 00:00 0
      7fe689081000-7fe6890a5000 r-xp 00000000 08:04 64424993837                /usr/lib64/libsmime3.so
      7fe6890a5000-7fe6892a4000 ---p 00024000 08:04 64424993837                /usr/lib64/libsmime3.so
      7fe6892a4000-7fe6892a7000 r--p 00023000 08:04 64424993837                /usr/lib64/libsmime3.so
      7fe6892a7000-7fe6892a8000 rw-p 00026000 08:04 64424993837                /usr/lib64/libsmime3.so
      7fe6892a8000-7fe6892e6000 r-xp 00000000 08:04 64426776729                /usr/lib64/libssl3.so
      7fe6892e6000-7fe6894e5000 ---p 0003e000 08:04 64426776729                /usr/lib64/libssl3.so
      7fe6894e5000-7fe6894e9000 r--p 0003d000 08:04 64426776729                /usr/lib64/libssl3.so
      7fe6894e9000-7fe6894ea000 rw-p 00041000 08:04 64426776729                /usr/lib64/libssl3.so
      7fe6894ea000-7fe6894eb000 rw-p 00000000 00:00 0
      7fe6894eb000-7fe689513000 r-xp 00000000 08:04 64424963118                /usr/lib64/libssh2.so.1.0.1
      7fe689513000-7fe689713000 ---p 00028000 08:04 64424963118                /usr/lib64/libssh2.so.1.0.1
      7fe689713000-7fe689714000 r--p 00028000 08:04 64424963118                /usr/lib64/libssh2.so.1.0.1
      7fe689714000-7fe689715000 rw-p 00029000 08:04 64424963118                /usr/lib64/libssh2.so.1.0.1
      7fe689715000-7fe689747000 r-xp 00000000 08:04 64424512526                /usr/lib64/libidn.so.11.6.11
      7fe689747000-7fe689946000 ---p 00032000 08:04 64424512526                /usr/lib64/libidn.so.11.6.11
      7fe689946000-7fe689947000 r--p 00031000 08:04 64424512526                /usr/lib64/libidn.so.11.6.11
      7fe689947000-7fe689948000 rw-p 00032000 08:04 64424512526                /usr/lib64/libidn.so.11.6.11
      7fe689948000-7fe6899ad000 r-xp 00000000 08:04 64434945109                /usr/lib64/libcurl.so.4.3.0
      7fe6899ad000-7fe689bad000 ---p 00065000 08:04 64434945109                /usr/lib64/libcurl.so.4.3.0
      7fe689bad000-7fe689baf000 r--p 00065000 08:04 64434945109                /usr/lib64/libcurl.so.4.3.0
      7fe689baf000-7fe689bb0000 rw-p 00067000 08:04 64434945109                /usr/lib64/libcurl.so.4.3.0
      7fe689bb0000-7fe689bb1000 rw-p 00000000 00:00 0
      7fe689bb1000-7fe689bc4000 r-xp 00000000 08:04 66581724169                /usr/lib64/R/modules/internet.so
      7fe689bc4000-7fe689dc3000 ---p 00013000 08:04 66581724169                /usr/lib64/R/modules/internet.so
      7fe689dc3000-7fe689dc4000 r--p 00012000 08:04 66581724169                /usr/lib64/R/modules/internet.so
      7fe689dc4000-7fe689dc5000 rw-p 00013000 08:04 66581724169                /usr/lib64/R/modules/internet.so
      7fe689dc5000-7fe68dfe6000 rw-p 00000000 00:00 0
      7fe68e02a000-7fe68e164000 rw-p 00000000 00:00 0
      7fe68e164000-7fe68e16c000 r-xp 00000000 08:04 6445979743                 /usr/lib64/R/library/methods/libs/methods.so
      7fe68e16c000-7fe68e36b000 ---p 00008000 08:04 6445979743                 /usr/lib64/R/library/methods/libs/methods.so
      7fe68e36b000-7fe68e36c000 r--p 00007000 08:04 6445979743                 /usr/lib64/R/library/methods/libs/methods.so
      7fe68e36c000-7fe68e36d000 rw-p 00008000 08:04 6445979743                 /usr/lib64/R/library/methods/libs/methods.so
      7fe68e36d000-7fe68e39a000 rw-p 00000000 00:00 0
      7fe68e39a000-7fe68e58f000 r-xp 00000000 08:04 19333693112                /usr/lib64/R/lib/libRlapack.so
      7fe68e58f000-7fe68e78e000 ---p 001f5000 08:04 19333693112                /usr/lib64/R/lib/libRlapack.so
      7fe68e78e000-7fe68e78f000 r--p 001f4000 08:04 19333693112                /usr/lib64/R/lib/libRlapack.so
      7fe68e78f000-7fe68e790000 rw-p 001f5000 08:04 19333693112                /usr/lib64/R/lib/libRlapack.so
      7fe68e790000-7fe68e836000 r-xp 00000000 08:04 45100675141                /usr/lib64/R/library/stats/libs/stats.so
      7fe68e836000-7fe68ea36000 ---p 000a6000 08:04 45100675141                /usr/lib64/R/library/stats/libs/stats.so
      7fe68ea36000-7fe68ea38000 r--p 000a6000 08:04 45100675141                /usr/lib64/R/library/stats/libs/stats.so
      7fe68ea38000-7fe68ea3a000 rw-p 000a8000 08:04 45100675141                /usr/lib64/R/library/stats/libs/stats.so
      7fe68ea3a000-7fe68ebc0000 rw-p 00000000 00:00 0
      7fe68ebc0000-7fe68ebfd000 r-xp 00000000 08:04 6445979741                 /usr/lib64/R/library/graphics/libs/graphics.so
      7fe68ebfd000-7fe68edfd000 ---p 0003d000 08:04 6445979741                 /usr/lib64/R/library/graphics/libs/graphics.so
      7fe68edfd000-7fe68edfe000 r--p 0003d000 08:04 6445979741                 /usr/lib64/R/library/graphics/libs/graphics.so
      7fe68edfe000-7fe68edff000 rw-p 0003e000 08:04 6445979741                 /usr/lib64/R/library/graphics/libs/graphics.so
      7fe68edff000-7fe68ee4c000 rw-p 00000000 00:00 0
      7fe68ee4c000-7fe68ee80000 r-xp 00000000 08:04 57982776471                /usr/lib64/R/library/grDevices/libs/grDevices.so
      7fe68ee80000-7fe68f080000 ---p 00034000 08:04 57982776471                /usr/lib64/R/library/grDevices/libs/grDevices.so
      7fe68f080000-7fe68f085000 r--p 00034000 08:04 57982776471                /usr/lib64/R/library/grDevices/libs/grDevices.so
      7fe68f085000-7fe68f087000 rw-p 00039000 08:04 57982776471                /usr/lib64/R/library/grDevices/libs/grDevices.so
      7fe68f087000-7fe68f0ba000 rw-p 00000000 00:00 0
      7fe68f0ba000-7fe68f0c4000 r-xp 00000000 08:04 62293320708                /usr/lib64/R/library/utils/libs/utils.so
      7fe68f0c4000-7fe68f2c3000 ---p 0000a000 08:04 62293320708                /usr/lib64/R/library/utils/libs/utils.so
      7fe68f2c3000-7fe68f2c4000 r--p 00009000 08:04 62293320708                /usr/lib64/R/library/utils/libs/utils.so
      7fe68f2c4000-7fe68f2c5000 rw-p 0000a000 08:04 62293320708                /usr/lib64/R/library/utils/libs/utils.so
      7fe68f2c5000-7fe68f39f000 rw-p 00000000 00:00 0
      7fe68f3f7000-7fe68f705000 rw-p 00000000 00:00 0
      7fe68f705000-7fe695c2c000 r--p 00000000 08:04 53687238663                /usr/lib/locale/locale-archive
      7fe695c2c000-7fe695d15000 r-xp 00000000 08:04 64424511848                /usr/lib64/libstdc++.so.6.0.19
      7fe695d15000-7fe695f15000 ---p 000e9000 08:04 64424511848                /usr/lib64/libstdc++.so.6.0.19
      7fe695f15000-7fe695f1e000 r--p 000e9000 08:04 64424511848                /usr/lib64/libstdc++.so.6.0.19
      7fe695f1e000-7fe695f20000 rw-p 000f2000 08:04 64424511848                /usr/lib64/libstdc++.so.6.0.19
      7fe695f20000-7fe695f35000 rw-p 00000000 00:00 0
      7fe695f35000-7fe697308000 r-xp 00000000 08:04 64424511959                /usr/lib64/libicudata.so.50.1.2
      7fe697308000-7fe697507000 ---p 013d3000 08:04 64424511959                /usr/lib64/libicudata.so.50.1.2
      7fe697507000-7fe697508000 r--p 013d2000 08:04 64424511959                /usr/lib64/libicudata.so.50.1.2
      7fe697508000-7fe697509000 rw-p 013d3000 08:04 64424511959                /usr/lib64/libicudata.so.50.1.2
      7fe697509000-7fe69752e000 r-xp 00000000 08:04 64424511873                /usr/lib64/libtinfo.so.5.9
      7fe69752e000-7fe69772e000 ---p 00025000 08:04 64424511873                /usr/lib64/libtinfo.so.5.9
      7fe69772e000-7fe697732000 r--p 00025000 08:04 64424511873                /usr/lib64/libtinfo.so.5.9
      7fe697732000-7fe697733000 rw-p 00029000 08:04 64424511873                /usr/lib64/libtinfo.so.5.9
      7fe697733000-7fe697748000 r-xp 00000000 08:04 64434760744                /usr/lib64/libgcc_s-4.8.5-20150702.so.1
      7fe697748000-7fe697947000 ---p 00015000 08:04 64434760744                /usr/lib64/libgcc_s-4.8.5-20150702.so.1
      7fe697947000-7fe697948000 r--p 00014000 08:04 64434760744                /usr/lib64/libgcc_s-4.8.5-20150702.so.1
      7fe697948000-7fe697949000 rw-p 00015000 08:04 64434760744                /usr/lib64/libgcc_s-4.8.5-20150702.so.1
      7fe697949000-7fe697b39000 r-xp 00000000 08:04 64424511961                /usr/lib64/libicui18n.so.50.1.2
      7fe697b39000-7fe697d39000 ---p 001f0000 08:04 64424511961                /usr/lib64/libicui18n.so.50.1.2
      7fe697d39000-7fe697d45000 r--p 001f0000 08:04 64424511961                /usr/lib64/libicui18n.so.50.1.2
      7fe697d45000-7fe697d47000 rw-p 001fc000 08:04 64424511961                /usr/lib64/libicui18n.so.50.1.2
      7fe697d47000-7fe697d48000 rw-p 00000000 00:00 0
      7fe697d48000-7fe697eac000 r-xp 00000000 08:04 64424511973                /usr/lib64/libicuuc.so.50.1.2
      7fe697eac000-7fe6980ac000 ---p 00164000 08:04 64424511973                /usr/lib64/libicuuc.so.50.1.2
      7fe6980ac000-7fe6980bc000 r--p 00164000 08:04 64424511973                /usr/lib64/libicuuc.so.50.1.2
      7fe6980bc000-7fe6980bd000 rw-p 00174000 08:04 64424511973                /usr/lib64/libicuuc.so.50.1.2
      7fe6980bd000-7fe6980c1000 rw-p 00000000 00:00 0
      7fe6980c1000-7fe6980c4000 r-xp 00000000 08:04 64425907333                /usr/lib64/libdl-2.17.so
      7fe6980c4000-7fe6982c3000 ---p 00003000 08:04 64425907333                /usr/lib64/libdl-2.17.so
      7fe6982c3000-7fe6982c4000 r--p 00002000 08:04 64425907333                /usr/lib64/libdl-2.17.so
      7fe6982c4000-7fe6982c5000 rw-p 00003000 08:04 64425907333                /usr/lib64/libdl-2.17.so
      7fe6982c5000-7fe6982cc000 r-xp 00000000 08:04 64425907357                /usr/lib64/librt-2.17.so
      7fe6982cc000-7fe6984cb000 ---p 00007000 08:04 64425907357                /usr/lib64/librt-2.17.so
      7fe6984cb000-7fe6984cc000 r--p 00006000 08:04 64425907357                /usr/lib64/librt-2.17.so
      7fe6984cc000-7fe6984cd000 rw-p 00007000 08:04 64425907357                /usr/lib64/librt-2.17.so
      7fe6984cd000-7fe6984e2000 r-xp 00000000 08:04 64425230795                /usr/lib64/libz.so.1.2.7
      7fe6984e2000-7fe6986e1000 ---p 00015000 08:04 64425230795                /usr/lib64/libz.so.1.2.7
      7fe6986e1000-7fe6986e2000 r--p 00014000 08:04 64425230795                /usr/lib64/libz.so.1.2.7
      7fe6986e2000-7fe6986e3000 rw-p 00015000 08:04 64425230795                /usr/lib64/libz.so.1.2.7
      7fe6986e3000-7fe6986f2000 r-xp 00000000 08:04 64424511910                /usr/lib64/libbz2.so.1.0.6
      7fe6986f2000-7fe6988f1000 ---p 0000f000 08:04 64424511910                /usr/lib64/libbz2.so.1.0.6
      7fe6988f1000-7fe6988f2000 r--p 0000e000 08:04 64424511910                /usr/lib64/libbz2.so.1.0.6
      7fe6988f2000-7fe6988f3000 rw-p 0000f000 08:04 64424511910                /usr/lib64/libbz2.so.1.0.6
      7fe6988f3000-7fe698918000 r-xp 00000000 08:04 64432774884                /usr/lib64/liblzma.so.5.2.2
      7fe698918000-7fe698b17000 ---p 00025000 08:04 64432774884                /usr/lib64/liblzma.so.5.2.2
      7fe698b17000-7fe698b18000 r--p 00024000 08:04 64432774884                /usr/lib64/liblzma.so.5.2.2
      7fe698b18000-7fe698b19000 rw-p 00025000 08:04 64432774884                /usr/lib64/liblzma.so.5.2.2
      7fe698b19000-7fe698b79000 r-xp 00000000 08:04 64424511837                /usr/lib64/libpcre.so.1.2.0
      7fe698b79000-7fe698d78000 ---p 00060000 08:04 64424511837                /usr/lib64/libpcre.so.1.2.0
      7fe698d78000-7fe698d79000 r--p 0005f000 08:04 64424511837                /usr/lib64/libpcre.so.1.2.0
      7fe698d79000-7fe698d7a000 rw-p 00060000 08:04 64424511837                /usr/lib64/libpcre.so.1.2.0
      7fe698d7a000-7fe698d89000 r-xp 00000000 08:04 64432775393                /usr/lib64/libtre.so.5.0.0
      7fe698d89000-7fe698f88000 ---p 0000f000 08:04 64432775393                /usr/lib64/libtre.so.5.0.0
      7fe698f88000-7fe698f89000 r--p 0000e000 08:04 64432775393                /usr/lib64/libtre.so.5.0.0
      7fe698f89000-7fe698f8a000 rw-p 0000f000 08:04 64432775393                /usr/lib64/libtre.so.5.0.0
      7fe698f8a000-7fe698fc6000 r-xp 00000000 08:04 64424511924                /usr/lib64/libreadline.so.6.2
      7fe698fc6000-7fe6991c6000 ---p 0003c000 08:04 64424511924                /usr/lib64/libreadline.so.6.2
      7fe6991c6000-7fe6991c8000 r--p 0003c000 08:04 64424511924                /usr/lib64/libreadline.so.6.2
      7fe6991c8000-7fe6991ce000 rw-p 0003e000 08:04 64424511924                /usr/lib64/libreadline.so.6.2
      7fe6991ce000-7fe6991d0000 rw-p 00000000 00:00 0
      7fe6991d0000-7fe69920b000 r-xp 00000000 08:04 64424589353                /usr/lib64/libquadmath.so.0.0.0
      7fe69920b000-7fe69940a000 ---p 0003b000 08:04 64424589353                /usr/lib64/libquadmath.so.0.0.0
      7fe69940a000-7fe69940b000 r--p 0003a000 08:04 64424589353                /usr/lib64/libquadmath.so.0.0.0
      7fe69940b000-7fe69940c000 rw-p 0003b000 08:04 64424589353                /usr/lib64/libquadmath.so.0.0.0
      7fe69940c000-7fe69950d000 r-xp 00000000 08:04 64425907335                /usr/lib64/libm-2.17.so
      7fe69950d000-7fe69970c000 ---p 00101000 08:04 64425907335                /usr/lib64/libm-2.17.so
      7fe69970c000-7fe69970d000 r--p 00100000 08:04 64425907335                /usr/lib64/libm-2.17.so
      7fe69970d000-7fe69970e000 rw-p 00101000 08:04 64425907335                /usr/lib64/libm-2.17.so
      7fe69970e000-7fe69982d000 r-xp 00000000 08:04 64425157647                /usr/lib64/libgfortran.so.3.0.0
      7fe69982d000-7fe699a2d000 ---p 0011f000 08:04 64425157647                /usr/lib64/libgfortran.so.3.0.0
      7fe699a2d000-7fe699a2f000 r--p 0011f000 08:04 64425157647                /usr/lib64/libgfortran.so.3.0.0
      7fe699a2f000-7fe699a31000 rw-p 00121000 08:04 64425157647                /usr/lib64/libgfortran.so.3.0.0
      7fe699a31000-7fe699be8000 r-xp 00000000 08:04 64426776661                /usr/lib64/libc-2.17.so
      7fe699be8000-7fe699de8000 ---p 001b7000 08:04 64426776661                /usr/lib64/libc-2.17.so
      7fe699de8000-7fe699dec000 r--p 001b7000 08:04 64426776661                /usr/lib64/libc-2.17.so
      7fe699dec000-7fe699dee000 rw-p 001bb000 08:04 64426776661                /usr/lib64/libc-2.17.so
      7fe699dee000-7fe699df3000 rw-p 00000000 00:00 0
      7fe699df3000-7fe699e09000 r-xp 00000000 08:04 64425907353                /usr/lib64/libpthread-2.17.so
      7fe699e09000-7fe69a009000 ---p 00016000 08:04 64425907353                /usr/lib64/libpthread-2.17.so
      7fe69a009000-7fe69a00a000 r--p 00016000 08:04 64425907353                /usr/lib64/libpthread-2.17.so
      7fe69a00a000-7fe69a00b000 rw-p 00017000 08:04 64425907353                /usr/lib64/libpthread-2.17.so
      7fe69a00b000-7fe69a00f000 rw-p 00000000 00:00 0
      7fe69a00f000-7fe69a034000 r-xp 00000000 08:04 64424513594                /usr/lib64/libgomp.so.1.0.0
      7fe69a034000-7fe69a233000 ---p 00025000 08:04 64424513594                /usr/lib64/libgomp.so.1.0.0
      7fe69a233000-7fe69a234000 r--p 00024000 08:04 64424513594                /usr/lib64/libgomp.so.1.0.0
      7fe69a234000-7fe69a235000 rw-p 00025000 08:04 64424513594                /usr/lib64/libgomp.so.1.0.0
      7fe69a235000-7fe69c267000 r-xp 00000000 08:04 19333693109                /usr/lib64/R/lib/libRblas.so
      7fe69c267000-7fe69c466000 ---p 02032000 08:04 19333693109                /usr/lib64/R/lib/libRblas.so
      7fe69c466000-7fe69c46a000 r--p 02031000 08:04 19333693109                /usr/lib64/R/lib/libRblas.so
      7fe69c46a000-7fe69c482000 rw-p 02035000 08:04 19333693109                /usr/lib64/R/lib/libRblas.so
      7fe69c482000-7fe69c48d000 rw-p 00000000 00:00 0
      7fe69c48d000-7fe69c73b000 r-xp 00000000 08:04 19333693111                /usr/lib64/R/lib/libR.so
      7fe69c73b000-7fe69c93a000 ---p 002ae000 08:04 19333693111                /usr/lib64/R/lib/libR.so
      7fe69c93a000-7fe69c958000 r--p 002ad000 08:04 19333693111                /usr/lib64/R/lib/libR.so
      7fe69c958000-7fe69c964000 rw-p 002cb000 08:04 19333693111                /usr/lib64/R/lib/libR.so
      7fe69c964000-7fe69cab5000 rw-p 00000000 00:00 0
      7fe69cab5000-7fe69cad6000 r-xp 00000000 08:04 64424511800                /usr/lib64/ld-2.17.so
      7fe69cade000-7fe69ccba000 rw-p 00000000 00:00 0
      7fe69ccba000-7fe69ccbb000 rw-p 00000000 00:00 0
      7fe69ccbb000-7fe69ccbc000 r--p 00000000 08:04 49394932767                /usr/lib64/R/library/translations/en/LC_MESSAGES/R.mo
      7fe69ccbc000-7fe69cccc000 rwxp 00000000 00:00 0
      7fe69cccc000-7fe69ccd3000 r--s 00000000 08:04 4295610652                 /usr/lib64/gconv/gconv-modules.cache
      7fe69ccd3000-7fe69ccd6000 rw-p 00000000 00:00 0
      7fe69ccd6000-7fe69ccd7000 r--p 00021000 08:04 64424511800                /usr/lib64/ld-2.17.so
      7fe69ccd7000-7fe69ccd8000 rw-p 00022000 08:04 64424511800                /usr/lib64/ld-2.17.so
      7fe69ccd8000-7fe69ccd9000 rw-p 00000000 00:00 0
      7fff43955000-7fff439b1000 rw-p 00000000 00:00 0                          [stack]
      7fff439f2000-7fff439f4000 r-xp 00000000 00:00 0                          [vdso]
      ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
      

      The related test - https://github.com/apache/spark/blob/master/R/pkg/tests/fulltests/test_sparkSQL.R#L3007-L3095 passes in

      Windows Server 2012 R2 / R 3.3.1
      macOS Sierra 10.12.3 / R 3.4.0
      macOS Sierra 10.12.3 / R 3.2.3
      Ubuntu 17 / R 3.3

      failed in

      Centos 7.2.1511 with R 3.4.0/3.3.3

        Attachments

          Activity

            People

            • Assignee:
              hyukjin.kwon Hyukjin Kwon
              Reporter:
              hyukjin.kwon Hyukjin Kwon
              Shepherd:
              Felix Cheung
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: