Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10566

SnappyCompressionCodec init exception handling masks important error information

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.5.0
    • 1.5.1, 1.6.0
    • Input/Output
    • None

    Description

      The change to always throw an IllegalArgumentException when failing to load in SnappyCompressionCodec (CompressionCodec.scala:151) throws away the description from the exception thrown, which makes it really difficult to actually figure out what the problem is:

      : java.lang.IllegalArgumentException
      at org.apache.spark.io.SnappyCompressionCodec.<init>(CompressionCodec.scala:151)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

      Removing this try...catch I get the following error, which actually gives some information about how to fix the problem:

      : java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
      at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1864)
      at java.lang.Runtime.loadLibrary0(Runtime.java:870)
      at java.lang.System.loadLibrary(System.java:1122)
      at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:178)
      at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:152)

      A change to initialize the IllegalArgumentException with the value of e.getMessage() would be great, as the current error without any description just leads to a lot of frustrating guesswork.

      Attachments

        Activity

          People

            dimfeld Daniel Imfeld
            dimfeld Daniel Imfeld
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: