Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10566

SnappyCompressionCodec init exception handling masks important error information

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.5.1, 1.6.0
    • Component/s: Input/Output
    • Labels:
      None

      Description

      The change to always throw an IllegalArgumentException when failing to load in SnappyCompressionCodec (CompressionCodec.scala:151) throws away the description from the exception thrown, which makes it really difficult to actually figure out what the problem is:

      : java.lang.IllegalArgumentException
      at org.apache.spark.io.SnappyCompressionCodec.<init>(CompressionCodec.scala:151)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

      Removing this try...catch I get the following error, which actually gives some information about how to fix the problem:

      : java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
      at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1864)
      at java.lang.Runtime.loadLibrary0(Runtime.java:870)
      at java.lang.System.loadLibrary(System.java:1122)
      at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:178)
      at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:152)

      A change to initialize the IllegalArgumentException with the value of e.getMessage() would be great, as the current error without any description just leads to a lot of frustrating guesswork.

        Attachments

          Activity

            People

            • Assignee:
              dimfeld Daniel Imfeld
              Reporter:
              dimfeld Daniel Imfeld
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: