[ARROW-16608] [Gandiva][Java] Unsatisfied Link Error on M1 Mac when using mavencentral artifacts

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 8.0.0
• Component/s: C++ - Gandiva, Java
• External issue URL: https://github.com/apache/arrow/issues/31962
• Assignee: Rok Mihevc
• Reporter: Jonathan Swenson
• Created: 18/May/22 17:46
• Updated: 11/Jan/23 11:45
• Resolved: 08/Nov/22 20:18

Description

Potentially a blocker for Arrow Integration into Calcite: CALCITE-2040, however it may be possible to move forward without M1 Mac support. 

potentially somewhat related to ARROW-11135

Getting an instance of the JNILoader throw a Unsatisfied Link Error when it tries to load the libgandiva_jni.dylib that it has extracted from the jar into a temporary directory. 

Simplified error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp_dir/libgandiva_jni.dylib_uuid: dlopen(/tmp_dir/libgandiva_jni.dylib_uuid, 0x0001): tried: '/tmp_dir/libgandiva_jni.dylib_uuid' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))

 

Full error and stack trace:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe: dlopen(/private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe, 0x0001): tried: '/private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832)
    at java.lang.Runtime.load0(Runtime.java:811)
    at java.lang.System.load(System.java:1088)
    at org.apache.arrow.gandiva.evaluator.JniLoader.loadGandivaLibraryFromJar(JniLoader.java:74)
    at org.apache.arrow.gandiva.evaluator.JniLoader.setupInstance(JniLoader.java:63)
    at org.apache.arrow.gandiva.evaluator.JniLoader.getInstance(JniLoader.java:53)
    at org.apache.arrow.gandiva.evaluator.JniLoader.getDefaultConfiguration(JniLoader.java:144)
    at org.apache.arrow.gandiva.evaluator.Filter.make(Filter.java:67)
    at io.acme.Main.main(Main.java:26) 

 

This example loads three libraries from mavencentral using gradle: 

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.apache.arrow:arrow-memory-netty:8.0.0")
    implementation("org.apache.arrow:arrow-vector:8.0.0")
    implementation("org.apache.arrow.gandiva:arrow-gandiva:8.0.0")
} 

Example code: 

public class Main {
  public static void main(String[] args) throws GandivaException {
    Field field = new Field("int_field", FieldType.nullable(new ArrowType.Int(32, true)), null);

    Schema schema = makeSchema(field);
    Condition condition = makeCondition(field);

    Filter.make(schema, condition);
  }

  private static Schema makeSchema(Field field) {
    List<Field> fieldList = new ArrayList<>();
    fieldList.add(field);

    return new Schema(fieldList, null);
  }

  private static Condition makeCondition(Field f) {
    List<TreeNode> treeNodes = new ArrayList<>(2);
    treeNodes.add(TreeBuilder.makeField(f));
    treeNodes.add(TreeBuilder.makeLiteral(4));
    TreeNode comparison = TreeBuilder.makeFunction("less_than", treeNodes, new ArrowType.Bool());
    return TreeBuilder.makeCondition(comparison);
  }
} 

While I haven't tested this exact example, a similar example executes without issue on an intel x86 mac.

Issue Links

• duplicates: ARROW-17267 Failure loading arrow_dataset_jni library on Mac M1
• is depended upon by: CALCITE-2040 Create adapter for Apache Arrow