[SPARK-44910] Encoders.bean does not support superclasses with generic type arguments

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.1, 3.5.0, 4.0.0
• Fix Version/s: 3.4.2, 4.0.0, 3.5.1
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Giambattista Bloisi
• Reporter: Giambattista Bloisi
• Created: 22/Aug/23 11:54
• Updated: 15/Dec/23 04:52
• Resolved: 19/Sep/23 04:37

Description

As per SPARK-44634 another unsupported feature of bean encoder is when the superclass of the bean has generic type arguments. For example:

class JavaBeanWithGenericsA<T> {
    public T getPropertyA() {
        return null;
    }

    public void setPropertyA(T a) {

    }
}

class JavaBeanWithGenericBase extends JavaBeanWithGenericsA<String> {
}

Encoders.bean(JavaBeanWithGenericBase.class); // Exception

Issue Links

• Blocked: SPARK-45311 Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"
• links to: [Github] Pull Request #42634 (gbloisi-openaire)
• links to: GitHub Pull Request #42634
• links to: GitHub Pull Request #42914