[SPARK-19751] Create Data frame API fails with a self referencing bean

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Takeshi Yamamuro
• Reporter: Avinash Venkateshaiah
• Created: 27/Feb/17 15:42
• Updated: 16/Mar/17 00:51
• Resolved: 16/Mar/17 00:50

Description

createDataset API throws a stack overflow exception when we try creating a
Dataset using a bean encoder. The bean is self referencing

BEAN:
public class HierObj implements Serializable {
String name;
List<HierObj> children;

public String getName()

{ return name; }

public void setName(String name)

{ this.name = name; }

public List<HierObj> getChildren()

{ return children; }

public void setChildren(List<HierObj> children)

{ this.children = children; }

}

// create an object
HierObj hierObj = new HierObj();
hierObj.setName("parent");
List children = new ArrayList();

HierObj child1 = new HierObj();
child1.setName("child1");
HierObj child2 = new HierObj();
child2.setName("child2");
children.add(child1);
children.add(child2);

hierObj.setChildren(children);

// create a dataset
Dataset ds = sparkSession().createDataset(Arrays.asList(hierObj), Encoders.bean(HierObj.class));

Issue Links

• is related to: SPARK-19896 toDS throws StackOverflowError if case classes have circular references
• links to: [Github] Pull Request #17188 (maropu)