[HIVE-12537] RLEv2 doesn't seem to work

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.14.0, 1.0.1, 1.1.1, 1.2.1, 1.3.0, 2.0.0
• Fix Version/s: 2.0.0
• Component/s: File Formats, ORC
• Labels: orc orcfile
• Target Version/s: 1.3.0, 2.0.0, 2.1.0
• Assignee: Prasanth Jayachandran
• Reporter: Bogdan Raducanu
• Created: 28/Nov/15 00:33
• Updated: 02/Jun/16 10:17
• Resolved: 03/Dec/15 06:59

Description

Perhaps I'm doing something wrong or is actually working as expected.

Putting 1 million constant int32 values produces an ORC file of 1MB. Surprisingly, 1 million consecutive ints produces a much smaller file.
Code and FileDump attached.

ObjectInspector inspector = ObjectInspectorFactory.getReflectionObjectInspector(
		Integer.class, ObjectInspectorFactory.ObjectInspectorOptions.JAVA);
Writer w = OrcFile.createWriter(new Path("/tmp/my.orc"), 
			OrcFile.writerOptions(new Configuration())
				.compress(CompressionKind.NONE)
				.inspector(inspector)
				.encodingStrategy(OrcFile.EncodingStrategy.COMPRESSION)
				.version(OrcFile.Version.V_0_12)
		);

for (int i = 0; i < 1000000; ++i) {
	w.addRow(123);
}
w.close();