-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 1.5.6
-
Fix Version/s: None
-
Component/s: Java
-
Labels:None
Unlike 1.5.5, ORC 1.5.6-rc1 seems to have a following issue which causes Apache Spark unit test failures.
$ build/sbt "sql/testOnly *.SQLQuerySuite"
...
[info] Cause: java.lang.Throwable:
[info] at org.apache.spark.DebugFilesystem$.addOpenStream(DebugFilesystem.scala:36)
[info] at org.apache.spark.DebugFilesystem.open(DebugFilesystem.scala:70)
[info] at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:769)
[info] at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:535)
[info] at org.apache.orc.impl.ReaderImpl.<init>(ReaderImpl.java:368)
[info] at org.apache.orc.OrcFile.createReader(OrcFile.java:343)
ORC-498 (ReaderImpl and RecordReaderImpl open separate file handles) seems to be the reason of this change.
- is caused by
-
SPARK-28208 Upgrade to ORC 1.5.6
-
- Resolved
-