[HIVE-13876] Vectorization: Port HIVE-11544 to LazySimpleDeserializeRead

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.0
• Component/s: File Formats, Vectorization

Release Note

Vectorization: Port HIVE-11544 to LazySimpleDeserializeRead (Gopal V, reviewed by Sergey Shelukhin)

• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 27/May/16 09:28
• Updated: 23/Nov/16 10:06
• Resolved: 01/Jun/16 10:46

Description

High CPU usage due to exception handling code.

 TezTaskRunner [RUNNABLE] [DAEMON]
java.lang.Throwable.fillInStackTrace(int) Throwable.java (native)
java.lang.Throwable.fillInStackTrace() Throwable.java:783
java.lang.Throwable.<init>(String) Throwable.java:265
java.lang.Exception.<init>(String) Exception.java:66
java.lang.RuntimeException.<init>(String) RuntimeException.java:62
java.lang.IllegalArgumentException.<init>(String) IllegalArgumentException.java:52
java.lang.NumberFormatException.<init>(String) NumberFormatException.java:55
sun.misc.FloatingDecimal.readJavaFormatString(String) FloatingDecimal.java:1842
sun.misc.FloatingDecimal.parseFloat(String) FloatingDecimal.java:122
java.lang.Float.parseFloat(String) Float.java:451
org.apache.hadoop.hive.serde2.lazy.fast.LazySimpleDeserializeRead.readCheckNull() LazySimpleDeserializeRead.java:309
org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeRowColumn(VectorizedRowBatch, int, int) VectorDeserializeRow.java:346
org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorizedRowBatch, int) VectorDeserializeRow.java:659
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(Writable) VectorMapOperator.java:814
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(Object) MapRecordSource.java:86
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord() MapRecordSource.java:70
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run() MapRecordProcessor.java:361
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(Map, Map) TezProcessor.java:172
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(Map, Map) TezProcessor.java:160
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run() LogicalIOProcessorRuntimeTask.java:370
org.apache.tez.runtime.task.TaskRunner2Callable$1.run() TaskRunner2Callable.java:73
org.apache.tez.runtime.task.TaskRunner2Callable$1.run() TaskRunner2Callable.java:61
java.security.AccessController.doPrivileged(PrivilegedExceptionAction, AccessControlContext) AccessController.java (native)
javax.security.auth.Subject.doAs(Subject, PrivilegedExceptionAction) Subject.java:422
org.apache.hadoop.security.UserGroupInformation.doAs(PrivilegedExceptionAction) UserGroupInformation.java:1657
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal() TaskRunner2Callable.java:61
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal() TaskRunner2Callable.java:37
org.apache.tez.common.CallableWithNdc.call() CallableWithNdc.java:36
java.util.concurrent.FutureTask.run() FutureTask.java:266
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) ThreadPoolExecutor.java:1142
java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:617
java.lang.Thread.run() Thread.java:745

Issue Links

• is part of: HIVE-13878 Vectorization: Column pruning for Text vectorization
• is required by: HIVE-13887 LazySimpleSerDe should parse "NULL" dates faster