[#HIVE-593] UDF and GenericUDF clean-up

Hadoop Hive

UDF and GenericUDF clean-up

Created: 30/Jun/09 01:24 AM Updated: 02/Jul/09 11:47 PM

Return to search

Component/s:

None

Affects Version/s:

0.3.0, 0.3.1

Fix Version/s:

0.4.0

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
HIVE-593.1.patch	2009-07-01 09:03 PM	Zheng Shao	58 kB
HIVE-593.2.patch	2009-07-02 06:03 PM	Zheng Shao	58 kB

Resolution Date:

02/Jul/09 11:47 PM

Description

« Hide

We should deprecate ExprNodeIndexEvaluator and ExprNodeFieldEvaluator and use GenericUDF.
We should also clean up ExprNodeFuncEvaluator - there are a lot of hacks to be removed (hacks to get map, list and struct to work)

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Zheng Shao made changes - 01/Jul/09 05:25 AM

Field	Original Value	New Value
Assignee		Zheng Shao [ zshao ]

[ Permlink | « Hide ]

Zheng Shao added a comment - 01/Jul/09 09:03 PM

This patch replaces exprNodeIndexDesc and exprNodeIndexEvaluator with GenericUDF "index".

The removal of exprNodeFieldDesc will require some additional work (mainly that we need to serialize the field name together with the GenericUDF, but currently, GenericUDF is not serializable). I think we should move that to a separate JIRA, in which we will make GenericUDF optional implement Serializable, so that we can store compile-time information for use at the runtime.

Zheng Shao made changes - 01/Jul/09 09:03 PM

Attachment

HIVE-593.1.patch [ 12412316 ]

Zheng Shao made changes - 01/Jul/09 09:07 PM

Status

Open [ 1 ]

Patch Available [ 10002 ]

[ Permlink | « Hide ]

He Yongqiang added a comment - 02/Jul/09 12:56 PM

there is one error in line 97 of GenericUDFIndex, isWritable is not defined.

[ Permlink | « Hide ]

Zheng Shao added a comment - 02/Jul/09 06:03 PM

Updated with trunk and fixed the problem.

Zheng Shao made changes - 02/Jul/09 06:03 PM

Attachment

HIVE-593.2.patch [ 12412410 ]

[ Permlink | « Hide ]

Prasad Chakka added a comment - 02/Jul/09 09:56 PM

looks good. submit once tests are passed.

Repository	Revision	Date	User	Message
ASF	#790762	Thu Jul 02 23:47:17 UTC 2009	prasadc	~~HIVE-593~~. UDF and GenericUDF clean-up. (Zheng Shao via pchakka)
				Files Changed
				MODIFY /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/PartitionPruner.java MODIFY /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java MODIFY /hadoop/hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestExpressionEvaluator.java MODIFY /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeIndexEvaluator.java MODIFY /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/exprNodeIndexDesc.java MODIFY /hadoop/hive/trunk/ql/src/test/results/clientnegative/udf_size_wrong_args_len.q.out MODIFY /hadoop/hive/trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml MODIFY /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluatorFactory.java MODIFY /hadoop/hive/trunk/CHANGES.txt MODIFY /hadoop/hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml MODIFY /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java MODIFY /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java MODIFY /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSize.java

[ Permlink | « Hide ]

Prasad Chakka added a comment - 02/Jul/09 11:47 PM

Thanks Zheng.

Prasad Chakka made changes - 02/Jul/09 11:47 PM

Resolution		Fixed [ 1 ]
Fix Version/s		0.4.0 [ 12313714 ]
Status	Patch Available [ 10002 ]	Resolved [ 5 ]

Repository	Revision	Date	User	Message
ASF	#790770	Fri Jul 03 00:37:13 UTC 2009	prasadc	straggler from ~~HIVE-593~~
				Files Changed
				ADD /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIndex.java

Repository	Revision	Date	User	Message
ASF	#792366	Thu Jul 09 01:06:02 UTC 2009	prasadc	~~HIVE-593~~, remove zeroed out file
				Files Changed
				DEL /hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/exprNodeIndexDesc.java