[HIVE-4304] Remove unused builtins and pdk submodules

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.11.0
• Assignee: Travis Crawford Assign to me
• Reporter: Travis Crawford
• Created: 06/Apr/13 00:02
• Updated: 17/Jan/14 19:29
• Resolved: 20/Apr/13 17:41

Description

Moving from email. The builtins and pdk submodules are not believed to be in use and should be removed. The main benefits are simplification and maintainability of the Hive code base.

Forwarded conversation
Subject: builtins submodule - is it still needed?
------------------------

From: Travis Crawford <[e-mail]>
Date: Thu, Apr 4, 2013 at 2:01 PM
To: [e-mail], [e-mail]

Hey hive gurus -

Is the "builtins" hive submodule in use? The submodule was added in
HIVE-2523 as a location for builtin-UDFs, but it appears to not have
taken off. Any objections to removing it?

DETAILS

For HIVE-4278 I'm making some build changes for the HCatalog
integration. The "builtins" submodule causes issues because it delays
building until the packaging phase - so HCatalog can't depend on
builtins, which it does transitively.

While investigating a path forward I discovered the "builtins"
submodule contains very little code, and likely could either go away
entirely or merge into "ql", simplifying things both for users and
developers.

Thoughts? Can anyone with context help me understand "builtins", both
in general and around its non-standard build? For your trouble I'll
either make the submodule go away/merge into another submodule, or
update the docs with what we learn.

Thanks!
Travis

----------
From: Ashutosh Chauhan <[e-mail]>
Date: Fri, Apr 5, 2013 at 3:10 PM
To: [e-mail]
Cc: "[e-mail]" <[e-mail]>

I haven't used it myself anytime till now. Neither have met anyone who used
it or plan to use it.

Ashutosh

On Thu, Apr 4, 2013 at 2:01 PM, Travis Crawford <[e-mail]>wrote:

----------
From: Gunther Hagleitner <[e-mail]>
Date: Fri, Apr 5, 2013 at 3:11 PM
To: [e-mail]
Cc: [e-mail]

+1

I would actually go a step further and propose to remove both PDK and
builtins. I've went through the code for both and here is what I found:

Builtins:

• BuiltInUtils.java: Empty file
• UDAFUnionMap: Merges maps. Doesn't seem to be useful by itself, but was
  intended as a building block for PDK

PDK:

• some helper build.xml/test setup + teardown scripts
• Classes/annotations to help run unit tests
• rot13 as an example

From what I can tell it's a fair assessment that it hasn't taken off, last
commits to it seem to have happened more than 1.5 years ago.

Thanks,
Gunther.

On Thu, Apr 4, 2013 at 2:01 PM, Travis Crawford <[e-mail]>wrote:

----------
From: Owen O'Malley <[e-mail]>
Date: Fri, Apr 5, 2013 at 4:45 PM
To: [e-mail]

+1 to removing them.

We have a Rot13 example in ql/src/test/org/apache/hadoop/hive/ql/io/udf/Rot13

{In,Out}

putFormat.java anyways. smile

– Owen

Issue Links

• is related to: OOZIE-1665 Oozie 3.3.2 build error with Hive 0.11.0