Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4304

Remove unused builtins and pdk submodules

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.11.0
    • None
    • None

    Description

      Moving from email. The builtins and pdk submodules are not believed to be in use and should be removed. The main benefits are simplification and maintainability of the Hive code base.

      Forwarded conversation
      Subject: builtins submodule - is it still needed?
      ------------------------

      From: Travis Crawford <traviscrawford@gmail.com>
      Date: Thu, Apr 4, 2013 at 2:01 PM
      To: user@hive.apache.org, dev@hive.apache.org

      Hey hive gurus -

      Is the "builtins" hive submodule in use? The submodule was added in
      HIVE-2523 as a location for builtin-UDFs, but it appears to not have
      taken off. Any objections to removing it?

      DETAILS

      For HIVE-4278 I'm making some build changes for the HCatalog
      integration. The "builtins" submodule causes issues because it delays
      building until the packaging phase - so HCatalog can't depend on
      builtins, which it does transitively.

      While investigating a path forward I discovered the "builtins"
      submodule contains very little code, and likely could either go away
      entirely or merge into "ql", simplifying things both for users and
      developers.

      Thoughts? Can anyone with context help me understand "builtins", both
      in general and around its non-standard build? For your trouble I'll
      either make the submodule go away/merge into another submodule, or
      update the docs with what we learn.

      Thanks!
      Travis

      ----------
      From: Ashutosh Chauhan <ashutosh.chauhan@gmail.com>
      Date: Fri, Apr 5, 2013 at 3:10 PM
      To: dev@hive.apache.org
      Cc: "user@hive.apache.org" <user@hive.apache.org>

      I haven't used it myself anytime till now. Neither have met anyone who used
      it or plan to use it.

      Ashutosh

      On Thu, Apr 4, 2013 at 2:01 PM, Travis Crawford <traviscrawford@gmail.com>wrote:

      ----------
      From: Gunther Hagleitner <ghagleitner@hortonworks.com>
      Date: Fri, Apr 5, 2013 at 3:11 PM
      To: dev@hive.apache.org
      Cc: user@hive.apache.org

      +1

      I would actually go a step further and propose to remove both PDK and
      builtins. I've went through the code for both and here is what I found:

      Builtins:

      • BuiltInUtils.java: Empty file
      • UDAFUnionMap: Merges maps. Doesn't seem to be useful by itself, but was
        intended as a building block for PDK

      PDK:

      • some helper build.xml/test setup + teardown scripts
      • Classes/annotations to help run unit tests
      • rot13 as an example

      From what I can tell it's a fair assessment that it hasn't taken off, last
      commits to it seem to have happened more than 1.5 years ago.

      Thanks,
      Gunther.

      On Thu, Apr 4, 2013 at 2:01 PM, Travis Crawford <traviscrawford@gmail.com>wrote:

      ----------
      From: Owen O'Malley <omalley@apache.org>
      Date: Fri, Apr 5, 2013 at 4:45 PM
      To: user@hive.apache.org

      +1 to removing them.

      We have a Rot13 example in ql/src/test/org/apache/hadoop/hive/ql/io/udf/Rot13

      {In,Out}

      putFormat.java anyways. smile

      – Owen

      Attachments

        1. HIVE-4304.1.patch
          60 kB
          Gunther Hagleitner
        2. HIVE-4304.patch
          65 kB
          Travis Crawford

        Issue Links

          Activity

            People

              traviscrawford Travis Crawford
              traviscrawford Travis Crawford
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: