Hive
  1. Hive
  2. HIVE-2655

Ability to define functions in HQL

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Ability to create functions in HQL as a substitute for creating them in Java.

      Jonathan Chang requested I create this issue.

      1. HIVE-2655-10.patch
        65 kB
        Brock Noland
      2. HIVE-2655-10.patch
        619 kB
        Brock Noland
      3. HIVE-2655-9.patch
        65 kB
        Brock Noland
      4. ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.4.patch
        25 kB
        Phabricator
      5. ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.3.patch
        23 kB
        Phabricator
      6. ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.2.patch
        23 kB
        Phabricator
      7. ASF.LICENSE.NOT.GRANTED--HIVE-2655.D915.1.patch
        14 kB
        Phabricator

        Issue Links

          Activity

          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
          Hide
          Brock Noland added a comment -

          Marking resolved as this was committed in 54ec1cb0d0540edf7946738bc113e90adcc09a6d.

          Show
          Brock Noland added a comment - Marking resolved as this was committed in 54ec1cb0d0540edf7946738bc113e90adcc09a6d.
          Hide
          Brock Noland added a comment -

          Hi,

          I wonder if it's an environmental issue? I cannot reproduce with the following command:

          ant clean package && ant test -Dtestcase=TestCliDriver -Dqfile=nonreserved_keywords_input37.q
          
          Show
          Brock Noland added a comment - Hi, I wonder if it's an environmental issue? I cannot reproduce with the following command: ant clean package && ant test -Dtestcase=TestCliDriver -Dqfile=nonreserved_keywords_input37.q
          Hide
          Edward Capriolo added a comment -

          Can you think of any reason why nonreserved_keywords_input37.q is failing?

          CREATE TABLE table(string string) STORED AS TEXTFILE;

          LOAD DATA LOCAL INPATH '../data/files/docurl.txt' INTO TABLE table;

          SELECT table, count(1)
          FROM
          (
          FROM table
          SELECT TRANSFORM (table.string)
          USING 'java -cp ../build/ql/test/classes org.apache.hadoop.hive.scripts.extracturl' AS (table, count)
          ) subq
          GROUP BY table;
          {

          Show
          Edward Capriolo added a comment - Can you think of any reason why nonreserved_keywords_input37.q is failing? CREATE TABLE table(string string) STORED AS TEXTFILE; LOAD DATA LOCAL INPATH '../data/files/docurl.txt' INTO TABLE table; SELECT table, count(1) FROM ( FROM table SELECT TRANSFORM (table.string) USING 'java -cp ../build/ql/test/classes org.apache.hadoop.hive.scripts.extracturl' AS (table, count) ) subq GROUP BY table; {
          Hide
          Brock Noland added a comment -

          Nice! Thank you Edward and Jonathon!

          Show
          Brock Noland added a comment - Nice! Thank you Edward and Jonathon!
          Hide
          Edward Capriolo added a comment -

          +1 Committed to trunk. Thanks Brock and Jonathon! Really cool.

          Show
          Edward Capriolo added a comment - +1 Committed to trunk. Thanks Brock and Jonathon! Really cool.
          Hide
          Edward Capriolo added a comment -

          Patch is clean. Running tests now.

          Show
          Edward Capriolo added a comment - Patch is clean. Running tests now.
          Hide
          Brock Noland added a comment -

          OK here is the correct patch, the previous one had the .orig files from the --merge.

          Show
          Brock Noland added a comment - OK here is the correct patch, the previous one had the .orig files from the --merge.
          Hide
          Brock Noland added a comment -

          Actually that latest patch is bad. Regenerating.

          Show
          Brock Noland added a comment - Actually that latest patch is bad. Regenerating.
          Hide
          Brock Noland added a comment -

          Sorry for the slow response. I have attached the rebased patch here and on https://reviews.facebook.net/D8673.

          FWIW, I ran the unit test suite and they all passed.

          Show
          Brock Noland added a comment - Sorry for the slow response. I have attached the rebased patch here and on https://reviews.facebook.net/D8673 . FWIW, I ran the unit test suite and they all passed.
          Hide
          Edward Capriolo added a comment -

          +1. Sorry for the delay. Patch does not apply clean now. Please rebase and I will commit.

          Show
          Edward Capriolo added a comment - +1. Sorry for the delay. Patch does not apply clean now. Please rebase and I will commit.
          Hide
          Brock Noland added a comment -

          Ping

          Show
          Brock Noland added a comment - Ping
          Hide
          Brock Noland added a comment -

          Carl Steinbach I did here https://reviews.facebook.net/D8673 and setup as a weblink not sure why it's not updating the JIRA automatically.

          Show
          Brock Noland added a comment - Carl Steinbach I did here https://reviews.facebook.net/D8673 and setup as a weblink not sure why it's not updating the JIRA automatically.
          Hide
          Carl Steinbach added a comment -

          @Brock: Can you open a new phabricator review request for your updated version of the patch? Thanks.

          Show
          Carl Steinbach added a comment - @Brock: Can you open a new phabricator review request for your updated version of the patch? Thanks.
          Hide
          Brock Noland added a comment -

          Marking as "Patch Available" since this is ready for a review.

          Show
          Brock Noland added a comment - Marking as "Patch Available" since this is ready for a review.
          Hide
          Brock Noland added a comment -

          Namit,

          Thanks for moving this to "Open". I should have noticed that. I took this patch over from Jonathan and have added a bunch of tests, error messages, and a little functionality. I will have a new patch posted in a day or two.

          Cheers!
          Brock

          Show
          Brock Noland added a comment - Namit, Thanks for moving this to "Open". I should have noticed that. I took this patch over from Jonathan and have added a bunch of tests, error messages, and a little functionality. I will have a new patch posted in a day or two. Cheers! Brock
          Hide
          Namit Jain added a comment -

          Can you add more tests ?

          Show
          Namit Jain added a comment - Can you add more tests ?
          Hide
          Phabricator added a comment -

          brock has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          Regarding

          • Defining the same macro twice.
          • Dropping a macro that doesn't exist.

          I assume the behavior should be the same as functions with allows both of these behaviors. As such I wonder why they were listed under negative tests?

          REVISION DETAIL
          https://reviews.facebook.net/D915

          BRANCH
          macro

          ARCANIST PROJECT
          hive

          To: JIRA, jsichi, cwsteinbach, jonchang
          Cc: jonchang, ikabiljo, brock

          Show
          Phabricator added a comment - brock has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". Regarding Defining the same macro twice. Dropping a macro that doesn't exist. I assume the behavior should be the same as functions with allows both of these behaviors. As such I wonder why they were listed under negative tests? REVISION DETAIL https://reviews.facebook.net/D915 BRANCH macro ARCANIST PROJECT hive To: JIRA, jsichi, cwsteinbach, jonchang Cc: jonchang, ikabiljo, brock
          Hide
          Brock Noland added a comment -

          Hi,

          OK great! Yes the patch required only minor rebasing. I'll work on the unit tests and then post the resulting patch here.

          Thanks!
          Brock

          Show
          Brock Noland added a comment - Hi, OK great! Yes the patch required only minor rebasing. I'll work on the unit tests and then post the resulting patch here. Thanks! Brock
          Hide
          Jonathan Chang added a comment -

          Yeah, I haven't had a chance to work on this. Looks like all that needs to be done at this point is unittests. I would be more than happy to have you take this the rest of the way!

          Show
          Jonathan Chang added a comment - Yeah, I haven't had a chance to work on this. Looks like all that needs to be done at this point is unittests. I would be more than happy to have you take this the rest of the way!
          Hide
          Brock Noland added a comment -

          Jonathan,

          I haven't seen any updates to this JIRA in a while. Are you still working on it? If not, would you mind if I took it forward?

          Brock

          Show
          Brock Noland added a comment - Jonathan, I haven't seen any updates to this JIRA in a while. Are you still working on it? If not, would you mind if I took it forward? Brock
          Hide
          Phabricator added a comment -

          cwsteinbach has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          INLINE COMMENTS
          ql/src/test/queries/clientpositive/macro.q:3 A couple more:

          • Reference the same macro twice in the same query
          • Reference two different macros in the same query

          REVISION DETAIL
          https://reviews.facebook.net/D915

          BRANCH
          macro

          Show
          Phabricator added a comment - cwsteinbach has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". INLINE COMMENTS ql/src/test/queries/clientpositive/macro.q:3 A couple more: Reference the same macro twice in the same query Reference two different macros in the same query REVISION DETAIL https://reviews.facebook.net/D915 BRANCH macro
          Hide
          Phabricator added a comment -

          cwsteinbach has requested changes to the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          INLINE COMMENTS
          ql/src/test/queries/clientpositive/macro.q:3 The test coverage for this feature needs to be expanded.

          What happens if the macro references a hiveconf or hivevar parameter?

          Missing positive coverage for:

          • Macro with zero input parameters
          • Macro with two input parameters
          • EXPLAIN output for a query using each variation of the above.
          • EXPLAIN EXTENDED output for a query using each variation of the above.

          Negative coverage for:

          • macro body references an undefined input parameter
          • input parameter not referenced in macro body
          • macro that defines two input parameters with the same name
          • Defining the same macro twice.
          • Droping a macro that doesn't exist.

          REVISION DETAIL
          https://reviews.facebook.net/D915

          BRANCH
          macro

          Show
          Phabricator added a comment - cwsteinbach has requested changes to the revision " HIVE-2655 [jira] Ability to define functions in HQL". INLINE COMMENTS ql/src/test/queries/clientpositive/macro.q:3 The test coverage for this feature needs to be expanded. What happens if the macro references a hiveconf or hivevar parameter? Missing positive coverage for: Macro with zero input parameters Macro with two input parameters EXPLAIN output for a query using each variation of the above. EXPLAIN EXTENDED output for a query using each variation of the above. Negative coverage for: macro body references an undefined input parameter input parameter not referenced in macro body macro that defines two input parameters with the same name Defining the same macro twice. Droping a macro that doesn't exist. REVISION DETAIL https://reviews.facebook.net/D915 BRANCH macro
          Hide
          Phabricator added a comment -

          ikabiljo has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          Cool, in the long run it might be good to give that option to all GenericUDFs, but this is good for this diff.

          Now you have to find somebody that can accept this

          P.S. Can't wait to use this

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - ikabiljo has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". Cool, in the long run it might be good to give that option to all GenericUDFs, but this is good for this diff. Now you have to find somebody that can accept this P.S. Can't wait to use this REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          jonchang has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          INLINE COMMENTS
          ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java:66-72 See above @ 1106 and 1113 of FunctionRegistry in this diff.

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - jonchang has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java:66-72 See above @ 1106 and 1113 of FunctionRegistry in this diff. REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          ikabiljo has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          INLINE COMMENTS
          ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java:66-72 I don't think this works for GenericUDF ?
          FunctionRegistry looks like is only looking at annotations.

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - ikabiljo has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java:66-72 I don't think this works for GenericUDF ? FunctionRegistry looks like is only looking at annotations. REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          jonchang updated the revision "HIVE-2655 [jira] Ability to define functions in HQL".
          Reviewers: JIRA, jsichi

          handle stateful/deterministic

          REVISION DETAIL
          https://reviews.facebook.net/D915

          AFFECTED FILES
          ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluator.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
          ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateMacroDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/DropMacroDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/FunctionWork.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java
          ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java
          ql/src/test/queries/clientpositive/macro.q
          ql/src/test/results/clientpositive/macro.q.out

          Show
          Phabricator added a comment - jonchang updated the revision " HIVE-2655 [jira] Ability to define functions in HQL". Reviewers: JIRA, jsichi handle stateful/deterministic REVISION DETAIL https://reviews.facebook.net/D915 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeEvaluator.java ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateMacroDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/DropMacroDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/FunctionWork.java ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java ql/src/test/queries/clientpositive/macro.q ql/src/test/results/clientpositive/macro.q.out
          Hide
          Phabricator added a comment -

          jonchang has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          To accept diffs you have to become a committer. I think it's a pretty arduous process =/.

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - jonchang has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". To accept diffs you have to become a committer. I think it's a pretty arduous process =/. REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          ikabiljo has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          Looks good, see inline.
          What do I have to do to have power to accept hive diffs?

          INLINE COMMENTS
          ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java:43 why stateful? (is that related to serialization?)
          also it should probably be undeterministic since it can expres undeterministic function.
          later you can make it check body to see if it is deterministic, and than do what is needed in initialize. But no need.

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - ikabiljo has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". Looks good, see inline. What do I have to do to have power to accept hive diffs? INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java:43 why stateful? (is that related to serialization?) also it should probably be undeterministic since it can expres undeterministic function. later you can make it check body to see if it is deterministic, and than do what is needed in initialize. But no need. REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          jonchang has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          pingping

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - jonchang has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". pingping REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          jonchang updated the revision "HIVE-2655 [jira] Ability to define functions in HQL".
          Reviewers: JIRA, jsichi

          Oops, for some reason an extra file made its way in there.

          REVISION DETAIL
          https://reviews.facebook.net/D915

          AFFECTED FILES
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
          ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateMacroDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/DropMacroDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/FunctionWork.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java
          ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java
          ql/src/test/queries/clientpositive/macro.q
          ql/src/test/results/clientpositive/macro.q.out

          Show
          Phabricator added a comment - jonchang updated the revision " HIVE-2655 [jira] Ability to define functions in HQL". Reviewers: JIRA, jsichi Oops, for some reason an extra file made its way in there. REVISION DETAIL https://reviews.facebook.net/D915 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateMacroDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/DropMacroDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/FunctionWork.java ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java ql/src/test/queries/clientpositive/macro.q ql/src/test/results/clientpositive/macro.q.out
          Hide
          Phabricator added a comment -

          jonchang updated the revision "HIVE-2655 [jira] Ability to define functions in HQL".
          Reviewers: JIRA, jsichi

          Everything works now. Merry xmas!

          REVISION DETAIL
          https://reviews.facebook.net/D915

          AFFECTED FILES
          data/files/complex.seq
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
          ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateMacroDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/DropMacroDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/FunctionWork.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java
          ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java
          ql/src/test/queries/clientpositive/macro.q
          ql/src/test/results/clientpositive/macro.q.out

          Show
          Phabricator added a comment - jonchang updated the revision " HIVE-2655 [jira] Ability to define functions in HQL". Reviewers: JIRA, jsichi Everything works now. Merry xmas! REVISION DETAIL https://reviews.facebook.net/D915 AFFECTED FILES data/files/complex.seq ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateMacroDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/DropMacroDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/FunctionWork.java ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMacro.java ql/src/test/queries/clientpositive/macro.q ql/src/test/results/clientpositive/macro.q.out
          Hide
          Phabricator added a comment -

          jonchang has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          INLINE COMMENTS
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java:105 Yes. Hence work in progress =).

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - jonchang has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java:105 Yes. Hence work in progress =). REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          ikabiljo has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          This is awesome!
          And looks quite short.

          INLINE COMMENTS
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java:105 don't you need to at least register function here?

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - ikabiljo has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". This is awesome! And looks quite short. INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java:105 don't you need to at least register function here? REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          jonchang has commented on the revision "HIVE-2655 [jira] Ability to define functions in HQL".

          This is a work in progress. Can't test right now because builtins are not loading for me in the test suite.

          How does this general framework look though?

          REVISION DETAIL
          https://reviews.facebook.net/D915

          Show
          Phabricator added a comment - jonchang has commented on the revision " HIVE-2655 [jira] Ability to define functions in HQL". This is a work in progress. Can't test right now because builtins are not loading for me in the test suite. How does this general framework look though? REVISION DETAIL https://reviews.facebook.net/D915
          Hide
          Phabricator added a comment -

          jonchang requested code review of "HIVE-2655 [jira] Ability to define functions in HQL".
          Reviewers: JIRA

          HIVE-2655

          Ability to create functions in HQL as a substitute for creating them in Java.

          Jonathan Chang requested I create this issue.

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D915

          AFFECTED FILES
          ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
          ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateMacroDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/FunctionWork.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/DropMacroDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java
          ql/src/test/queries/clientpositive/macro.q

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/1917/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - jonchang requested code review of " HIVE-2655 [jira] Ability to define functions in HQL". Reviewers: JIRA HIVE-2655 Ability to create functions in HQL as a substitute for creating them in Java. Jonathan Chang requested I create this issue. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D915 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/MacroSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateMacroDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/FunctionWork.java ql/src/java/org/apache/hadoop/hive/ql/plan/DropMacroDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java ql/src/test/queries/clientpositive/macro.q MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1917/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.

            People

            • Assignee:
              Brock Noland
              Reporter:
              Jonathan Perlow
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development