Hive
  1. Hive
  2. HIVE-656

Add a PMOD (POSITIVE_MOD) function

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      HIVE-656. Add a PMOD (POSITIVE MOD) function. (Scott Chen via zshao)

      Description

      There are a lot of cases people want to get a positive modulo result.
      For example, people want to bucket the data into 10 buckets. They use the hash code in Hive (based on Java) which can return a negative number. Then they need this POSITIVE_MOD(a, b) to return the results.

      Otherwise they can still do it but it will be very verbose:

      ((hash(xxx) % 10) + 10) % 10
      
      1. hive-656-2009-07-27-2.patch
        6 kB
        Scott Chen
      2. hive-656-2009-07-27-1.patch
        6 kB
        Scott Chen
      3. hive-656-2009-07-27.patch
        6 kB
        Scott Chen

        Activity

        Hide
        Scott Chen added a comment -

        Thank you very much Zheng.

        Show
        Scott Chen added a comment - Thank you very much Zheng.
        Hide
        Zheng Shao added a comment -

        I overwrote the output for show_functions and committed it. Thanks Scott.

        Show
        Zheng Shao added a comment - I overwrote the output for show_functions and committed it. Thanks Scott.
        Hide
        Namit Jain added a comment -

        There is a problem with the current tests - drop function is not called at the end of some tests - so, the output depends on the order of execution.
        You may need to regenerate the patch after https://issues.apache.org/jira/browse/HIVE-700

        Show
        Namit Jain added a comment - There is a problem with the current tests - drop function is not called at the end of some tests - so, the output depends on the order of execution. You may need to regenerate the patch after https://issues.apache.org/jira/browse/HIVE-700
        Hide
        Scott Chen added a comment -

        sorry about the stupid mistake. show_functions.q.out is included in the diff now.

        Show
        Scott Chen added a comment - sorry about the stupid mistake. show_functions.q.out is included in the diff now.
        Hide
        Scott Chen added a comment -

        Thanks. Now I see what's going on.
        I run the test with ant "test -Doverwrite=true -Dtest.silent=false". And it built successfully.

        But than when I uploaded the diff. I manually removed show_function.q.out in the diff because I thought it is from my last change in hive-580. I should have also included that file.

        I will soon upload the new diff.

        Show
        Scott Chen added a comment - Thanks. Now I see what's going on. I run the test with ant "test -Doverwrite=true -Dtest.silent=false". And it built successfully. But than when I uploaded the diff. I manually removed show_function.q.out in the diff because I thought it is from my last change in hive-580. I should have also included that file. I will soon upload the new diff.
        Hide
        Zheng Shao added a comment -

        Did all the tests pass?

        If you run the tests with "-Doverwrite=true", you should see a modified file in ql/src/test/results.
        Then when you do "svn diff", that modified file should be included.

        Show
        Zheng Shao added a comment - Did all the tests pass? If you run the tests with "-Doverwrite=true", you should see a modified file in ql/src/test/results. Then when you do "svn diff", that modified file should be included.
        Hide
        Scott Chen added a comment -

        changed the name to PMOD. have also rerun all the tests.

        Show
        Scott Chen added a comment - changed the name to PMOD. have also rerun all the tests.
        Hide
        Scott Chen added a comment -

        @Namit: I did run all the test before I uploaded. This is weird. I will run it again and figure out. Thanks.

        @Zheng: Good idea. That's a better name. I will change it.

        Show
        Scott Chen added a comment - @Namit: I did run all the test before I uploaded. This is weird. I will run it again and figure out. Thanks. @Zheng: Good idea. That's a better name. I will change it.
        Hide
        Zheng Shao added a comment -

        Talked with Alex Smith offline and he suggests to use a little cryptic name "PMOD", which will help simplify the queries.
        I would vote for "PMOD". (It should be good enough to just change the line in FunctionRegistry.java that register the function)
        Thoughts?

        Show
        Zheng Shao added a comment - Talked with Alex Smith offline and he suggests to use a little cryptic name "PMOD", which will help simplify the queries. I would vote for "PMOD". (It should be good enough to just change the line in FunctionRegistry.java that register the function) Thoughts?
        Hide
        Namit Jain added a comment -

        did you run all the tests ? I think there is a test which does 'show functions' which will break and needs to be updated

        Show
        Namit Jain added a comment - did you run all the tests ? I think there is a test which does 'show functions' which will break and needs to be updated
        Hide
        Scott Chen added a comment -

        Add positive_mod to udf. A new class UDFPosMod is added and registered to FunctionRegistry. The new class is just a slightly modified version of UDFOPMod.

        Show
        Scott Chen added a comment - Add positive_mod to udf. A new class UDFPosMod is added and registered to FunctionRegistry. The new class is just a slightly modified version of UDFOPMod.
        Hide
        Scott Chen added a comment -

        working on it now.

        Show
        Scott Chen added a comment - working on it now.
        Hide
        Zheng Shao added a comment -

        MySQL mod function (which returns negative numbers when the first number is negative) http://dev.mysql.com/doc/refman/5.0/en/mathematical-functions.html#function_mod

        Show
        Zheng Shao added a comment - MySQL mod function (which returns negative numbers when the first number is negative) http://dev.mysql.com/doc/refman/5.0/en/mathematical-functions.html#function_mod

          People

          • Assignee:
            Scott Chen
            Reporter:
            Zheng Shao
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development