Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4810 DECIMAL datatype changes for Impala 2.9
  3. IMPALA-2020

Rounding should be done instead of truncating when casting DECIMAL to DECIMAL, FLOAT/DOUBLE to DECIMAL, DECIMAL to INT

    Details

      Description

      The current behavior is to truncate which people don't expect and often differs from what other databases do (does any other system truncate?). We've had several questions from users that stem from the truncation.

        Issue Links

          Activity

          Hide
          cchanning Chris Channing added a comment -

          I spoke to Juan Yu regarding this issue, she pointed me at an internal JIRA with Alexander Behm's recommendation, namely:

          "We could add a configurable parameter (query option or impalad option) that would control Impala's FLOAT/DOUBLE to STRING CAST behavior with respect to rounding. Currently we simply do not round. We could choose a "reasonable" default value, e.g., round to two digits or whatever."

          I'm in favour of a query option to resolve this which would be defaulted to reasonable value if not specified.

          Show
          cchanning Chris Channing added a comment - I spoke to Juan Yu regarding this issue, she pointed me at an internal JIRA with Alexander Behm 's recommendation, namely: "We could add a configurable parameter (query option or impalad option) that would control Impala's FLOAT/DOUBLE to STRING CAST behavior with respect to rounding. Currently we simply do not round. We could choose a "reasonable" default value, e.g., round to two digits or whatever." I'm in favour of a query option to resolve this which would be defaulted to reasonable value if not specified.
          Hide
          caseyc casey added a comment -

          I don't think that will be needed.

          Show
          caseyc casey added a comment - I don't think that will be needed.
          Hide
          caseyc casey added a comment -

          If you want to add the option that's fine too. If you do, can you also run a benchmark to see how much overhead that adds?

          Show
          caseyc casey added a comment - If you want to add the option that's fine too. If you do, can you also run a benchmark to see how much overhead that adds?
          Hide
          cchanning Chris Channing added a comment -

          Yes, no problem. I'll catch up with you offline about that.

          Show
          cchanning Chris Channing added a comment - Yes, no problem. I'll catch up with you offline about that.
          Hide
          cchanning Chris Channing added a comment -

          After speaking to Alexander Behm about this issue in more detail, we decided that it would be logical to examine what other database vendors are doing in terms of rounding values. In order to observe the behaviour of each database, I put together a small set of “sanity test casts”, namely:

          Generic Sanity Test Cases:

          Test # Test
          1 string:1.666666 -> double
          2 string:1.666666 -> decimal
          3 string:1.666666 -> decimal(10,2)
          4 string:1.663666 -> decimal(10,2)
          5 float:1.666666 -> double
          6 float:1.666666 -> decimal
          7 float:1.666666 -> decimal(10, 2)
          8 double:1.666666 -> decimal
          9 double:1.666666 -> decimal(10, 2)
          10 double:1.666666 -> float
          11 double:1.666666 -> string(255)
          12 double:1.66666 -> string(4)
          13 double:1.66666 -> string(1)
          14 float:1.66666 -> string(4)

          The databases tested were:

          • Impala (trunk)
          • Postgresql 9.3
          • Oracle 11g
          • MySQL 5.5
          • SQL Server 2012

          The table below lists the test results:

          Database Test Query Result
          Impala [cdh5-trunk] 1 select cast('1.666666' as double) 1.666666
            2 select cast('1.666666' as decimal) 1
            3 select cast('1.666666' as decimal(10,2)) 1.66
            4 select cast('1.663666' as decimal(10,2)) 1.66
            5 select cast(cast(1.666666 as float) as double) 1.666666030883789
            6 select cast(cast(1.666666 as float) as decimal) 1
            7 select cast(cast(1.666666 as float) as decimal(10,2)) 1.66
            8 select cast(cast(1.666666 as double) as decimal) 1
            9 select cast(cast(1.666666 as double) as decimal(10,2)) 1.66
            10 select cast(cast(1.666666 as double) as float) 1.666666030883789
            11 select cast(cast(1.666666 as double) as varchar(255)) 1.666666
            12 select cast(cast(1.666666 as double) as varchar(4)) 1.66
            13 select cast(cast(1.666666 as double) as varchar(1)) 1
            14 select cast(cast(1.666666 as float) as varchar(4)) 1.66
                 
          Postgresql [9.3] 1 select cast('1.666666' as double precision) 1.666666
            2 select cast('1.666666' as decimal) 1.666666
            3 select cast('1.666666' as decimal(10,2)) 1.67
            4 select cast('1.663666' as decimal(10,2)) 1.66
            5 select cast(cast(1.666666 as float) as double precision) 1.666666
            6 select cast(cast(1.666666 as float) as decimal) 1.666666
            7 select cast(cast(1.666666 as float) as decimal(10,2)) 1.67
            8 select cast(cast(1.666666 as double precision) as decimal) 1.666666
            9 select cast(cast(1.666666 as double precision) as decimal(10,2)) 1.67
            10 select cast(cast(1.666666 as double precision) as float 1.666666
            11 select cast(cast(1.666666 as double precision) as varchar(255)) 1.666666
            12 select cast(cast(1.666666 as double precision) as varchar(4)) 1.66
            13 select cast(cast(1.666666 as double precision) as varchar(1)) 1
            14 select cast(cast(1.666666 as float) as varchar(4)) 1.66
                 
          Oracle [11.2.0.4] 1 select cast('1.666666' as double precision) as "cast_col" from foo 1.666666
            2 select cast('1.666666' as decimal) as "cast_col" from foo 2
            3 select cast('1.666666' as decimal(10,2)) as "cast_col" from foo 1.67
            4 select cast('1.663666' as decimal(10,2)) as "cast_col" from foo 1.66
            5 select cast(cast(1.666666 as float) as double precision) as "cast_col" from foo 1.666666
            6 select cast(cast(1.666666 as float) as decimal) as "cast_col" from foo 2
            7 select cast(cast(1.666666 as float) as decimal(10, 2)) as "cast_col" from foo 1.67
            8 select cast(cast(1.666666 as double precision) as decimal) as "cast_col" from foo 2
            9 select cast(cast(1.666666 as double precision) as decimal(10, 2)) as "cast_col" from foo 1.67
            10 select cast(cast(1.666666 as double precision) as float) as "cast_col" from foo 1.666666
            11 select cast(cast(1.666666 as double precision) as varchar(255)) as "cast_col" from foo 1.666666
            12 select cast(cast(1.666666 as double precision) as varchar(4)) as "cast_col" from foo 1.67
            13 select cast(cast(1.666666 as double precision) as varchar(1)) as "cast_col" from foo 2
            14 select cast(cast(1.666666 as float) as varchar(4)) as "cast_col" from foo 1.67
                 
          MySQL [5.5.43] 1 N/A  
            2 select cast('1.666666' as decimal) 2
            3 select cast('1.666666' as decimal(10,2)) 1.67
            4 select cast('1.663666' as decimal(10,2)) 1.66
            5 N/A  
            6 N/A  
            7 N/A  
            8 select cast(1.666666 as decimal) 2
            9 select cast(1.666666 as decimal(10, 2)) 1.67
            10 N/A  
            11 select cast(1.666666 as char(255)) 1.666666
            12 select cast(1.666666 as char(4)) 1.66
            13 select cast(1.666666 as char(1)) 1
            14 N/A  
                 
          SQL Server SE 2012 [11.00.2100.60.v1] 1 select cast('1.666666' as float(54)) 1.666666
            2 select cast('1.666666' as decimal) 2
            3 select cast('1.666666' as decimal(10,2)) 1.67
            4 select cast('1.663666' as decimal(10,2)) 1.66
            5 select cast(cast(1.666666 as float) as float(54)) 1.666666
            6 select cast(cast(1.666666 as float) as decimal) 2
            7 select cast(cast(1.666666 as float) as decimal(10,2)) 1.67
            8 select cast(cast(1.666666 as float(54)) as decimal) 2
            9 select cast(cast(1.666666 as float(54)) as decimal(10,2)) 1.67
            10 select cast(cast(1.666666 as float(54)) as float 1.666666
            11 select cast(cast(1.666666 as float(54)) as varchar(255)) 1.666667
            12 select cast(cast(1.666666 as float(54)) as varchar(4)) Error: Overflow
            13 select cast(cast(1.666666 as float(54)) as varchar(1)) Error: Overflow
            14 select cast(cast(1.666666 as float) as varchar(4)) Error: Overflow

          The results clearly indicate that there are differences between databases, particularly when casting from numeric to string types. Is there a preference on what behaviour we’d like to mimic here?

          Show
          cchanning Chris Channing added a comment - After speaking to Alexander Behm about this issue in more detail, we decided that it would be logical to examine what other database vendors are doing in terms of rounding values. In order to observe the behaviour of each database, I put together a small set of “sanity test casts”, namely: Generic Sanity Test Cases: Test # Test 1 string:1.666666 -> double 2 string:1.666666 -> decimal 3 string:1.666666 -> decimal(10,2) 4 string:1.663666 -> decimal(10,2) 5 float:1.666666 -> double 6 float:1.666666 -> decimal 7 float:1.666666 -> decimal(10, 2) 8 double:1.666666 -> decimal 9 double:1.666666 -> decimal(10, 2) 10 double:1.666666 -> float 11 double:1.666666 -> string(255) 12 double:1.66666 -> string(4) 13 double:1.66666 -> string(1) 14 float:1.66666 -> string(4) The databases tested were: Impala (trunk) Postgresql 9.3 Oracle 11g MySQL 5.5 SQL Server 2012 The table below lists the test results: Database Test Query Result Impala [cdh5-trunk] 1 select cast('1.666666' as double) 1.666666   2 select cast('1.666666' as decimal) 1   3 select cast('1.666666' as decimal(10,2)) 1.66   4 select cast('1.663666' as decimal(10,2)) 1.66   5 select cast(cast(1.666666 as float) as double) 1.666666030883789   6 select cast(cast(1.666666 as float) as decimal) 1   7 select cast(cast(1.666666 as float) as decimal(10,2)) 1.66   8 select cast(cast(1.666666 as double) as decimal) 1   9 select cast(cast(1.666666 as double) as decimal(10,2)) 1.66   10 select cast(cast(1.666666 as double) as float) 1.666666030883789   11 select cast(cast(1.666666 as double) as varchar(255)) 1.666666   12 select cast(cast(1.666666 as double) as varchar(4)) 1.66   13 select cast(cast(1.666666 as double) as varchar(1)) 1   14 select cast(cast(1.666666 as float) as varchar(4)) 1.66         Postgresql [9.3] 1 select cast('1.666666' as double precision) 1.666666   2 select cast('1.666666' as decimal) 1.666666   3 select cast('1.666666' as decimal(10,2)) 1.67   4 select cast('1.663666' as decimal(10,2)) 1.66   5 select cast(cast(1.666666 as float) as double precision) 1.666666   6 select cast(cast(1.666666 as float) as decimal) 1.666666   7 select cast(cast(1.666666 as float) as decimal(10,2)) 1.67   8 select cast(cast(1.666666 as double precision) as decimal) 1.666666   9 select cast(cast(1.666666 as double precision) as decimal(10,2)) 1.67   10 select cast(cast(1.666666 as double precision) as float 1.666666   11 select cast(cast(1.666666 as double precision) as varchar(255)) 1.666666   12 select cast(cast(1.666666 as double precision) as varchar(4)) 1.66   13 select cast(cast(1.666666 as double precision) as varchar(1)) 1   14 select cast(cast(1.666666 as float) as varchar(4)) 1.66         Oracle [11.2.0.4] 1 select cast('1.666666' as double precision) as "cast_col" from foo 1.666666   2 select cast('1.666666' as decimal) as "cast_col" from foo 2   3 select cast('1.666666' as decimal(10,2)) as "cast_col" from foo 1.67   4 select cast('1.663666' as decimal(10,2)) as "cast_col" from foo 1.66   5 select cast(cast(1.666666 as float) as double precision) as "cast_col" from foo 1.666666   6 select cast(cast(1.666666 as float) as decimal) as "cast_col" from foo 2   7 select cast(cast(1.666666 as float) as decimal(10, 2)) as "cast_col" from foo 1.67   8 select cast(cast(1.666666 as double precision) as decimal) as "cast_col" from foo 2   9 select cast(cast(1.666666 as double precision) as decimal(10, 2)) as "cast_col" from foo 1.67   10 select cast(cast(1.666666 as double precision) as float) as "cast_col" from foo 1.666666   11 select cast(cast(1.666666 as double precision) as varchar(255)) as "cast_col" from foo 1.666666   12 select cast(cast(1.666666 as double precision) as varchar(4)) as "cast_col" from foo 1.67   13 select cast(cast(1.666666 as double precision) as varchar(1)) as "cast_col" from foo 2   14 select cast(cast(1.666666 as float) as varchar(4)) as "cast_col" from foo 1.67         MySQL [5.5.43] 1 N/A     2 select cast('1.666666' as decimal) 2   3 select cast('1.666666' as decimal(10,2)) 1.67   4 select cast('1.663666' as decimal(10,2)) 1.66   5 N/A     6 N/A     7 N/A     8 select cast(1.666666 as decimal) 2   9 select cast(1.666666 as decimal(10, 2)) 1.67   10 N/A     11 select cast(1.666666 as char(255)) 1.666666   12 select cast(1.666666 as char(4)) 1.66   13 select cast(1.666666 as char(1)) 1   14 N/A           SQL Server SE 2012 [11.00.2100.60.v1] 1 select cast('1.666666' as float(54)) 1.666666   2 select cast('1.666666' as decimal) 2   3 select cast('1.666666' as decimal(10,2)) 1.67   4 select cast('1.663666' as decimal(10,2)) 1.66   5 select cast(cast(1.666666 as float) as float(54)) 1.666666   6 select cast(cast(1.666666 as float) as decimal) 2   7 select cast(cast(1.666666 as float) as decimal(10,2)) 1.67   8 select cast(cast(1.666666 as float(54)) as decimal) 2   9 select cast(cast(1.666666 as float(54)) as decimal(10,2)) 1.67   10 select cast(cast(1.666666 as float(54)) as float 1.666666   11 select cast(cast(1.666666 as float(54)) as varchar(255)) 1.666667   12 select cast(cast(1.666666 as float(54)) as varchar(4)) Error: Overflow   13 select cast(cast(1.666666 as float(54)) as varchar(1)) Error: Overflow   14 select cast(cast(1.666666 as float) as varchar(4)) Error: Overflow The results clearly indicate that there are differences between databases, particularly when casting from numeric to string types. Is there a preference on what behaviour we’d like to mimic here?
          Hide
          caseyc casey added a comment -

          If you'd like to come up with a recommendation that would be great. I took a quick look, imo what Oracle does looks pretty good.

          Show
          caseyc casey added a comment - If you'd like to come up with a recommendation that would be great. I took a quick look, imo what Oracle does looks pretty good.
          Hide
          cchanning Chris Channing added a comment -

          Agreed, I'll mimic Oracles behaviour.

          Show
          cchanning Chris Channing added a comment - Agreed, I'll mimic Oracles behaviour.
          Hide
          alex.behm Alexander Behm added a comment -

          +1 for Oracle behavior.

          Show
          alex.behm Alexander Behm added a comment - +1 for Oracle behavior.
          Hide
          jbeard Jeremy Beard added a comment -

          1.666666 might not be the only value to test as some rounding behavior differs when the rounding digit is 5. For example, Teradata rounds either up or down on a rounding digit 5 depending on whether the previous digit to that is even or odd (??): http://www.info.teradata.com/HTMLPubs/DB_TTU_14_00/index.html#page/SQL_Reference/B035_1143_111A/ch03.042.65.html

          Show
          jbeard Jeremy Beard added a comment - 1.666666 might not be the only value to test as some rounding behavior differs when the rounding digit is 5. For example, Teradata rounds either up or down on a rounding digit 5 depending on whether the previous digit to that is even or odd (??): http://www.info.teradata.com/HTMLPubs/DB_TTU_14_00/index.html#page/SQL_Reference/B035_1143_111A/ch03.042.65.html
          Hide
          srus Silvius Rus added a comment -

          Not tracking as a blocker even though it's technically a correctness issue based on impact.

          Show
          srus Silvius Rus added a comment - Not tracking as a blocker even though it's technically a correctness issue based on impact.
          Hide
          dhecht Dan Hecht added a comment -

          In the first step, let's implement an option to make these casts do rounding. We'll want to code gen the runtime option away (using IMPALA-4809).

          From Type To Type
          FLOAT/DOUBLE DECIMAL
          DECIMAL DECIMAL
          DECIMAL INT
          DECIMAL TIMESTAMP

          These ones don't require any changes:

          From Type To Type Reason
          INT types DECIMAL Never loss of scale
          TIMESTAMP DECIMAL Invalid cast
          BOOL DECIMAL Invalid cast
          DECIMAL STRING Never loss of scale
          DECIMAL BOOL Compared with 0

          Let's save these for a later step:

          From Type To Type
          STRING DECIMAL
          DECIMAL FLOAT/DOUBLE
          DECIMAL VARCHAR/CHAR
          Show
          dhecht Dan Hecht added a comment - In the first step, let's implement an option to make these casts do rounding. We'll want to code gen the runtime option away (using IMPALA-4809 ). From Type To Type FLOAT/DOUBLE DECIMAL DECIMAL DECIMAL DECIMAL INT DECIMAL TIMESTAMP These ones don't require any changes: From Type To Type Reason INT types DECIMAL Never loss of scale TIMESTAMP DECIMAL Invalid cast BOOL DECIMAL Invalid cast DECIMAL STRING Never loss of scale DECIMAL BOOL Compared with 0 Let's save these for a later step: From Type To Type STRING DECIMAL DECIMAL FLOAT/DOUBLE DECIMAL VARCHAR/CHAR
          Hide
          tarmstrong Tim Armstrong added a comment -

          A user reported another manifestation of this rounding problem in IMPALA-4626 - converting decimals of different precision/scale to double produces inconsistent results.

          Show
          tarmstrong Tim Armstrong added a comment - A user reported another manifestation of this rounding problem in IMPALA-4626 - converting decimals of different precision/scale to double produces inconsistent results.
          Hide
          tarmstrong Tim Armstrong added a comment -

          Although I think that may be a consequence of the cast being implemented in a way that implements neither truncation or rounding correctly - I think it loses even more precision than that:

          template<typename T>
          inline double DecimalValue<T>::ToDouble(int scale) const {
            return static_cast<double>(value_) / powf(10.0, scale);
          } 
          
          Show
          tarmstrong Tim Armstrong added a comment - Although I think that may be a consequence of the cast being implemented in a way that implements neither truncation or rounding correctly - I think it loses even more precision than that: template<typename T> inline double DecimalValue<T>::ToDouble( int scale) const { return static_cast< double >(value_) / powf(10.0, scale); }
          Hide
          zamsden_impala_ad21 Zachary added a comment -

          Do we really care about rounding in casts to TIMESTAMP? It only matters if the scale being used is > 9, which is better than nanosecond precision. I don't think it really matters whether we round or not there.

          Show
          zamsden_impala_ad21 Zachary added a comment - Do we really care about rounding in casts to TIMESTAMP? It only matters if the scale being used is > 9, which is better than nanosecond precision. I don't think it really matters whether we round or not there.
          Show
          zamsden_impala_ad21 Zachary added a comment - https://gerrit.cloudera.org/#/c/5951/
          Hide
          zamsden_impala_ad21 Zachary added a comment -

          Still some more casts to cover

          Show
          zamsden_impala_ad21 Zachary added a comment - Still some more casts to cover
          Hide
          jbapple Jim Apple added a comment -

          Removed fix version since not fixed yet

          Show
          jbapple Jim Apple added a comment - Removed fix version since not fixed yet
          Hide
          dhecht Dan Hecht added a comment -

          Let's split the remaining cases into a separate JIRA (IMPALA-5014) so it's easier to keep track of which Impala versions got which changes. This JIRA will be for:

          From Type To Type
          DECIMAL DECIMAL
          FLOAT/DOUBLE DECIMAL
          DECIMAL INT

          Which was addressed in these two commits:

          commit f982c3f76e762f646b52a01659796b4edcfbc1ad
          Author: Michael Ho <kwho@cloudera.com>
          Date:   Fri Feb 3 20:55:18 2017 -0800
          
              IMPALA-2020, IMPALA-4809: Codegen support for DECIMAL_V2
          
              Currently, codegen supports converting type attributes (e.g.
              decimal type's precision and scale, type's size) obtained via
              calls to FunctionContextImpl::GetFnAttr() (previously
              Expr::GetConstantInt()) in cross-compiled code to runtime constants.
              This change extends this support for the query option DECIMAL_V2.
          
              To test this change, this change also handles a subset of IMPALA-2020:
              casting between decimal values is  updated to support rounding (instead
              of truncation) when decimal_v2 is true.
          
              This change also cleans up the existing code by moving the codegen
              logic Expr::InlineConstant() to the codegen module and the type
              related logic in Expr::GetConstantInt() to FunctionContextImpl.
          
              Change-Id: I2434d240f65b81389b8a8ba027f980a0e1d1f981
              Reviewed-on: http://gerrit.cloudera.org:8080/5950
              Reviewed-by: Michael Ho <kwho@cloudera.com>
              Tested-by: Impala Public Jenkins
          

          and

          commit d2c540f1e3d63eea39311a03f1ee8741042312eb
          Author: Zach Amsden <zamsden@cloudera.com>
          Date:   Thu Feb 9 03:23:07 2017 +0000
          
              IMPALA-2020, 4915, 4936: Add rounding for decimal casts
          
              This change adds support for DECIMAL_V2 rounding behavior for both
              DECIMAL to INT and DOUBLE to DECIMAL casts.  The round behavior
              implemented for exact halves is round halves away from zero (e.g
              (0.5 -> 1) and (-0.5 -> -1)).
          
              This change also fixes two open bugs regarding overflow checking.
              The root cause of both behaviors turned out to be the same - a
              missing std:: caused the wrong abs() function to be used.  Due
              to details of IEEE floating point representation, this actually
              masked another bug, as NaN is often represented as all 1-bits,
              which fails the overflow test.  Since the implicit conversion to
              int lost precision, we ended up storing large numbers that don't
              actually represent valid decimal numbers in the range when the
              value happened to be +/- Infinity.  This caused the rendering
              back to ASCII characters to go awry, but is otherwise harmless.
          
              Testing: Added expr-test and decimal-test test coverage as well as
              manual testing.  I tried to update the expr benchmark to get some
              kind of results but the benchmark is pretty bit-rotted.  It was
              throwing JNI exceptions.  Fixed up the JNI init call, but there is
              still a lot of work to do to get this back in a runnable state.
              Even with the hack to get at the RuntimeContext, we end up getting
              null derefs due to the slot descriptor table not being initialized.
          <snip>
              Change-Id: I2daf186b4770a022f9cb349d512067a1dd624810
              Reviewed-on: http://gerrit.cloudera.org:8080/5951
              Reviewed-by: Dan Hecht <dhecht@cloudera.com>
              Tested-by: Impala Public Jenkins
          
          Show
          dhecht Dan Hecht added a comment - Let's split the remaining cases into a separate JIRA ( IMPALA-5014 ) so it's easier to keep track of which Impala versions got which changes. This JIRA will be for: From Type To Type DECIMAL DECIMAL FLOAT/DOUBLE DECIMAL DECIMAL INT Which was addressed in these two commits: commit f982c3f76e762f646b52a01659796b4edcfbc1ad Author: Michael Ho <kwho@cloudera.com> Date: Fri Feb 3 20:55:18 2017 -0800 IMPALA-2020, IMPALA-4809: Codegen support for DECIMAL_V2 Currently, codegen supports converting type attributes (e.g. decimal type's precision and scale, type's size) obtained via calls to FunctionContextImpl::GetFnAttr() (previously Expr::GetConstantInt()) in cross-compiled code to runtime constants. This change extends this support for the query option DECIMAL_V2. To test this change, this change also handles a subset of IMPALA-2020: casting between decimal values is updated to support rounding (instead of truncation) when decimal_v2 is true . This change also cleans up the existing code by moving the codegen logic Expr::InlineConstant() to the codegen module and the type related logic in Expr::GetConstantInt() to FunctionContextImpl. Change-Id: I2434d240f65b81389b8a8ba027f980a0e1d1f981 Reviewed-on: http: //gerrit.cloudera.org:8080/5950 Reviewed-by: Michael Ho <kwho@cloudera.com> Tested-by: Impala Public Jenkins and commit d2c540f1e3d63eea39311a03f1ee8741042312eb Author: Zach Amsden <zamsden@cloudera.com> Date: Thu Feb 9 03:23:07 2017 +0000 IMPALA-2020, 4915, 4936: Add rounding for decimal casts This change adds support for DECIMAL_V2 rounding behavior for both DECIMAL to INT and DOUBLE to DECIMAL casts. The round behavior implemented for exact halves is round halves away from zero (e.g (0.5 -> 1) and (-0.5 -> -1)). This change also fixes two open bugs regarding overflow checking. The root cause of both behaviors turned out to be the same - a missing std:: caused the wrong abs() function to be used. Due to details of IEEE floating point representation, this actually masked another bug, as NaN is often represented as all 1-bits, which fails the overflow test. Since the implicit conversion to int lost precision, we ended up storing large numbers that don't actually represent valid decimal numbers in the range when the value happened to be +/- Infinity. This caused the rendering back to ASCII characters to go awry, but is otherwise harmless. Testing: Added expr-test and decimal-test test coverage as well as manual testing. I tried to update the expr benchmark to get some kind of results but the benchmark is pretty bit-rotted. It was throwing JNI exceptions. Fixed up the JNI init call, but there is still a lot of work to do to get this back in a runnable state. Even with the hack to get at the RuntimeContext, we end up getting null derefs due to the slot descriptor table not being initialized. <snip> Change-Id: I2daf186b4770a022f9cb349d512067a1dd624810 Reviewed-on: http: //gerrit.cloudera.org:8080/5951 Reviewed-by: Dan Hecht <dhecht@cloudera.com> Tested-by: Impala Public Jenkins

            People

            • Assignee:
              zamsden Zach Amsden
              Reporter:
              caseyc casey
            • Votes:
              4 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development