Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4237

Incorrect results when materializing a decimal in a data source scan node.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.3.0, Impala 2.5.0, Impala 2.4.0, Impala 2.6.0, Impala 2.7.0
    • Fix Version/s: Impala 2.6.3, Impala 2.8.0
    • Component/s: Backend
    • Labels:

      Description

      When scanning a DECIMAL-typed column with precision <= 9 a data source scan node may return incorrect results.

      The reason is that 8 bytes are accidentally written to a 4 byte slot because of a missing break in a switch. See data-source-scan-node.cc:

      ...
      inline Status SetDecimalVal(const ColumnType& type, char* bytes, int len,
          void* slot) {
        uint8_t* buffer = reinterpret_cast<uint8_t*>(bytes);
        switch (type.GetByteSize()) {
          case 4: {
            Decimal4Value* val = reinterpret_cast<Decimal4Value*>(slot);
            if (UNLIKELY(len > sizeof(Decimal4Value) ||
                ParquetPlainEncoder::Decode(buffer, buffer + len, len, val) < 0)) {
              return Status(ERROR_INVALID_DECIMAL);
            } <-- missing break, falling through to case 8
          }
          case 8: {
            Decimal8Value* val = reinterpret_cast<Decimal8Value*>(slot);
            if (UNLIKELY(len > sizeof(Decimal8Value) ||
                ParquetPlainEncoder::Decode(buffer, buffer + len, len, val) < 0)) {
              return Status(ERROR_INVALID_DECIMAL);
            }
            break;
          }
      ...
      

        Activity

        Hide
        alex.behm Alexander Behm added a comment -

        commit d9dc9090ea45220dc07dda3e171574828c1c0828
        Author: Alex Behm <alex.behm@cloudera.com>
        Date: Fri Sep 30 21:47:24 2016 -0700

        IMPALA-4237: Fix materialization of 4 byte decimals in data source scan node.

        There was a missing break in a switch statement leading to bad fallthrough.

        An existing test already expected incorrect results. The bug is covered by
        expecting correct results.

        Change-Id: I5340c2eda813afc032ba72203bd59eb3f2c4f482
        Reviewed-on: http://gerrit.cloudera.org:8080/4585
        Reviewed-by: Alex Behm <alex.behm@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        alex.behm Alexander Behm added a comment - commit d9dc9090ea45220dc07dda3e171574828c1c0828 Author: Alex Behm <alex.behm@cloudera.com> Date: Fri Sep 30 21:47:24 2016 -0700 IMPALA-4237 : Fix materialization of 4 byte decimals in data source scan node. There was a missing break in a switch statement leading to bad fallthrough. An existing test already expected incorrect results. The bug is covered by expecting correct results. Change-Id: I5340c2eda813afc032ba72203bd59eb3f2c4f482 Reviewed-on: http://gerrit.cloudera.org:8080/4585 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Internal Jenkins

          People

          • Assignee:
            alex.behm Alexander Behm
            Reporter:
            alex.behm Alexander Behm
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development