Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5184

Error result of compareSerialized in RowComparator class

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.0, 1.1.4
    • Component/s: Table API & SQL
    • Labels:
      None

      Description

      RowSerializer will write null mask for all fields in a row before serialize row data to DataOutputView.

      RowSerializer.scala
      override def serialize(value: Row, target: DataOutputView) {
          val len = fieldSerializers.length
      
          if (value.productArity != len) {
            throw new RuntimeException("Row arity of value does not match serializers.")
          }
      
          // write a null mask
          writeNullMask(len, value, target)
      
      ......
      }
      
      

      RowComparator will deserialize a row data from DataInputView when call compareSerialized method. However, the first parameter value of readIntoNullMask method is wrong, which should be the count of all fields, rather than the length of serializers (to deserialize the first n fields for comparison).

      RowComparator.scala
      override def compareSerialized(firstSource: DataInputView, secondSource: DataInputView): Int = {
          val len = serializers.length
          val keyLen = keyPositions.length
      
          readIntoNullMask(len, firstSource, nullMask1)
          readIntoNullMask(len, secondSource, nullMask2)
      ......
      }
      

        Issue Links

          Activity

          Hide
          fhueske Fabian Hueske added a comment -

          Fixed for 1.2.0 with ecfb5b5f6fd6bf1555c7240d77dd9aca982f4416
          Fixed for 1.1.4 with 0758d0be60647f09f0254092c05601d6698ae90f

          Show
          fhueske Fabian Hueske added a comment - Fixed for 1.2.0 with ecfb5b5f6fd6bf1555c7240d77dd9aca982f4416 Fixed for 1.1.4 with 0758d0be60647f09f0254092c05601d6698ae90f
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/2894

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/2894
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on the issue:

          https://github.com/apache/flink/pull/2894

          Merging

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on the issue: https://github.com/apache/flink/pull/2894 Merging
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on the issue:

          https://github.com/apache/flink/pull/2894

          Thanks for fixing this bug @godfreyhe!
          +1 to merge

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on the issue: https://github.com/apache/flink/pull/2894 Thanks for fixing this bug @godfreyhe! +1 to merge
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user godfreyhe opened a pull request:

          https://github.com/apache/flink/pull/2894

          FLINK-5184 fix bug: Error result of compareSerialized in RowComparator class

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/godfreyhe/flink master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/2894.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2894


          commit e6c728025ec7bcfefe62a7130c2ce8c1716ffdd6
          Author: godfreyhe <godfreyhe@163.com>
          Date: 2016-11-29T11:27:58Z

          fix bug: Error result of compareSerialized in RowComparator class


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user godfreyhe opened a pull request: https://github.com/apache/flink/pull/2894 FLINK-5184 fix bug: Error result of compareSerialized in RowComparator class You can merge this pull request into a Git repository by running: $ git pull https://github.com/godfreyhe/flink master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/2894.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2894 commit e6c728025ec7bcfefe62a7130c2ce8c1716ffdd6 Author: godfreyhe <godfreyhe@163.com> Date: 2016-11-29T11:27:58Z fix bug: Error result of compareSerialized in RowComparator class

            People

            • Assignee:
              godfreyhe godfrey he
              Reporter:
              godfreyhe godfrey he
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development