Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4501

[C++] Unique returns non-unique strings

    XMLWordPrintableJSON

    Details

      Description

      Calling Unique on  e. g. {"some long string data","some long string data","other data"} returns

      dictionary with "some long string data" appearing twice. This is caused by off by 1 error in DoubleCrcHash, which caused it to read 1 byte past the end of the strings with length  higher than 16, and not divisible by 4. In such cases, we never hash p[0], and we always read one extra byte.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tocarip Ilya Tokar
                Reporter:
                tocarip Ilya Tokar
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m