Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3112

Partial row scan not handled correctly

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 4.7.0
    • 4.12.0
    • None
    • None

    Description

      When doing a select of a relatively large table (a few touthands rows) some rows return partially missing.
      When increasing the fitler to return those specific rows, the values appear as expected

      CREATE TABLE IF NOT EXISTS TEST (
              BUCKET VARCHAR,
              TIMESTAMP_DATE TIMESTAMP,
              TIMESTAMP UNSIGNED_LONG NOT NULL,
              SRC VARCHAR,
              DST VARCHAR,
              ID VARCHAR,
              ION VARCHAR,
              IC BOOLEAN NOT NULL,
              MI UNSIGNED_LONG,
              AV UNSIGNED_LONG,
              MA UNSIGNED_LONG,
              CNT UNSIGNED_LONG,
              DUMMY VARCHAR
          CONSTRAINT pk PRIMARY KEY (BUCKET, TIMESTAMP DESC, SRC, DST, ID, ION, IC)
      );

      using a python script to generate a CSV with 5000 rows

      for i in xrange(5000):
          print "5SEC,2016-07-21 07:25:35.{i},146908593500{i},WWWWWWWW,AAA,BBBB,CCCCCCCC,false,{i}1181000,1788000{i},2497001{i},{i},aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa{i}".format(i=i)
      

      bulk inserting the csv in the table

      phoenix/bin/psql.py localhost -t TEST large.csv
      

      here we can see one row that contains no TIMESTAMP_DATE and null values in MI and MA

      0: jdbc:phoenix:localhost:2181> select * from TEST 
      ....
      +---------+--------------------------+-------------------+-----------+------+-------+-----------+--------+--------------+--------------+--------------+-------+----------------------------------------------------------------------------+
      | BUCKET  |      TIMESTAMP_DATE      |     TIMESTAMP     |    SRC    | DST  |  ID   |    ION    |   IC   |      MI      |      AV      |      MA      |  CNT  |                                   DUMMY                                    |
      +---------+--------------------------+-------------------+-----------+------+-------+-----------+--------+--------------+--------------+--------------+-------+----------------------------------------------------------------------------+
      | 5SEC    | 2016-07-21 07:25:35.100  | 1469085935001000  | WWWWWWWW  | AAA  | BBBB  | CCCCCCCC  | false  | 10001181000  | 17880001000  | 24970011000  | 1000  | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa1000  |
      | 5SEC    | 2016-07-21 07:25:35.999  | 146908593500999   | WWWWWWWW  | AAA  | BBBB  | CCCCCCCC  | false  | 9991181000   | 1788000999   | 2497001999   | 999   | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa999   |
      | 5SEC    | 2016-07-21 07:25:35.998  | 146908593500998   | WWWWWWWW  | AAA  | BBBB  | CCCCCCCC  | false  | 9981181000   | 1788000998   | 2497001998   | 998   | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa998   |
      | 5SEC    |                          | 146908593500997   | WWWWWWWW  | AAA  | BBBB  | CCCCCCCC  | false  | null         | 1788000997   | null         | 997   |                                                                            |
      | 5SEC    | 2016-07-21 07:25:35.996  | 146908593500996   | WWWWWWWW  | AAA  | BBBB  | CCCCCCCC  | false  | 9961181000   | 1788000996   | 2497001996   | 996   | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa996   |
      | 5SEC    | 2016-07-21 07:25:35.995  | 146908593500995   | WWWWWWWW  | AAA  | BBBB  | CCCCCCCC  | false  | 9951181000   | 1788000995   | 2497001995   | 995   | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa995   |
      | 5SEC    | 2016-07-21 07:25:35.994  | 146908593500994   | WWWWWWWW  | AAA  | BBBB  | CCCCCCCC  | false  | 9941181000   | 1788000994   | 2497001994   | 994   | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa994   |
      ....
      

      but when selecting that row specifically the values are correct

      0: jdbc:phoenix:localhost:2181> select * from TEST where timestamp = 146908593500997;
      +---------+--------------------------+------------------+-----------+------+-------+-----------+--------+-------------+-------------+-------------+------+---------------------------------------------------------------------------+
      | BUCKET  |      TIMESTAMP_DATE      |    TIMESTAMP     |    SRC    | DST  |  ID   |    ION    |   IC   |     MI      |     AV      |     MA      | CNT  |                                   DUMMY                                   |
      +---------+--------------------------+------------------+-----------+------+-------+-----------+--------+-------------+-------------+-------------+------+---------------------------------------------------------------------------+
      | 5SEC    | 2016-07-21 07:25:35.997  | 146908593500997  | WWWWWWWW  | AAA  | BBBB  | CCCCCCCC  | false  | 9971181000  | 1788000997  | 2497001997  | 997  | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa997  |
      +---------+--------------------------+------------------+-----------+------+-------+-----------+--------+-------------+-------------+-------------+------+---------------------------------------------------------------------------+
      1 row selected (0.159 seconds)

      Attachments

        1. PHOENIX-3112-v6.patch
          33 kB
          Sergey Soldatov
        2. PHOENIX-3112-ssa-v5.patch
          23 kB
          Sergey Soldatov
        3. PHOENIX-3112-ssa-v4.patch
          23 kB
          Sergey Soldatov
        4. PHOENIX-3112-1.patch
          18 kB
          Sergey Soldatov
        5. PHOENIX-3112_wip2.patch
          15 kB
          James R. Taylor
        6. PHOENIX-3112_v3.patch
          11 kB
          James R. Taylor

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sergey.soldatov Sergey Soldatov
            pierre.lacave Pierre Lacave
            Votes:
            3 Vote for this issue
            Watchers:
            20 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment