Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-4630

Reverse scan with row_timestamp does not working

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 4.13.0, 4.13.1, 4.13.2, 4.13.2-cdh
    • Fix Version/s: 4.14.0
    • Labels:
      None

      Description

      In version 4.13, if the query plan is a reverse scan, an error or incorrect data is returned.

      This is a problem that occurs when the query plan is a reverse range scan in the case of an "order by desc" query for the row key.

      The table schem is as follows.

      create table if not exists app_log (
        app_tag varchar not null,
        timestamp date not null,
        uuid varchar not null,
        log varchar
        constraint pk primary key(app_tag, timestamp row_timestamp, uuid)
      )
      data_block_encoding='FAST_DIFF',
      compression='LZ4',
      update_cache_frequency=600000,
      column_encoded_bytes = 1,
      ttl=2592000,
      salt_buckets=50
      ;
      

      The current data is as follows.

      upsert into app_log values ('test', now(), 'test', 'test');
      ...
      
      select * from app_log order by timestamp;
      +-----------+--------------------------+-------+-------+
      |  APP_TAG  |        TIMESTAMP         | UUID  |  LOG  |
      +-----------+--------------------------+-------+-------+
      | test      | 2018-02-28 01:02:16.985  | test  | test  |
      | test      | 2018-02-28 01:02:19.472  | test  | test  |
      | test      | 2018-02-28 01:02:21.568  | test  | test  |
      | test      | 2018-02-28 01:02:23.332  | test  | test  |
      | test      | 2018-02-28 01:02:25.200  | test  | test  |
      | test      | 2018-02-28 01:02:27.055  | test  | test  |
      | test      | 2018-02-28 01:02:29.008  | test  | test  |
      | test      | 2018-02-28 01:02:30.911  | test  | test  |
      | test      | 2018-02-28 01:02:32.775  | test  | test  |
      | test      | 2018-02-28 01:02:34.663  | test  | test  |
      +-----------+--------------------------+-------+-------+
      

      You can see errors if you run a simple query after adding some data.

      Depending on the data, an error may occur and incorrect data may be output.

      select * from app_log where app_tag = 'test' and timestamp between to_date('2018-02-28 01:02:16') and to_date('2018-02-28 01:02:34') order by timestamp desc;
      
      Error: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: APP_LOG,\x0D\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1519778082466.6dd30a7d7a26a38c5c06d63008bbff3d.: seekToPreviousRow must not be called on a non-reversed scanner
      at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:96)
      at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:62)
      at org.apache.phoenix.iterate.RegionScannerFactory$1.nextRaw(RegionScannerFactory.java:212)
      at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:82)
      at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:82)
      at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:293)
      at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2561)
      at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)
      at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2183)
      at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
      at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
      at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
      Caused by: org.apache.commons.lang.NotImplementedException: seekToPreviousRow must not be called on a non-reversed scanner
      at org.apache.hadoop.hbase.regionserver.NonReversedNonLazyKeyValueScanner.seekToPreviousRow(NonReversedNonLazyKeyValueScanner.java:44)
      at org.apache.hadoop.hbase.regionserver.ReversedKeyValueHeap.seekToPreviousRow(ReversedKeyValueHeap.java:89)
      at org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.nextRow(ReversedRegionScannerImpl.java:71)
      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:5938)
      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5673)
      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5659)
      at org.apache.phoenix.iterate.RegionScannerFactory$1.nextRaw(RegionScannerFactory.java:175)
      ... 9 more (state=08000,code=101)
      

       Query plan is as follow.

      explain select * from app_log where app_tag = 'test' and timestamp between to_date('2018-02-28 01:02:20') and to_date('2018-02-28 01:02:30') order by timestamp desc;
      +------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+
      |                                                                            PLAN                                                                            | EST_BYTES_READ  | EST_ROWS_READ  | EST_INFO_TS  |
      +------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+
      | CLIENT 14-CHUNK PARALLEL 14-WAY REVERSE RANGE SCAN OVER APP_LOG [0,'test','2018-02-28 01:02:20.000'] - [49,'test','2018-02-28 01:02:30.000']  | null            | null           | null         |
      |     ROW TIMESTAMP FILTER [1519779740000, 1519779750001)                                                                                                    | null            | null           | null         |
      | CLIENT MERGE SORT                                                                                                                                          | null            | null           | null         |
      +------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+
      

      Notice that the query plan is REVERSE RANGE SCAN.

      Refer to my comment on PHOENIX-4622 for causes and solutions.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                comnetwork chenglei
                Reporter:
                mini666 JeongMin Ju
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: