Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1429

Druid adapter must send "fromNext" when requesting rows from Druid

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.11.0
    • Component/s: druid
    • Labels:
      None

      Description

      Druid adapter gets an error while fetching data. As mentioned in Druid documentation,

      Note that in the second query, an offset is specified and that it is 1 greater than the largest offset found in the initial results. To return the next "page", this offset must be incremented by 1 (should be decremented by 1 for descending query), with each new query, but with option fromNext enabled, this operation is not needed. When an empty results set is received, the very last page has been returned.

      I think Druid adapter presumes this feature is turned on but it's not set in code.

      Also, previousOffset may not be reasonable, see DruidQuery. I think whether we have already fetched all data is decided by the next request. If we don't have any data returned on next request, that means fetching is done.

        Activity

        Hide
        VcamX Jiarong Wei added a comment -
        Show
        VcamX Jiarong Wei added a comment - A pull request is sent: https://github.com/apache/calcite/pull/303
        Hide
        julianhyde Julian Hyde added a comment -

        Is there a straightforward test case that illustrates the problem that you are solving? It would make it easier to understand the API, and would prevent regressions.

        Show
        julianhyde Julian Hyde added a comment - Is there a straightforward test case that illustrates the problem that you are solving? It would make it easier to understand the API, and would prevent regressions.
        Hide
        VcamX Jiarong Wei added a comment -

        Thanks to the comprehensive tests for Druid adapter, I just need to update some existing test cases. I've updated the pull request. Please review it

        Show
        VcamX Jiarong Wei added a comment - Thanks to the comprehensive tests for Druid adapter, I just need to update some existing test cases. I've updated the pull request. Please review it
        Hide
        julianhyde Julian Hyde added a comment -
        Show
        julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/97ccd6de ; thanks for the PR, Jiarong Wei !
        Hide
        julianhyde Julian Hyde added a comment -

        Resolved in release 1.11.0 (2017-01-11).

        Show
        julianhyde Julian Hyde added a comment - Resolved in release 1.11.0 (2017-01-11).

          People

          • Assignee:
            julianhyde Julian Hyde
            Reporter:
            VcamX Jiarong Wei
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development