Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1424

Druid type is called FLOAT, not DOUBLE

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.11.0
    • Component/s: druid
    • Labels:
      None

      Description

      Currently the supported column types in Druid adapter are: LONG, DOUBLE, STRING and hyperUnique

      There's no clue to show Druid has DOUBLE as its column type. But instead it should be FLOAT as mentioned in Druid Documentation:

      Dimension columns will have type STRING. Metric columns will have type FLOAT or LONG or name of the underlying complex type such as hyperUnique in case of COMPLEX metric. Timestamp column will have type LONG.

      Fangjin Yang didn't mentioned it in related posts either. See 1 and 2

        Activity

        Hide
        julianhyde Julian Hyde added a comment -

        I think I was misled because Druid has aggregate functions such as doubleSum.

        For the record:

        • In SQL, the types DOUBLE and FLOAT are both 64-bit floating point, REAL is 32-bit floating point, and BIGINT is 64-bit signed integer.
        • In Java, double is 64 bit floating point, float is 32 bit floating point, long is 64 bit signed integer.

        Can you please clarify whether Druid's FLOAT type is 32-bit or 64-bit? I presume that Druid's LONG is 64-bit signed integer.

        Once we have clarified the content of Druid's types (how many bits, etc.), we should make sure that Calcite's Druid adapter surfaces columns with the appropriate SQL types for that content.

        Show
        julianhyde Julian Hyde added a comment - I think I was misled because Druid has aggregate functions such as doubleSum . For the record: In SQL, the types DOUBLE and FLOAT are both 64-bit floating point, REAL is 32-bit floating point, and BIGINT is 64-bit signed integer. In Java, double is 64 bit floating point, float is 32 bit floating point, long is 64 bit signed integer. Can you please clarify whether Druid's FLOAT type is 32-bit or 64-bit? I presume that Druid's LONG is 64-bit signed integer. Once we have clarified the content of Druid's types (how many bits, etc.), we should make sure that Calcite's Druid adapter surfaces columns with the appropriate SQL types for that content.
        Hide
        VcamX Jiarong Wei added a comment - - edited

        The doubleSum is 64-bit as mentioned in Aggregations section of Druid documentation. There're four types of column in Druid, see this. I think FLOAT corresponds to doubleSum and LONG corresponds to longSum. It's also proved in my tests.

        I've already sent a pull request for this: https://github.com/apache/calcite/pull/301
        Hope it'll help

        Show
        VcamX Jiarong Wei added a comment - - edited The doubleSum is 64-bit as mentioned in Aggregations section of Druid documentation. There're four types of column in Druid, see this . I think FLOAT corresponds to doubleSum and LONG corresponds to longSum . It's also proved in my tests. I've already sent a pull request for this: https://github.com/apache/calcite/pull/301 Hope it'll help
        Hide
        julianhyde Julian Hyde added a comment -

        Jiarong Wei, Thanks for the PR. Looks good. I fixed up a bit, creating a new enum DruidType rather than relying on a switch. Also, I used SQL's DOUBLE type rather than FLOAT; they're both the same, but we don't use FLOAT very much elsewhere in Calcite because people get confused and think it is 32 bit.

        Please review my changes in https://github.com/julianhyde/calcite/commit/3e4dd71f695a91060517f78dca834b1d7f225e67 and if you +1 I'll squash and commit after 1.10 is released.

        Show
        julianhyde Julian Hyde added a comment - Jiarong Wei , Thanks for the PR. Looks good. I fixed up a bit, creating a new enum DruidType rather than relying on a switch . Also, I used SQL's DOUBLE type rather than FLOAT; they're both the same, but we don't use FLOAT very much elsewhere in Calcite because people get confused and think it is 32 bit. Please review my changes in https://github.com/julianhyde/calcite/commit/3e4dd71f695a91060517f78dca834b1d7f225e67 and if you +1 I'll squash and commit after 1.10 is released.
        Hide
        VcamX Jiarong Wei added a comment -

        Yes, using DruidType is more reasonable and DOUBLE is less confusing!

        Show
        VcamX Jiarong Wei added a comment - Yes, using DruidType is more reasonable and DOUBLE is less confusing!
        Hide
        julianhyde Julian Hyde added a comment -
        Show
        julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/0f2d666d . Thanks for the PR, Jiarong Wei !
        Hide
        julianhyde Julian Hyde added a comment -

        Resolved in release 1.11.0 (2017-01-11).

        Show
        julianhyde Julian Hyde added a comment - Resolved in release 1.11.0 (2017-01-11).

          People

          • Assignee:
            julianhyde Julian Hyde
            Reporter:
            VcamX Jiarong Wei
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development