Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1959

Reduce the amount of Metadata and table name calls in Druid

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.14.0
    • Component/s: druid
    • Labels:
      None

      Description

      Currently in the Druid adapter, when a model definition file is not provided, query times can be quite slow due to excessive metadata and table names calls. Initial investigation reveals that a simple query like

      SELECT * from table
      

      produces at least 30 http calls for fetching the table names, and another 30 calls for meta data. This slows down the query considerably, even when the individual calls themselves are relatively quick.

      The source of the issue seems to be DruidSchema#getTableMap, as this method dispatches calls to both DruidConnectionImpl#metadata and DruidConnectionImpl#tableNames. In addition, DruidTable#create makes another call to DruidConnectionImpl#metadata even if it's given the populated fields as arguments.

      getTableMap is called fairly often indirectly from many places, including the validator, so as the query becomes larger, the problems gets a lot worse. Ideally, one would provide a model definition to prevent this, but doing so is not always possible/preferred for users.

        Attachments

          Activity

            People

            • Assignee:
              zhumayun Zain Humayun
              Reporter:
              zhumayun Zain Humayun
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: