Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4146

Concurrent queries hang in planner in ReflectiveRelMetadataProvider

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      At concurrency levels of 30 or more for certain workloads we have seen queries hang in the planning phase in Calcite. The top of the jstack is shown below:

      "29b47a17-6ef3-4b7f-98e7-a7c1a702c32f:foreman" daemon prio=10 tid=0x00007f55484a1800 nid=0x289a runnable [0x00007f54b4369000]
         java.lang.Thread.State: RUNNABLE
              at java.util.HashMap.getEntry(HashMap.java:465)
              at java.util.HashMap.get(HashMap.java:417)
              at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider.apply(ReflectiveRelMetadataProvider.java:251)
              at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider.apply(ChainedRelMetadataProvider.java:60)
              at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider.apply(ChainedRelMetadataProvider.java:60)
      

      After some investigations, we found that this issue was actually addressed by CALCITE-874 (ReflectiveRelMetadataProvider is not thread-safe). This JIRA is a placeholder to merge that Calcite fix since Drill is currently not up-to-date with Calcite and there is an immediate need for running queries in a high concurrency environment.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amansinha100 Aman Sinha
            amansinha100 Aman Sinha
            Dechang Gu Dechang Gu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment