Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5507

Millions of "Failure finding Drillbit running on host" info messages in foreman logs

    Details

      Description

      When foreman tries to execute a query with data files that reside on nodes other than the nodes that are running drillbits, there are millions of messages in the log files like this:

      2017-05-10 13:22:10,916 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp0535.mydomain.com. Skipping affinity to that host.

      After 200 MB of these messages, 960K of them and 7 seconds of planning, it continues to execute this query. Is there a way to disable / suppress these messages? What's causing this? Should this be printed with INFO logging as we're seeing or only with DEBUG logging? Is there a way to turn off this check? Below is a snippet from foreman logs:

      2017-05-10 13:22:10,916 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp0535.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1402.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp0929.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1098.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1230.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1388.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp0535.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1291.mydomain.com. Skipping affinity to that host.

      The hostname and domain are scrubbed.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                timothyfarkas Timothy Farkas
                Reporter:
                veeran Veera Naranammalpuram
                Reviewer:
                Paul Rogers
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: