Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5274

Exception thrown in Drillbit shutdown in UDF cleanup code

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.10.0
    • 1.10.0
    • None

    Description

      I ran a very simple query: a single-line text file in an embedded Drillbit. The UDF directory was placed in /tmp. During the run, the directory was deleted. On Drillbit shutdown, the following occurred:

      25328 DEBUG [main] [org.apache.drill.exec.server.Drillbit] - Shutdown begun.
      26344 INFO [pool-1-thread-2] [org.apache.drill.exec.rpc.data.DataServer] - closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7d1c0d85 in 1007 ms
      26345 INFO [pool-1-thread-1] [org.apache.drill.exec.rpc.user.UserServer] - closed eventLoopGroup io.netty.channel.nio.NioEventLoopGroup@7cdb3b56 in 1008 ms
      26345 INFO [pool-1-thread-1] [org.apache.drill.exec.service.ServiceEngine] - closed userServer in 1009 ms
      26345 INFO [pool-1-thread-2] [org.apache.drill.exec.service.ServiceEngine] - closed dataPool in 1009 ms
      26356 WARN [main] [org.apache.drill.exec.server.Drillbit] - Failure on close()
      java.lang.IllegalArgumentException: /tmp/drill/udf/udf/local does not exist
      	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1637) ~[commons-io-2.4.jar:2.4]
      	at org.apache.drill.exec.expr.fn.FunctionImplementationRegistry.close(FunctionImplementationRegistry.java:469) ~[classes/:na]
      	at org.apache.drill.exec.server.DrillbitContext.close(DrillbitContext.java:209) ~[classes/:na]
      	at org.apache.drill.exec.work.WorkManager.close(WorkManager.java:152) ~[classes/:na]
      	at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) ~[classes/:na]
      	at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) ~[classes/:na]
      	at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:171) ~[classes/:na]
      ...
      

      The following patch makes the problem go away, but I'm not sure if the above is an indication of deeper problems.

      public class FunctionImplementationRegistry implements FunctionLookupContext, AutoCloseable {
        ...
        public void close() {
          if (deleteTmpDir) {
            ...
          } else {
            try {
              File dir = new File(localUdfDir.toUri().getPath());
              if (dir.exists()) {
                FileUtils.cleanDirectory(dir);
              }
            ...
          }
        }
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            arina Arina Ielchiieva
            paul-rogers Paul Rogers
            Paul Rogers Paul Rogers
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment