Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-10265

PutDatabaseRecord hangs and keeps file in queue when DB is unavailable instead of sending to fail

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 1.17.0
    • Extensions
    • None

    Description

      When PutDatabaseRecord encounters an unreachable DB (tested with MSSQL), it alerts at the processor and at the bulletin, but hangs on to the flowfile and keeps it in queue rather than routing it to failure.

       

      In Slack, exceptionfactory suggested: It looks like PutDatabaseRecord does not wrap the getConnection() call instead the general try-catch exception handling. This results in the behavior observed, keeping the file in the queue instead of sending the file to failure.

       

      This is the dump I'm getting:

      2022-07-20 11:36:17,498 ERROR [Timer-Driven Process Thread-98] o.a.n.p.standard.PutDatabaseRecord PutDatabaseRecord[id=d2123fa7-9cb0-1f8e-8e04-b30babb3bdf6] Processing failed
      org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException: Cannot create PoolableConnectionFactory (The TCP/IP connection to the host hostname, port 1433 has failed. Error: "connect timed out. Verify the connection
       properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".)
              at org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:667)
              at org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:644)
              at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:55)
              at jdk.internal.reflect.GeneratedMethodAccessor1028.invoke(Unknown Source)
              at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.base/java.lang.reflect.Method.invoke(Method.java:566)
              at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
              at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)
              at com.sun.proxy.$Proxy145.getConnection(Unknown Source)
              at org.apache.nifi.processors.standard.PutDatabaseRecord.onTrigger(PutDatabaseRecord.java:482)
              at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
              at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283)
              at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
              at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
              at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
              at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at java.base/java.lang.Thread.run(Thread.java:834)
      Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory (The TCP/IP connection to the host hostname, port 1433 has failed. Error: "connect timed out. Verify the connection properties. Make sure that an instance of
      SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".)
              at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:653)
              at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:531)
              at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:731)
              at org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:654)
              ... 20 common frames omitted
      Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host hostname, port 1433 has failed. Error: "connect timed out. Verify the connection properties. Make sure that an instance of SQL Server is r
      unning on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
              at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)
              at com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:241)
              at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2243)
              at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:491)
              at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1311)
              at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:993)
              at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:829)
              at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1012)
              at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:52)
              at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:374)
              at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:106)
              at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:649)
              ... 23 common frames omitted

      Attachments

        Issue Links

          Activity

            People

              exceptionfactory David Handermann
              stlaarons Aaron Schultz
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m