Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-10097

Camel aws-s3 Timeout waiting for connection from pool

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • 2.15.1, 2.17.1
    • None
    • camel-aws

    Description

      I'm a beginner to apache camel framework and i'm using 2.17.1 version. My use case is to periodically sync newly created files in s3 to local folder in linux in java, for which i'm using camel aws-s3 component.

      I'm using deleteAfterRead=false, not to delete files in s3 after download and idempotentConsumer for not to poll and download duplicate files again and again.

      RouteBuilder
      from("aws-s3://" + Constants.BUCKET_NAME + "?amazonS3Client=#client&deleteAfterRead=false")
      .idempotentConsumer(header("CamelAwsS3Key"), FileIdempotentRepository.fileIdempotentRepository(new File("data", "repo.dat")))
      .routeId("awsFileCopyRoute")
      .log(LoggingLevel.INFO, "awsFileCopyRoute", "Camel AWS Key: ${header.CamelAwsS3Key}")
      .to("file://" + Constants.REQ_FOLDER + "?fileName=${header.CamelAwsS3Key}");

      In applicationContext.xml, i'm setting amazonS3Client's client bean as follows
      <bean id="defaultAWSCredentialsProviderChain" class="com.amazonaws.auth.DefaultAWSCredentialsProviderChain" />
      <bean id="clientConfiguration" class="com.amazonaws.ClientConfiguration" />

      <bean id="client" class="com.amazonaws.services.s3.AmazonS3Client">
      <constructor-arg ref="defaultAWSCredentialsProviderChain" />
      <constructor-arg ref="clientConfiguration" />
      </bean>

      It's working first time downloading non-duplicate files from s3 to the REQ_FOLDER, but the issue is after 1 min, i've been getting following exception repeatedly.

      2016-06-28 10:15:13,884 INFO AmazonHttpClient.executeHelper:638 - Unable to execute HTTP request: Timeout waiting for connection from pool
      org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
      at org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:230)
      at org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:199)
      at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)
      at com.amazonaws.http.conn.$Proxy44.getConnection(Unknown Source)
      at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:424)
      at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
      at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:860)
      at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:631)
      at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:400)
      at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:362)
      at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:311)
      at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3673)
      at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3626)
      at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:693)
      at org.apache.camel.component.aws.s3.S3Consumer.poll(S3Consumer.java:87)
      at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)
      at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            acosentino Andrea Cosentino
            raghavan aravamudhan Raghavan Aravamudhan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment