Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5828

BlobServer create cache dir has concurrency safety problem

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.2.1, 1.3.0
    • None
    • None

    Description

      Caused by: java.lang.RuntimeException: org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Could not upload the jar files to the job manager.
      at FlinkJob_20170217_161058_000004.bind(FlinkJob_20170217_161058_000004.java:45)
      at com.aliyun.kepler.rc.query.schedule.FlinkQueryJob.call(FlinkQueryJob.java:53)
      at com.aliyun.kepler.rc.query.schedule.FlinkQueryJob.call(FlinkQueryJob.java:13)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.AbstractExecutorService$2.run(AbstractExecutorService.java:120)
      ... 3 common frames omitted
      Caused by: org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Could not upload the jar files to the job manager.
      at com.aliyun.kepler.rc.flink.client.Client.runBlocking(Client.java:178)
      at org.apache.flink.api.java.ClientEnvironment.execute(ClientEnvironment.java:169)
      at org.apache.flink.api.java.ClientEnvironment.execute(ClientEnvironment.java:225)
      at FlinkJob_20170217_161058_000004.bind(FlinkJob_20170217_161058_000004.java:42)
      ... 7 common frames omitted
      Caused by: org.apache.flink.runtime.client.JobSubmissionException: Could not upload the jar files to the job manager.
      at org.apache.flink.runtime.client.JobClientActor$2.call(JobClientActor.java:359)
      at akka.dispatch.Futures$$anonfun$future$1.apply(Future.scala:94)
      at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
      at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
      at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
      ... 3 common frames omitted
      Caused by: java.io.IOException: Could not retrieve the JobManager's blob port.
      at org.apache.flink.runtime.blob.BlobClient.uploadJarFiles(BlobClient.java:706)
      at org.apache.flink.runtime.jobgraph.JobGraph.uploadUserJars(JobGraph.java:556)
      at org.apache.flink.runtime.client.JobClientActor$2.call(JobClientActor.java:357)
      ... 7 common frames omitted
      Caused by: java.io.IOException: PUT operation failed: Server side error: Could not create cache directory '/home/kepler/kepler3012/data/work/blobs/blobStore-c3566cb2-b3d6-40ae-bdcf-594a81c8881b/cache'.
      at org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:476)
      at org.apache.flink.runtime.blob.BlobClient.put(BlobClient.java:338)
      at org.apache.flink.runtime.blob.BlobClient.uploadJarFiles(BlobClient.java:730)
      at org.apache.flink.runtime.blob.BlobClient.uploadJarFiles(BlobClient.java:701)
      ... 9 common frames omitted

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            bowen.zheng ZhengBowen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment