Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-875

Don't try to access the package during containerStart

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.10.1
    • Fix Version/s: None
    • Component/s: container
    • Labels:
      None

      Description

      When the job is submitted using run-job.sh the package file is given to YARN. The job is the accepted, the container is created, the package is unpacked and is ready to execute.

      However, the startContainer method (ContainerUtil:159) then tries to access the original package file.

          try {
            fileStatus = packagePath.getFileSystem(yarnConfiguration).getFileStatus(packagePath);
          } catch (IOException ioe) {
            log.error("IO Exception when accessing the package status from the filesystem", ioe);
            throw new SamzaException("IO Exception when accessing the package status from the filesystem");
          }
      

      It wants to do it just to set the length of the file and the modification time to the resource:

          packageResource.setSize(fileStatus.getLen());
          packageResource.setTimestamp(fileStatus.getModificationTime());
      

      If these attributes (length and timestamp) are really needed then I think they could be captured and submitted by run-job.sh which would allow to avoid this issue.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              alexeyraga Alexey Raga
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: