Uploaded image for project: 'Hadoop Submarine'
  1. Hadoop Submarine
  2. SUBMARINE-15

[Submarine] Job should not be submitted if "--input_path" option is missing

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.2.0
    • Component/s: None
    • Labels:
      None

      Description

      If a user doesn't set "–input_path" option, the job will still be submitted. Here is my command to run the job:

      yarn jar $HADOOP_BASE_DIR/home/share/hadoop/yarn/hadoop-yarn-submarine-3.2.0-SNAPSHOT.jar job run \
       -verbose \
       -wait_job_finish \
       --env DOCKER_JAVA_HOME=/usr/lib/jvm/java-8-oracle \
       --env DOCKER_HADOOP_HDFS_HOME=/hadoop-3.2.0-SNAPSHOT \
       --name tf-job-001 \
       --docker_image tangzhankun/tensorflow \
       --worker_resources memory=4G,vcores=2 \
       --worker_launch_cmd "cd /cifar10_estimator && python cifar10_main.py --data-dir=%input_path% --job-dir=%checkpoint_path% --num-gpus=0 --train-steps=5"

      Due to lack of invalidity check, the job is still submitted. We should add a check on this.

        Attachments

        1. YARN-8798-trunk.002.patch
          4 kB
          Zhankun Tang
        2. YARN-8798-trunk.001.patch
          4 kB
          Zhankun Tang

          Activity

            People

            • Assignee:
              tangzhankun Zhankun Tang
              Reporter:
              tangzhankun Zhankun Tang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: