Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7271

How to end Spark Submit and State Accepted

    XMLWordPrintableJSON

Details

    Description

      I'm running data cleaning job using apache griffin : https://griffin.apache.org/docs/quickstart.html

      and after submitting the spark job

      spark-submit --class org.apache.griffin.measure.Application --master yarn --deploy-mode client --queue default \ --driver-memory 1g --executor-memory 1g --num-executors 2 \ /home/bigdata/apache-hive-2.2.0-bin/measure-0.4.0.jar \ /home/bigdata/apache-hive-2.2.0-bin/env.json /home/bigdata/apache-hive-2.2.0-bin/dq.json

      My job is being submitted like the below:

      20/04/08 13:18:30 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:31 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:32 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:33 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:34 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:35 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:36 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:37 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:38 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:39 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED) 20/04/08 13:18:40 INFO yarn.Client: Application report for application_1586344612496_0247 (state: ACCEPTED)

      And never stops:

      and When I check the status of the yarn:

      bigdata@dq2:~$ yarn application -status application_1586344612496_0231 20/04/08 13:16:31 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 Application Report : Application-Id : application_1586344612496_0231 Application-Name : batch_accu Application-Type : SPARK User : bigdata Queue : default Start-Time : 1586348775760 Finish-Time : 0 Progress : 0% State : ACCEPTED Final-State : UNDEFINED Tracking-URL : N/A RPC Port : -1 AM Host : N/A Aggregate Resource Allocation : 0 MB-seconds, 0 vcore-seconds Diagnostics :

      Job is not moving can anyone pls help....

      Attachments

        Activity

          People

            Unassigned Unassigned
            priyank23 phani priyank
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: