Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2139

Gridmix test for submitting jobs with different traces and runtime options.

    XMLWordPrintableJSON

Details

    • Task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • test
    • None

    Description

      1.Generate input data based on cluster size and create the synthetic jobs by using the 7 min folded MR trace and submit
      the jobs with below arguments.

      GRIDMIX_JOB_TYPE = SleepJob
      GRIDMIX_USER_RESOLVER = SubmitterUserResolver
      GRIDMIX_MIN_FILE = 200 MB
      GRIDMIX_SUBMISSION_POLICY = STRESS
      GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
      Input Size = 400 MB * No. of nodes in cluster.
      TRACE_FILE = 7 min folded trace.
      Verify each job status, summary(QueueName, UserName, StatTime, FinishTime, maps, reducers and counters etc) after
      completion of execution. Make sure the queue should be default queue name.

      2. Generate input data based on cluster size and create the synthetic jobs by using the 10 min folded MR trace and
      submit the jobs with below arguments.

      GRIDMIX_JOB_TYPE = SleepJob
      GRIDMIX_USER_RESOLVER = RoundRobinUserResolver
      GRIDMIX_MIN_FILE = 200 MB
      GRIDMIX_SUBMISSION_POLICY = SERIAL
      GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
      SLEEPJOB_MAPTASK_ONLY = true
      GRIDMIX_SLEEP_MAX_MAP_TIME = 10 sec
      Input Size = 250 MB * No. of nodes in cluster.
      TRACE_FILE = 10 min folded trace.
      Verify each job status, summary(QueueName, UserName, StatTime, FinishTime, maps and counters etc) after completion of
      execution. Make sure the reducers should be zero.

      3. Generate input data based on cluster size and create the synthetic jobs by using the 12 min folded MR trace and submit the jobs with below arguments.

      GRIDMIX_JOB_TYPE = SleepJob
      GRIDMIX_USER_RESOLVER = SubmitterUserResolver
      GRIDMIX_MIN_FILE = 200 MB
      GRIDMIX_SUBMISSION_POLICY = STRESS
      GRIDMIX_SLEEP_MAX_MAP_TIME = 10 sec
      GRIDMIX_SLEEP_MAX_REDUCE_TIME = 5 sec
      Input Size = 150 MB * No. of nodes in cluster.
      TRACE_FILE = 12 min folded trace.
      Verify each job status, summary(QueueName, UserName, StatTime, FinishTime, maps,reducers and counters etc) after completion of execution.

      Attachments

        1. MAPREDUCE-2139.patch
          15 kB
          Vinay Kumar Thota
        2. MAPREDUCE-2139.patch
          9 kB
          Vinay Kumar Thota

        Issue Links

          Activity

            People

              vinaythota Vinay Kumar Thota
              vinaythota Vinay Kumar Thota
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated: