Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5065 Umbrella JIRA of SLS fixes / improvements
  3. YARN-6522

Make SLS JSON input file format simple and scalable

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      SLS input format is verbose, and it doesn't scale out. We can improve it in these ways:

      1. We need to configure tasks one by one if there are more than one task in a job, which means the job configuration usually includes lots of redundant items. To specify the number of task for task configuration will solve this issue.
      2. Container host is useful for locality testing. It is obnoxious to specify container host for each task for tests unrelated to locality. We would like to make it optional.
      3. For most tests, we don't care about job.id. Make it optional and generated automatically by default.
      4. job.finish.ms doesn't make sense, just remove it.
      5. container type and container priority should be optional as well.

        Attachments

        1. YARN-6522.001.patch
          10 kB
          Yufei Gu
        2. YARN-6522.002.patch
          17 kB
          Yufei Gu
        3. YARN-6522.003.patch
          17 kB
          Yufei Gu
        4. YARN-6522.004.patch
          18 kB
          Yufei Gu

          Activity

            People

            • Assignee:
              yufeigu Yufei Gu
              Reporter:
              yufeigu Yufei Gu
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: