Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1505

Cluster class should create the rpc client only when needed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.2
    • 0.22.0
    • client
    • None
    • Reviewed
    • Lazily construct a connection to the JobTracker from the job-submission client.

    Description

      It will be good to have the org.apache.hadoop.mapreduce.Cluster create the rpc client object only when needed (when a call to the jobtracker is actually required). org.apache.hadoop.mapreduce.Job constructs the Cluster object internally and in many cases the application that created the Job object really wants to look at the configuration only. It'd help to not have these connections to the jobtracker especially when Job is used in the tasks (for e.g., Pig calls mapreduce.FileInputFormat.setInputPath in the tasks and that requires a Job object to be passed).

      In Hadoop 20, the Job object internally creates the JobClient object, and the same argument applies there too.

      Attachments

        1. mapreduce-1505--2010-05-26.patch
          33 kB
          Dick King
        2. mapreduce-1505--2010-05-19.patch
          11 kB
          Dick King
        3. MAPREDUCE-1505_yhadoop20_9.patch
          3 kB
          Arun Murthy
        4. MAPREDUCE-1505_yhadoop20.patch
          3 kB
          Arun Murthy

        Issue Links

          Activity

            People

              dking Dick King
              ddas Devaraj Das
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: