Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1505

Cluster class should create the rpc client only when needed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2
    • Fix Version/s: 0.22.0
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Lazily construct a connection to the JobTracker from the job-submission client.

      Description

      It will be good to have the org.apache.hadoop.mapreduce.Cluster create the rpc client object only when needed (when a call to the jobtracker is actually required). org.apache.hadoop.mapreduce.Job constructs the Cluster object internally and in many cases the application that created the Job object really wants to look at the configuration only. It'd help to not have these connections to the jobtracker especially when Job is used in the tasks (for e.g., Pig calls mapreduce.FileInputFormat.setInputPath in the tasks and that requires a Job object to be passed).

      In Hadoop 20, the Job object internally creates the JobClient object, and the same argument applies there too.

      1. mapreduce-1505--2010-05-26.patch
        33 kB
        Dick King
      2. mapreduce-1505--2010-05-19.patch
        11 kB
        Dick King
      3. MAPREDUCE-1505_yhadoop20.patch
        3 kB
        Arun C Murthy
      4. MAPREDUCE-1505_yhadoop20_9.patch
        3 kB
        Arun C Murthy

        Issue Links

          Activity

          Devaraj Das created issue -
          Devaraj Das made changes -
          Field Original Value New Value
          Summary Job class should create the rpc client only when needed Cluster class should create the rpc client only when needed
          Arun C Murthy made changes -
          Attachment MAPREDUCE-1505_yhadoop20.patch [ 12436628 ]
          Amareshwari Sriramadasu made changes -
          Link This issue blocks MAPREDUCE-118 [ MAPREDUCE-118 ]
          Arun C Murthy made changes -
          Attachment MAPREDUCE-1505_yhadoop20_9.patch [ 12440995 ]
          Arun C Murthy made changes -
          Release Note Lazily construct a connection to the JobTracker from the job-submission client.
          Dick King made changes -
          Assignee Dick King [ dking ]
          Amareshwari Sriramadasu made changes -
          Link This issue blocks MAPREDUCE-118 [ MAPREDUCE-118 ]
          Amareshwari Sriramadasu made changes -
          Link This issue is depended upon by MAPREDUCE-118 [ MAPREDUCE-118 ]
          Dick King made changes -
          Attachment mapreduce-1505--2010-05-19.patch [ 12444965 ]
          Dick King made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Dick King made changes -
          Attachment mapreduce-1505--2010-05-26.patch [ 12445678 ]
          Dick King made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Dick King
              Reporter:
              Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development