Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-802

Add minicluster config as hadoop-site.xml for testcases

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 3.2.0
    • 3.2.0
    • tests
    • None

    Description

      in MR2 (Hadoop 0.23) the configuration used by a client to submit jobs to the cluster is used to configure the AM.

      Currently this configuration is whatever has been set in hadoop-conf/ which is minimal (setting yarn framework).

      A problem arises in how AM and HS communicate to each other info a about a job (via HDFS), AM using default conf values writes to /tmp while HS started by minicluster looks in target/test-dir thus producing random failures. Random because if the AM is still alive when the client checks for the job, the AM gives the info correctly, but if the AM is gone the HS does not know about the job.

      We should seed the configuration of minicluster as the hadoop-conf/ used by the testcases.

      Attachments

        Activity

          People

            tucu00 Alejandro Abdelnur
            tucu00 Alejandro Abdelnur
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: