Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2499

map-reduce launcher does not need distributed files, archives except jar of input/outputformat class

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 4.2.0
    • None
    • core
    • None

    Description

      In map-reduce action, when specify file, archives element, JavaActionExccutor add it to distributed cache for configuration of launcher.

      It cause slow launching mr job.

      In fact, all most launcher does not need file , archives. launcher need to inputformat/outputformat class and its dependency.

      I make a proposal below feature.

      • add property oozie.launcher.distributed.cache.filter=*.jar,*.properties. default is *
      • when configure launcher, filter cache files.

      Attachments

        1. OOZIE-2499.001.patch
          9 kB
          KWON BYUNGCHANG
        2. OOZIE-2499.002.patch
          9 kB
          KWON BYUNGCHANG
        3. OOZIE-2499.003.patch
          9 kB
          KWON BYUNGCHANG
        4. OOZIE-2499.004.patch
          5 kB
          KWON BYUNGCHANG

        Activity

          People

            Unassigned Unassigned
            magnum KWON BYUNGCHANG
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: