Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-476

extend DistributedCache to work locally (LocalJobRunner)

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Extended DistributedCache to work with LocalJobRunner.

      Description

      The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html

      Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development.

        Attachments

        1. 476.20S-2.patch
          97 kB
          Ravi Gummadi
        2. MAPREDUCE-476-v9.patch
          101 kB
          Philip Martin
        3. MAPREDUCE-476-v8.patch
          100 kB
          Philip Martin
        4. MAPREDUCE-476-v7.patch
          100 kB
          Philip Martin
        5. v6-to-v7.patch
          13 kB
          Philip Martin
        6. MAPREDUCE-476-20090818.txt
          94 kB
          Vinod Kumar Vavilapalli
        7. MAPREDUCE-476-20090814.1.txt
          94 kB
          Vinod Kumar Vavilapalli
        8. MAPREDUCE-476-v5-requires-MR711.patch
          92 kB
          Philip Martin
        9. MAPREDUCE-476-v4-requires-MR711.patch
          92 kB
          Philip Martin
        10. MAPREDUCE-476-v2-vs-v4.txt
          42 kB
          Philip Martin
        11. MAPREDUCE-476-v3.try2.patch
          0.4 kB
          Philip Martin
        12. MAPREDUCE-476-v2-vs-v3.try2.patch
          41 kB
          Philip Martin
        13. MAPREDUCE-476-v3.patch
          91 kB
          Philip Martin
        14. MAPREDUCE-476-v2-vs-v3.patch
          41 kB
          Philip Martin
        15. MAPREDUCE-476-v2.patch
          85 kB
          Philip Martin
        16. MAPREDUCE-476.patch
          24 kB
          Philip Martin
        17. HADOOP-2914-v3.patch
          84 kB
          Philip Martin
        18. HADOOP-2914-v2.patch
          112 kB
          Philip Martin
        19. HADOOP-2914-v1-full.patch
          116 kB
          Philip Martin
        20. HADOOP-2914-v1-since-4041.patch
          83 kB
          Philip Martin

          Issue Links

          blocks

          Sub-task - The sub-task of the issue MAPREDUCE-856 Localized files from DistributedCache should have right access-control

          • Major - Major loss of function.
          • Closed
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-5174 Pipes example throws NullPointerException on trunk when run with LocalJobRunner

          • Major - Major loss of function.
          • Closed
          incorporates

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-6125 extend DistributedCache to work locally (LocalJobRunner) (common half)

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-711 Move Distributed Cache from Common to Map/Reduce

          • Major - Major loss of function.
          • Closed
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-711 Move Distributed Cache from Common to Map/Reduce

          • Major - Major loss of function.
          • Closed

            Activity

              People

              • Assignee:
                philip Philip Martin
                Reporter:
                rash37 sam rash
              • Votes:
                2 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: