Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.3, 2.0.0-alpha, 3.0.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mrv2
    • Labels:
      None

      Description

      The distributed cache does not work like it does in 1.0.

      mapreduce.job.cache.symlink.create is completely ignored and symlinks are always created no matter what. Files and archives without a fragment will also have symlinks created.

      If two cache archives or cache files happen to have the same name, or same symlink fragment only the last one in the list is localized.

      The localCacheArchives and LocalCacheFiles are not set correctly when these duplicates happen causing off by one or more errors for anyone trying to use them.

      The reality is that use of symlinking is so common currently that these incompatibilities are not that likely to show up, but we still need to fix them.

      1. MR-4493.txt
        27 kB
        Robert Joseph Evans
      2. MR-4493.txt
        38 kB
        Robert Joseph Evans
      3. MR-4493.txt
        27 kB
        Robert Joseph Evans

        Issue Links

          Activity

          Robert Joseph Evans created issue -
          Robert Joseph Evans made changes -
          Field Original Value New Value
          Priority Blocker [ 1 ] Critical [ 2 ]
          Robert Joseph Evans made changes -
          Attachment MR-4493.txt [ 12538419 ]
          Robert Joseph Evans made changes -
          Link This issue depends on MAPREDUCE-4456 [ MAPREDUCE-4456 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-4493.txt [ 12538572 ]
          Thomas Graves made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Robert Joseph Evans made changes -
          Attachment MR-4493.txt [ 12538577 ]
          Thomas Graves made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Thomas Graves made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.23.3 [ 12320060 ]
          Fix Version/s 3.0.0 [ 12320355 ]
          Fix Version/s 2.2.0-alpha [ 12322471 ]
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Fix Version/s 3.0.0 [ 12320355 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue depends on MAPREDUCE-4456 [ MAPREDUCE-4456 ]
          Gavin made changes -
          Link This issue depends upon MAPREDUCE-4456 [ MAPREDUCE-4456 ]

            People

            • Assignee:
              Robert Joseph Evans
              Reporter:
              Robert Joseph Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development