Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3083

Distributed Cache Handling of public directories with private files

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.23.0
    • Fix Version/s: None
    • Component/s: distributed-cache, mrv2
    • Labels:
      None

      Description

      0.23 distributed cache fails to localize with public directory containing private files. 0.20 handles this a private directory. (seems to handles all directories as private)

        Activity

        Hide
        Robert Joseph Evans added a comment -

        Even if the dist cache is not supposed to handle directories it is being used on directories in 1.0. There were even some bug fixes that went in so that in 1.0 the size of the directories is handled correctly and counted against the dist cache. It is somewhat rare for people to use it, but it does work and it is used. I would consider it a regression, although a small one.

        Show
        Robert Joseph Evans added a comment - Even if the dist cache is not supposed to handle directories it is being used on directories in 1.0. There were even some bug fixes that went in so that in 1.0 the size of the directories is handled correctly and counted against the dist cache. It is somewhat rare for people to use it, but it does work and it is used. I would consider it a regression, although a small one.
        Hide
        Amol Kekre added a comment -

        Sid, is this a "won't fix"?

        Show
        Amol Kekre added a comment - Sid, is this a "won't fix"?
        Hide
        Siddharth Seth added a comment -

        Not a blocker. I believe the dist cache isn't supposed to handle directories (only files), in which case the fix would be different.

        Show
        Siddharth Seth added a comment - Not a blocker. I believe the dist cache isn't supposed to handle directories (only files), in which case the fix would be different.
        Hide
        Arun C Murthy added a comment -

        Sid, is this a blocker?

        Show
        Arun C Murthy added a comment - Sid, is this a blocker?

          People

          • Assignee:
            Siddharth Seth
            Reporter:
            Siddharth Seth
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development