Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1
    • Fix Version/s: 0.20.2
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Bugs fixed for Hadoop archives: character escaping in paths, LineReader and file system caching.

      Description

      Found and fixed several bugs involving Hadoop archives:

      • In makeQualified(), the sloppy conversion from Path to URI and back mangles the path if it contains an escape-worthy character.
      • It's possible that fileStatusInIndex() may have to read more than one segment of the index. The LineReader and count of bytes read need to be reset for each block.
      • har:// connections cannot be indexed by (scheme, authority, username) – the path is significant as well. Caching them in this way limits a hadoop client to opening one archive per filesystem. It seems to be safe not to cache them, since they wrap another connection that does the actual networking.
      1. HADOOP-6097-v2.patch
        1 kB
        Tom White
      2. HADOOP-6097-0.20.patch
        1 kB
        Mahadev konar
      3. HADOOP-6097-0.20.patch
        4 kB
        Mahadev konar
      4. HADOOP-6097-0.20.patch
        4 kB
        Mahadev konar
      5. HADOOP-6097.patch
        2 kB
        Ben Slusky

        Issue Links

          Activity

          Ben Slusky created issue -
          Ben Slusky made changes -
          Field Original Value New Value
          Attachment HADOOP-6097.patch [ 12411434 ]
          Ben Slusky made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Assignee Ben Slusky [ sluskyb ]
          Tom White made changes -
          Attachment HADOOP-6097-v2.patch [ 12418348 ]
          Vladimir Klimontovich made changes -
          Link This issue is cloned as HADOOP-6231 [ HADOOP-6231 ]
          Owen O'Malley made changes -
          Fix Version/s 0.20.2 [ 12314203 ]
          Fix Version/s 0.20.1 [ 12313866 ]
          Ben Slusky made changes -
          Affects Version/s 0.20.1 [ 12313866 ]
          Affects Version/s 0.19.2 [ 12313650 ]
          Mahadev konar made changes -
          Link This issue incorporates MAPREDUCE-1010 [ MAPREDUCE-1010 ]
          Mahadev konar made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mahadev konar made changes -
          Attachment HADOOP-6097-0.20.patch [ 12420209 ]
          Mahadev konar made changes -
          Attachment HADOOP-6097-0.20.patch [ 12420224 ]
          Mahadev konar made changes -
          Attachment HADOOP-6097-0.20.patch [ 12422737 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Robert Chansler made changes -
          Release Note Bugs fixed for Hadoop archives: character escaping in paths, LineReader and file system caching.

            People

            • Assignee:
              Ben Slusky
              Reporter:
              Ben Slusky
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development