Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13891 HDFS RBF stabilization phase I
  3. HDFS-13845

RBF: The default MountTableResolver should fail resolving multi-destination paths

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 3.1.0, 2.9.1
    • Fix Version/s: 3.3.0, HDFS-13891
    • Component/s: federation, hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When we use the default MountTableResolver to resolve the path, we cannot get the destination paths for the default DestinationOrder.HASH.

      // Some comments here
      private static PathLocation buildLocation(
            ......
          List<RemoteLocation> locations = new LinkedList<>();
          for (RemoteLocation oneDst : entry.getDestinations()) {
            String nsId = oneDst.getNameserviceId();
            String dest = oneDst.getDest();
            String newPath = dest;
            if (!newPath.endsWith(Path.SEPARATOR) && !remainingPath.isEmpty()) {
              newPath += Path.SEPARATOR;
            }
            newPath += remainingPath;
            RemoteLocation remoteLocation = new RemoteLocation(nsId, newPath, path);
            locations.add(remoteLocation);
          }
          DestinationOrder order = entry.getDestOrder();
          return new PathLocation(srcPath, locations, order);
        }
      

      The default order will be hash, but the HashFirstResolver will not be invoked to order the location.
      It is ambiguous for the MountTableResolver that we will see the HASH order in the web ui for multi-destinations path but we cannot get the result.
      In my opinion, the MountTableResolver will be a simple resolver to implement 1 to 1 not including the 1 to n destinations. So we should check the buildLocation. If the entry has multi destinations, we should reject it.

        Attachments

        1. HDFS-13845.001.patch
          4 kB
          yanghuafeng
        2. HDFS-13845.002.patch
          9 kB
          yanghuafeng
        3. HDFS-13845.003.patch
          9 kB
          yanghuafeng
        4. HDFS-13845.004.patch
          13 kB
          yanghuafeng
        5. HDFS-13845.005.patch
          12 kB
          yanghuafeng

          Issue Links

            Activity

              People

              • Assignee:
                hfyang20071 yanghuafeng
                Reporter:
                hfyang20071 yanghuafeng
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: