Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15094

FileSystem.getCanonicalUri() to be public

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 2.7.4
    • None
    • fs
    • None

    Description

      Discussion around SPARK-22587 highlights how per-fs notions of a canonical URI make it hard to determine if a file is on a specific filesystem, or, put differently, if two filesystems are equivalent.

      You can't reliably use this.getUri == that.getUri as it doesn't handle FQDN == unqualified DN, bit you can't do nslookup as HDFS HA doesn't use hosnames.

      If FileSystem.getCanonicalUri() were public, then this could be used to compare things consistently.

      needs: filesystem.md coverage; contract test (two filesystem instances are equal, different filesystems aren't). Or at least: this method never returns null.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-22587 Spark job fails if fs.defaultFS and application jar are different url

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: