Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13393

[Azure Data Lake] Correction needed in fs.defaultFS support in documentation

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: fs/adl
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Azure data lake does not support fs.defaultFS pointing to adls storage as of today.So documentation to point fs.defaultFS configuration is confusing to customer integrating adls configuration in there storage.

      1. HADOOP-13393.001.patch
        2 kB
        Vishwajeet Dusane

        Activity

        Hide
        vishwajeet.dusane Vishwajeet Dusane added a comment -

        Removed

        <property>
        <name>fs.defaultFS</name>
        <value>YOUR ADL STORE URL (e.g., https://example.azuredatalakestore.net) </value>
        </property>

        section from the documentation.

        Show
        vishwajeet.dusane Vishwajeet Dusane added a comment - Removed <property> <name>fs.defaultFS</name> <value>YOUR ADL STORE URL (e.g., https://example.azuredatalakestore.net ) </value> </property> section from the documentation.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 7m 10s trunk passed
        +1 mvnsite 0m 19s trunk passed
        +1 mvnsite 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        8m 37s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819106/HADOOP-13393.001.patch
        JIRA Issue HADOOP-13393
        Optional Tests asflicense mvnsite
        uname Linux 941df255836b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1c9d2ab
        modules C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10030/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 7m 10s trunk passed +1 mvnsite 0m 19s trunk passed +1 mvnsite 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 8m 37s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819106/HADOOP-13393.001.patch JIRA Issue HADOOP-13393 Optional Tests asflicense mvnsite uname Linux 941df255836b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1c9d2ab modules C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10030/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 7m 27s trunk passed
        +1 mvnsite 0m 21s trunk passed
        +1 mvnsite 0m 18s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        9m 0s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819106/HADOOP-13393.001.patch
        JIRA Issue HADOOP-13393
        Optional Tests asflicense mvnsite
        uname Linux d99aa07f5547 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 38128ba
        modules C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10033/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 7m 27s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvnsite 0m 18s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 9m 0s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819106/HADOOP-13393.001.patch JIRA Issue HADOOP-13393 Optional Tests asflicense mvnsite uname Linux d99aa07f5547 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 38128ba modules C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10033/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 6m 43s trunk passed
        +1 mvnsite 0m 19s trunk passed
        +1 mvnsite 0m 16s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        8m 10s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819106/HADOOP-13393.001.patch
        JIRA Issue HADOOP-13393
        Optional Tests asflicense mvnsite
        uname Linux b42b97f21097 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 38128ba
        modules C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10037/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 6m 43s trunk passed +1 mvnsite 0m 19s trunk passed +1 mvnsite 0m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 8m 10s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819106/HADOOP-13393.001.patch JIRA Issue HADOOP-13393 Optional Tests asflicense mvnsite uname Linux b42b97f21097 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 38128ba modules C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10037/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        chris.douglas Chris Douglas added a comment -

        +1

        However, if configuring fs.defaultFS to point to ADLS is unsuppported, it might be worth noting that limitation in the documentation, also. I committed this fixup to trunk.

        Thanks Vishwajeet

        Show
        chris.douglas Chris Douglas added a comment - +1 However, if configuring fs.defaultFS to point to ADLS is unsuppported, it might be worth noting that limitation in the documentation, also. I committed this fixup to trunk. Thanks Vishwajeet
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10135 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10135/)
        HADOOP-13393. Omit unsupported fs.defaultFS setting in ADLS (cdouglas: rev dca6335dbc25c7c81444a294d68d821308eef69e)

        • hadoop-tools/hadoop-azure-datalake/src/site/markdown/index.md
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10135 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10135/ ) HADOOP-13393 . Omit unsupported fs.defaultFS setting in ADLS (cdouglas: rev dca6335dbc25c7c81444a294d68d821308eef69e) hadoop-tools/hadoop-azure-datalake/src/site/markdown/index.md
        Hide
        cnauroth Chris Nauroth added a comment -

        Vishwajeet Dusane, thank you for the patch. Chris Douglas, thank you for reviewing. I see the desired fix version is 3.0.0-alpha1. We have a separate branch-3.0.0-alpha1 now, so I cherry-picked the trunk patch to there.

        Show
        cnauroth Chris Nauroth added a comment - Vishwajeet Dusane , thank you for the patch. Chris Douglas , thank you for reviewing. I see the desired fix version is 3.0.0-alpha1. We have a separate branch-3.0.0-alpha1 now, so I cherry-picked the trunk patch to there.

          People

          • Assignee:
            vishwajeet.dusane Vishwajeet Dusane
            Reporter:
            vishwajeet.dusane Vishwajeet Dusane
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development