Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9269

Update the documentation and wrapper for fuse-dfs

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      To reproduce the bug in HDFS-9268, I followed the wiki, the doc and read the wrapper script of fuse-dfs, but found them super outdated. (the wrapper was last updated four years ago, and the hadoop project layout has dramatically changed since then). I am creating this JIRA to track the status of the update.

      There are quite a few external blogs/discussion threads floating around the internet which talked about how to update the scripts, but no one took the time to update them here.

      1. HDFS-9269.001.patch
        5 kB
        Wei-Chiu Chuang
      2. HDFS-9269.002.patch
        7 kB
        Wei-Chiu Chuang
      3. HDFS-9269.003.patch
        7 kB
        Wei-Chiu Chuang
      4. HDFS-9269.004.patch
        7 kB
        Wei-Chiu Chuang

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #655 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/655/)
          HDFS-9269. Update the documentation and wrapper for fuse-dfs. (zhz: rev 1cc7e614319a527ebc766b53b970852d5113f2f2)

          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/doc/README
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_dfs_wrapper.sh
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #655 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/655/ ) HDFS-9269 . Update the documentation and wrapper for fuse-dfs. (zhz: rev 1cc7e614319a527ebc766b53b970852d5113f2f2) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/doc/README hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_dfs_wrapper.sh
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Kihwal for the reminder. I just committed the patch to branch-2.8.

          Show
          zhz Zhe Zhang added a comment - Thanks Kihwal for the reminder. I just committed the patch to branch-2.8.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks for the reminder!

          Show
          zhz Zhe Zhang added a comment - Thanks for the reminder!
          Hide
          kihwal Kihwal Lee added a comment -

          Zhe Zhang branch-2.8 has been created. branch-2 is now 2.9. You might want to cherry-pick the commit to branch-2.8.

          Show
          kihwal Kihwal Lee added a comment - Zhe Zhang branch-2.8 has been created. branch-2 is now 2.9. You might want to cherry-pick the commit to branch-2.8.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8905 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8905/)
          HDFS-9269. Update the documentation and wrapper for fuse-dfs. (zhz: rev 1cc7e614319a527ebc766b53b970852d5113f2f2)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/doc/README
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_dfs_wrapper.sh
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8905 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8905/ ) HDFS-9269 . Update the documentation and wrapper for fuse-dfs. (zhz: rev 1cc7e614319a527ebc766b53b970852d5113f2f2) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/doc/README hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_dfs_wrapper.sh
          Hide
          zhz Zhe Zhang added a comment -

          I just committed the patch to trunk and branch-2. Thanks Wei-Chiu for the fix!

          Show
          zhz Zhe Zhang added a comment - I just committed the patch to trunk and branch-2. Thanks Wei-Chiu for the fix!
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          This is a follow-up of HDFS-9268 so target 2.8.0 should be fine.

          Thanks for the review!

          Show
          jojochuang Wei-Chiu Chuang added a comment - This is a follow-up of HDFS-9268 so target 2.8.0 should be fine. Thanks for the review!
          Hide
          zhz Zhe Zhang added a comment -

          +1, thanks Wei-Chiu for the work! I'll commit the patch shortly. Could you add a target version? 2.8.0?

          Show
          zhz Zhe Zhang added a comment - +1, thanks Wei-Chiu for the work! I'll commit the patch shortly. Could you add a target version? 2.8.0?
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          The warnings for the bash script are gone.
          This is just a wrapper script and there's no need for test cases.

          Show
          jojochuang Wei-Chiu Chuang added a comment - The warnings for the bash script are gone. This is just a wrapper script and there's no need for test cases.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 9m 23s trunk passed
          +1 compile 0m 29s trunk passed with JDK v1.8.0_66
          +1 compile 0m 27s trunk passed with JDK v1.7.0_85
          +1 mvnsite 0m 17s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 javadoc 0m 11s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 12s trunk passed with JDK v1.7.0_85
          +1 mvninstall 0m 14s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.8.0_66
          +1 javac 0m 27s the patch passed
          +1 compile 0m 28s the patch passed with JDK v1.7.0_85
          +1 javac 0m 28s the patch passed
          +1 mvnsite 0m 17s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 shellcheck 0m 9s There were no new shellcheck issues.
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 javadoc 0m 11s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 12s the patch passed with JDK v1.7.0_85
          +1 unit 0m 51s hadoop-hdfs-native-client in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 49s hadoop-hdfs-native-client in the patch passed with JDK v1.7.0_85.
          +1 asflicense 0m 27s Patch does not generate ASF License warnings.
          18m 33s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775056/HDFS-9269.004.patch
          JIRA Issue HDFS-9269
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shellcheck
          uname Linux 77f5d1144217 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1c05393
          shellcheck v0.4.1
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13711/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
          Max memory used 76MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13711/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 23s trunk passed +1 compile 0m 29s trunk passed with JDK v1.8.0_66 +1 compile 0m 27s trunk passed with JDK v1.7.0_85 +1 mvnsite 0m 17s trunk passed +1 mvneclipse 0m 14s trunk passed +1 javadoc 0m 11s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 12s trunk passed with JDK v1.7.0_85 +1 mvninstall 0m 14s the patch passed +1 compile 0m 27s the patch passed with JDK v1.8.0_66 +1 javac 0m 27s the patch passed +1 compile 0m 28s the patch passed with JDK v1.7.0_85 +1 javac 0m 28s the patch passed +1 mvnsite 0m 17s the patch passed +1 mvneclipse 0m 14s the patch passed +1 shellcheck 0m 9s There were no new shellcheck issues. +1 whitespace 0m 0s Patch has no whitespace issues. +1 javadoc 0m 11s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_85 +1 unit 0m 51s hadoop-hdfs-native-client in the patch passed with JDK v1.8.0_66. +1 unit 0m 49s hadoop-hdfs-native-client in the patch passed with JDK v1.7.0_85. +1 asflicense 0m 27s Patch does not generate ASF License warnings. 18m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775056/HDFS-9269.004.patch JIRA Issue HDFS-9269 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shellcheck uname Linux 77f5d1144217 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1c05393 shellcheck v0.4.1 JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13711/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13711/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev04:
          Updated the wrapper script to address several potential bugs.
          Found a superb website that provides instant feedback on shell script bugs/code style: http://www.shellcheck.net/

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev04: Updated the wrapper script to address several potential bugs. Found a superb website that provides instant feedback on shell script bugs/code style: http://www.shellcheck.net/
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 10m 7s trunk passed
          +1 compile 0m 31s trunk passed with JDK v1.8.0_66
          +1 compile 0m 34s trunk passed with JDK v1.7.0_85
          +1 mvnsite 0m 21s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 javadoc 0m 14s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 15s trunk passed with JDK v1.7.0_85
          +1 mvninstall 0m 17s the patch passed
          +1 compile 0m 31s the patch passed with JDK v1.8.0_66
          +1 javac 0m 31s the patch passed
          +1 compile 0m 32s the patch passed with JDK v1.7.0_85
          +1 javac 0m 32s the patch passed
          +1 mvnsite 0m 18s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          -1 shellcheck 0m 12s The applied patch generated 5 new shellcheck issues (total was 98, now 103).
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 javadoc 0m 14s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 13s the patch passed with JDK v1.7.0_85
          +1 unit 1m 1s hadoop-hdfs-native-client in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 56s hadoop-hdfs-native-client in the patch passed with JDK v1.7.0_85.
          +1 asflicense 0m 29s Patch does not generate ASF License warnings.
          20m 49s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12774945/HDFS-9269.003.patch
          JIRA Issue HDFS-9269
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shellcheck
          uname Linux e45edaa158f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1c05393
          shellcheck v0.4.1
          shellcheck https://builds.apache.org/job/PreCommit-HDFS-Build/13701/artifact/patchprocess/diff-patch-shellcheck.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13701/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
          Max memory used 75MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13701/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 10m 7s trunk passed +1 compile 0m 31s trunk passed with JDK v1.8.0_66 +1 compile 0m 34s trunk passed with JDK v1.7.0_85 +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 16s trunk passed +1 javadoc 0m 14s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_85 +1 mvninstall 0m 17s the patch passed +1 compile 0m 31s the patch passed with JDK v1.8.0_66 +1 javac 0m 31s the patch passed +1 compile 0m 32s the patch passed with JDK v1.7.0_85 +1 javac 0m 32s the patch passed +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 16s the patch passed -1 shellcheck 0m 12s The applied patch generated 5 new shellcheck issues (total was 98, now 103). +1 whitespace 0m 0s Patch has no whitespace issues. +1 javadoc 0m 14s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 13s the patch passed with JDK v1.7.0_85 +1 unit 1m 1s hadoop-hdfs-native-client in the patch passed with JDK v1.8.0_66. +1 unit 0m 56s hadoop-hdfs-native-client in the patch passed with JDK v1.7.0_85. +1 asflicense 0m 29s Patch does not generate ASF License warnings. 20m 49s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12774945/HDFS-9269.003.patch JIRA Issue HDFS-9269 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shellcheck uname Linux e45edaa158f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1c05393 shellcheck v0.4.1 shellcheck https://builds.apache.org/job/PreCommit-HDFS-Build/13701/artifact/patchprocess/diff-patch-shellcheck.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13701/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client Max memory used 75MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13701/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev03: fixed a small typo and removed a comment.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev03: fixed a small typo and removed a comment.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev02: Made the wrapper more self contained. Still need more testing (i.e. install it on a pristine machine) to make sure it works out of box.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev02: Made the wrapper more self contained. Still need more testing (i.e. install it on a pristine machine) to make sure it works out of box.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          rev1: work in progress. updated doc/README

          Show
          jojochuang Wei-Chiu Chuang added a comment - rev1: work in progress. updated doc/README

            People

            • Assignee:
              jojochuang Wei-Chiu Chuang
              Reporter:
              jojochuang Wei-Chiu Chuang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development