Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      A few HDFS subcommands and options are missing in the documentation.

      1. envvars: display computed Hadoop environment variables

      I also noticed (in HDFS-9927) that a few OIV options are missing, and I'll be looking for other missing options as well.

      Filling this JIRA to fix them all.

      1. HDFS-9928.001.patch
        12 kB
        Wei-Chiu Chuang
      2. HDFS-9928-branch-2.002.patch
        22 kB
        Wei-Chiu Chuang
      3. HDFS-9928-trunk.003.patch
        23 kB
        Wei-Chiu Chuang

        Activity

        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Updated all missing commands/subcommand/options.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Updated all missing commands/subcommand/options.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 6m 33s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvnsite 0m 48s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        8m 54s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792462/HDFS-9928.001.patch
        JIRA Issue HDFS-9928
        Optional Tests asflicense mvnsite
        uname Linux 1b26be0d607e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2e040d3
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14775/artifact/patchprocess/whitespace-eol.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14775/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 6m 33s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvnsite 0m 48s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 8m 54s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792462/HDFS-9928.001.patch JIRA Issue HDFS-9928 Optional Tests asflicense mvnsite uname Linux 1b26be0d607e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2e040d3 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14775/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14775/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        iwasakims Masatake Iwasaki added a comment -
        264	| `-h`\|`--help` | Display the tool usage and help information and exit. |
        

        There are duplicate entries of -h for oiv_legacy.

        Show
        iwasakims Masatake Iwasaki added a comment - 264 | `-h`\|`--help` | Display the tool usage and help information and exit. | There are duplicate entries of -h for oiv_legacy.
        Hide
        iwasakims Masatake Iwasaki added a comment -
        -Usage: `hdfs cacheadmin -addDirective -path <path> -pool <pool-name> [-force] [-replication <replication>] [-ttl <time-to-live>]`
        +Usage:
        +
        +    hdfs dfsadmin
        +          [-addDirective -path <path> -pool <pool-name> [-force] [-replication <replication>] [-ttl <time-to-live>]]
        +          [-modifyDirective -id <id> [-path <path>] [-force] [-replication <replication>] [-pool <pool-name>] [-ttl <time-to-live>]]
        +          [-listDirectives [-stats] [-path <path>] [-pool <pool>] [-id <id>]
        +          [-removeDirective <id>]
        +          [-removeDirectives -path <path>]
        +          [-addPool <name> [-owner <owner>] [-group <group>] [-mode <mode>] [-limit <limit>] [-maxTtl <maxTtl>]
        +          [-modifyPool <name> [-owner <owner>] [-group <group>] [-mode <mode>] [-limit <limit>] [-maxTtl <maxTtl>]]
        +          [-removePool <name>]
        +          [-listPools [-stats] [<name>]]
        +          [-help <command-name>]
        
        • hdfs dfsadmin should be hdfs cacheadmin.
        • This usage looks like we can use multiple directives at the same time but it is not true.
        Show
        iwasakims Masatake Iwasaki added a comment - -Usage: `hdfs cacheadmin -addDirective -path <path> -pool <pool-name> [-force] [-replication <replication>] [-ttl <time-to-live>]` +Usage: + + hdfs dfsadmin + [-addDirective -path <path> -pool <pool-name> [-force] [-replication <replication>] [-ttl <time-to-live>]] + [-modifyDirective -id <id> [-path <path>] [-force] [-replication <replication>] [-pool <pool-name>] [-ttl <time-to-live>]] + [-listDirectives [-stats] [-path <path>] [-pool <pool>] [-id <id>] + [-removeDirective <id>] + [-removeDirectives -path <path>] + [-addPool <name> [-owner <owner>] [-group <group>] [-mode <mode>] [-limit <limit>] [-maxTtl <maxTtl>] + [-modifyPool <name> [-owner <owner>] [-group <group>] [-mode <mode>] [-limit <limit>] [-maxTtl <maxTtl>]] + [-removePool <name>] + [-listPools [-stats] [<name>]] + [-help <command-name>] hdfs dfsadmin should be hdfs cacheadmin . This usage looks like we can use multiple directives at the same time but it is not true.
        Hide
        iwasakims Masatake Iwasaki added a comment -
                 hdfs dfsadmin [GENERIC_OPTIONS]
                       [-report [-live] [-dead] [-decommissioning]]
                       [-safemode enter | leave | get | wait | forceExit]
        -              [-saveNamespace]
        +              [-saveNamespace [-beforeShutdown]]
        

        Adding -beforeShutdown (by HDFS-6353) was trunk-only change. We should keep this as is if target is 2.8.0.

        -| `-reconfig` \<datanode \|...\> \<host:ipc\_port\> \<start\|status\> | Start reconfiguration or get the status of an ongoing reconfiguration. The second parameter specifies the node type. Currently, only reloading DataNode's configuration is supported. |
        +| `-reconfig` \<datanode \|namenode\> \<host:ipc\_port\> \<start\|status\|properties\> | Starts reconfiguration or gets the status of an ongoing reconfiguration, or gets a list of reconfigurable properties. The second parameter specifies the node type. |
        

        Similar to above, fix version of HDFS-9094 is 2.9.0.

        Show
        iwasakims Masatake Iwasaki added a comment - hdfs dfsadmin [GENERIC_OPTIONS] [-report [-live] [-dead] [-decommissioning]] [-safemode enter | leave | get | wait | forceExit] - [-saveNamespace] + [-saveNamespace [-beforeShutdown]] Adding -beforeShutdown (by HDFS-6353 ) was trunk-only change. We should keep this as is if target is 2.8.0. -| `-reconfig` \<datanode \|...\> \<host:ipc\_port\> \<start\|status\> | Start reconfiguration or get the status of an ongoing reconfiguration. The second parameter specifies the node type. Currently, only reloading DataNode's configuration is supported. | +| `-reconfig` \<datanode \|namenode\> \<host:ipc\_port\> \<start\|status\|properties\> | Starts reconfiguration or gets the status of an ongoing reconfiguration, or gets a list of reconfigurable properties. The second parameter specifies the node type. | Similar to above, fix version of HDFS-9094 is 2.9.0.
        Hide
        iwasakims Masatake Iwasaki added a comment -
        +      hdfs storagepolicies
        +          [-listPolicies]
        +          [-setStoragePolicy -path <path> -policy <policy>]
        +          [-getStoragePolicy -path <path>]
        +          [-unsetStoragePolicy -path <path>]
        +          [-help <command-name>]
        

        -unsetStoragePolicy was added by HDFS-9534 in 2.9.0.

        Show
        iwasakims Masatake Iwasaki added a comment - + hdfs storagepolicies + [-listPolicies] + [-setStoragePolicy -path <path> -policy <policy>] + [-getStoragePolicy -path <path>] + [-unsetStoragePolicy -path <path>] + [-help <command-name>] -unsetStoragePolicy was added by HDFS-9534 in 2.9.0.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        How about targeting 2.9.0 here by just reverting the part of -beforeShutdown?

        Show
        iwasakims Masatake Iwasaki added a comment - How about targeting 2.9.0 here by just reverting the part of -beforeShutdown ?
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev02: rebased to 2.9.0. I also had to remove envvars command because HADOOP-12366 just reverted it in branch-2. Other than that it's mostly cosmetic changes.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev02: rebased to 2.9.0. I also had to remove envvars command because HADOOP-12366 just reverted it in branch-2. Other than that it's mostly cosmetic changes.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Thanks for the update, Wei-Chiu Chuang. The patch looks good to me.

        Can you upload the patch for trunk too? I think omitting the part of hdfs namenode -rollingUpgrade will make it. (We don't need the part for trunk since downgrade option was removed in trunk.)

        Show
        iwasakims Masatake Iwasaki added a comment - Thanks for the update, Wei-Chiu Chuang . The patch looks good to me. Can you upload the patch for trunk too? I think omitting the part of hdfs namenode -rollingUpgrade will make it. (We don't need the part for trunk since downgrade option was removed in trunk.)
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks Masatake Iwasaki again for the review. Here's the patch for trunk.

        I've manually verified all command line options against this patch. Hopefully I'm not missing anything. The only thing added in addition to Masatake Iwasaki's suggestion is hdfs envvars command, which is in trunk only.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks Masatake Iwasaki again for the review. Here's the patch for trunk. I've manually verified all command line options against this patch. Hopefully I'm not missing anything. The only thing added in addition to Masatake Iwasaki 's suggestion is hdfs envvars command, which is in trunk only.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvninstall 7m 22s trunk passed
        +1 mvnsite 1m 5s trunk passed
        +1 mvnsite 1m 3s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 asflicense 0m 20s Patch does not generate ASF License warnings.
        10m 21s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793482/HDFS-9928-trunk.003.patch
        JIRA Issue HDFS-9928
        Optional Tests asflicense mvnsite
        uname Linux 74037850fcfc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / eba66a6
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14823/artifact/patchprocess/whitespace-eol.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14823/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 7m 22s trunk passed +1 mvnsite 1m 5s trunk passed +1 mvnsite 1m 3s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 10m 21s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793482/HDFS-9928-trunk.003.patch JIRA Issue HDFS-9928 Optional Tests asflicense mvnsite uname Linux 74037850fcfc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eba66a6 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14823/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14823/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        +1

        Show
        iwasakims Masatake Iwasaki added a comment - +1
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Committed to trunk and branch-2. Thanks, Wei-Chiu Chuang!

        Show
        iwasakims Masatake Iwasaki added a comment - Committed to trunk and branch-2. Thanks, Wei-Chiu Chuang !
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9463 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9463/)
        HDFS-9928. Make HDFS commands guide up to date (Wei-Chiu Chuang via (iwasakims: rev 5de848cd5d46527a8fba481c76089da21f533050)

        • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9463 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9463/ ) HDFS-9928 . Make HDFS commands guide up to date (Wei-Chiu Chuang via (iwasakims: rev 5de848cd5d46527a8fba481c76089da21f533050) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
        Hide
        jojochuang Wei-Chiu Chuang added a comment -
        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks Masatake Iwasaki !

          People

          • Assignee:
            jojochuang Wei-Chiu Chuang
            Reporter:
            jojochuang Wei-Chiu Chuang
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development