'dfsadmin -safemode enter' should prevent the namenode from leaving safemode automatically after startup

• Key: HADOOP-1980
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Minor
• Assignee: Konstantin Shvachko
• Reporter: Koji Noguchi
• Fix Version/s: 0.18.3

Description

When debugging, I'd like to be able to intentionally keep the FS in a safemode. (For example, when looking at HADOOP-1978).
Also, it'll be nice if the namenode can still update the webUI/report when it hits the dfs.safemode.threshold.pct.

Comments

[Allen Wittenauer]

These two are sort of related. When HDFS is started up, it should be possible to select the mode that HDFS comes up in.

[Raghu Angadi]

Suggested patch for trunk.

If admin puts NN in safemode during startup, NN will stay in safemode even after block ratios are satisfied. It also prints block ratios for convenience (just like the default case where admin does not enter safemode manually).

[Hadoop QA]

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12394206/HADOOP-1980.patch
against trunk revision 719748.

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 Eclipse classpath. The patch retains Eclipse classpath integrity.

-1 core tests. The patch failed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3625/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3625/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3625/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3625/console

This message is automatically generated.

[Konstantin Shvachko]

The requirement here as I understand it is to make it possible to extend the safe mode indefinitely.
First thing to do is just to start the name-node with a large extension or a >1 threshold. I guess this does not work if the name-node has already been started but the administrator needs to keep it in safe mode longer than the extension.
Then why don't we just set really long extension in this case.
Or even better provide an explicit option to extend safe mode from the admin command.

hadoop dfsadmin -safemode extend

This will be much simpler patch, which would satisfy all the requirements and will have clear user api.
Otherwise it seems rather strange from a user point of view: in order to get infinite safe mode one should enter safe mode again although it is already on.

[Raghu Angadi]

This does not preclude from setting a very large extension.

This patch only brings consistent behavior to '-safemode enter'. Once you enter safemode manually, it makes good sense for it to require a manual command to leave. It should not matter weather it is already in safemode or not.

[Raghu Angadi]

Updated patch with a unit test.

In some sense this is more of a bugfix rather than a new feature. It mainly aims to correct the the meaning of '-safemode enter'. Additionally normal information about % of blocks reported by datanodes is displayed to help administrators.

[Konstantin Shvachko]

Does not look like there is a big enthusiasm to make changes to the shell api.
I simplified a bit Raghu's implementation of the feature.
Renamed and tweaked the test so that it shutdowned the cluster in the final section if anything fails.

[Boris Shkolnik]

+1
patch looks good

[Konstantin Shvachko]

This is patch for 0.18 branch.

[Konstantin Shvachko]

I just committed this. Thank you Raghu.