Code optimization/cleanup in HDFS

• Key: HADOOP-4583
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Suresh Srinivas
• Reporter: Suresh Srinivas
• Affects Version/s: 0.20.0
• Fix Version/s: 0.20.0

Description

Some of the changes needed:

• FSNameSystem.allocateBlock creates a lot of object unnecessarily to find a blockID that is unused. It should reuse the Block object created by changing the blockID and reusing the GenerationStamp.
• InodeFile.addBlock should use System.arrayCopy instead of copying 1 at a time.
• Some of the for loops in FSEditLog.java can be optimized. Instead of getting the edit stream length in termination condition, a local variable should be used to do the same.

Comments

[Suresh Srinivas]

Thanks Nicholas for pointing out, GenerationStamp cannot be reused. But the Block object can be reused.

[dhruba borthakur]

>Instead of getting the edit stream length in termination condition, a local variable should be used to do the same

Won't the Java compiler do this on its own?

[Suresh Srinivas]

I am not sure if the method call in termination condition is optimized. I know that loops may be optimized from condition < len to condition <= 0. Based on the tests I ran, moving the method call out of terminating condition improves the performance.

[Tsz Wo (Nicholas), SZE]

Codes look good but the patch is out dated. Need sync with trunk.

[Suresh Srinivas]

New patch uploaded. Here are the results of testing the patch. No tests included as the functionality is tested by existing tests.

[exec] -1 overall.

[exec] +1 @author. The patch does not contain any @author tags.

[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.

[exec] +1 javadoc. The javadoc tool did not generate any warning messages.

[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.

[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.

[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

[Hadoop QA]

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12393485/4583.patch
against trunk revision 712305.

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

-1 javadoc. The javadoc tool appears to have generated 1 warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 Eclipse classpath. The patch retains Eclipse classpath integrity.

+1 core tests. The patch passed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3552/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3552/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3552/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3552/console

This message is automatically generated.

[Tsz Wo (Nicholas), SZE]

The javadoc warning is unrelated. See HADOOP-4621.

[Tsz Wo (Nicholas), SZE]

I just committed this. Thanks, Suresh!

[Hudson]

Integrated in Hadoop-trunk #655 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/655/)
. Several code optimizations in HDFS. (Suresh Srinivas via szetszwo)