[HDFS-1366] reduce namenode startup time by optimising checkBlockInfo while loading fsimage - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: None
Component/s: namenode
Labels:
None

Description

The namenode spends about 10 minutes reading in a 14 GB fsimage file into memory and creating all the in-memory data structures. A jstack based debugger clearly shows that most of the time during the fsimage load is spent in BlocksMap.checkBlockInfo. There is a easy way to optimize this method especially for this code path.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

FSImageRead_shortcut_proto.patch
05/Jul/11 22:59
2 kB
Matthew Foley

Activity

People

Assignee:: Unassigned

Reporter:: Dhruba Borthakur

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 02/Sep/10 05:32

Updated:: 30/Jul/14 23:48

Resolved:: 30/Jul/14 23:48