[HDFS-11086] DataNode disk check improvements - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Done
Affects Version/s: None
Fix Version/s: None
Component/s: datanode
Labels:
None

Description

This Jira tracks a few improvements to DataNode’s usage of DiskChecker to address the following problems:

Checks are serialized so a single slow disk can indefinitely delay checking the rest.
Related to 1, no detection of stalled checks.
Lack of granularity. A single IO error initiates checking all disks.
Inconsistent activation. Some DataNode IO failures trigger disk checks but not all.

Attachments

Sub-Tasks

Create Sub-Task

1.	Support for running async disk checks in DataNode	Resolved	Arpit Agarwal	Actions
2.	Support for parallel checking of StorageLocations on DataNode startup	Resolved	Arpit Agarwal	Actions
3.	Update DataNode to use StorageLocationChecker at startup	Resolved	Arpit Agarwal	Actions
4.	Support for parallel checking of FsVolumes	Resolved	Arpit Agarwal	Actions
5.	Update DataNode to use DatasetVolumeChecker	Resolved	Arpit Agarwal	Actions
6.	Implement equals and hashcode in FsVolumeSpi implementations	Resolved	Manjunath Anand	Actions

Activity

Comment

This comment will be Viewable by All Users Viewable by All Users

Cancel

People

Assignee:: Arpit Agarwal

Reporter:: Arpit Agarwal

Votes:: 0 Vote for this issue

Watchers:: 15 Start watching this issue

Dates

Created:: 01/Nov/16 21:51

Updated:: 06/Jun/17 13:38

Resolved:: 06/Jun/17 13:38

Agile

View on Board

DataNode disk check improvements