[HDFS-925] Make it harder to accidentally close a shared DFSClient

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 0.21.0
• Component/s: hdfs-client
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 28/May/09 11:27
• Updated: 18/Sep/11 18:04
• Resolved: 18/Sep/11 18:04

Description

Every so often I get stack traces telling me that DFSClient is closed, usually in {{org.apache.hadoop.hdfs.DFSClient.checkOpen() }} . The root cause of this is usually that one thread has closed a shared fsclient while another thread still has a reference to it. If the other thread then asks for a new client it will get one and the cache repopulated but if has one already, then I get to see a stack trace.

It's effectively a race condition between clients in different threads.

Issue Links

• is related to: MAPREDUCE-437 JobTracker must ask for a new FS instance and close it when terminated.