[HDFS-15972] Fedbalance only copies data partially when there's existing opened file

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Felix N
• Created: 13/Apr/21 03:59
• Updated: 13/Apr/21 12:10

Description

If there are opened files when fedbalance is run and data is being written to these files, fedbalance might skip the newly written data.

Steps to recreate the issue:

1. Create a dummy file /test/file with some data: echo "start" | hdfs dfs -appendToFile /test/file
2. Start writing to the file: hdfs dfs -appendToFile /test/file but do not stop writing
3. Run fedbalance: hadoop fedbalance submit hdfs://ns1/test hdfs://ns2/test
4. Write something to the file while fedbalance is running, "end" for example, then stop writing
5. After fedbalance is done, hdfs://ns2/test/file should only contain "start" while hdfs://ns1/user/hadoop/.Trash/Current/test/file contains "start\nend"

Fedbalance is run with default configs and arguments so no diff should happen.