|
|
|
HDFS-16499
|
HDFS-10285
[SPS]: Should not start indefinitely while another SPS process is running
|
Tao Li
|
Tao Li
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-16488
|
HDFS-10285
[SPS]: Expose metrics to JMX for external SPS
|
Tao Li
|
Tao Li
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-16485
|
HDFS-10285
[SPS]: allow re-satisfy path after restarting sps process
|
Unassigned
|
qinyuren
|
|
Open |
Unresolved
|
|
|
|
|
|
|
|
HDFS-16484
|
HDFS-10285
[SPS]: Fix an infinite loop bug in SPSPathIdProcessor thread
|
qinyuren
|
qinyuren
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-16477
|
HDFS-10285
[SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
|
Tao Li
|
Tao Li
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-16460
|
HDFS-10285
[SPS]: Handle failure retries for moving tasks
|
Tao Li
|
Tao Li
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-16458
|
HDFS-10285
[SPS]: Fix bug for unit test of reconfiguring SPS mode
|
Tao Li
|
Tao Li
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13808
|
HDFS-10285
[SPS]: Remove unwanted FSNamesystem #isFileOpenedForWrite() and #getFileInfo() function
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13381
|
HDFS-10285
[SPS]: Use DFSUtilClient#makePathFromFileId() to prepare satisfier file path
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13166
|
HDFS-10285
[SPS]: Implement caching mechanism to keep LIVE datanodes to minimize costly getLiveDatanodeStorageReport() calls
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13165
|
HDFS-10285
[SPS]: Collects successfully moved block details via IBR
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13110
|
HDFS-10285
[SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13097
|
HDFS-10285
[SPS]: Fix the branch review comments(Part1)
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13084
|
HDFS-10285
[SPS]: Fix the branch review comments
|
Rakesh Radhakrishnan
|
Uma Maheswara Rao G
|
|
Resolved |
Won't Fix
|
|
|
|
|
|
|
|
HDFS-13077
|
HDFS-10285
[SPS]: Fix review comments of external storage policy satisfier
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13076
|
HDFS-10285
[SPS]: Cleanup work for HDFS-10285
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13075
|
HDFS-10285
[SPS]: Provide External Context implementation.
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13057
|
HDFS-10285
[SPS]: Revisit configurations to make SPS service modes internal/external/none
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13050
|
HDFS-10285
[SPS] : Create start/stop script to start external SPS process.
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13033
|
HDFS-10285
[SPS]: Implement a mechanism to do file block movements for external SPS
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-13025
|
HDFS-10285
[SPS]: Implement a mechanism to scan the files for external SPS
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12995
|
HDFS-10285
[SPS] : Merge work for HDFS-10285 branch
|
Rakesh Radhakrishnan
|
Uma Maheswara Rao G
|
|
Resolved |
Information Provided
|
|
|
|
|
|
|
|
HDFS-12982
|
HDFS-10285
[SPS]: Reduce the locking and cleanup the Namesystem access
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12955
|
HDFS-10285
[SPS]: Move SPS classes to a separate package
|
Rakesh Radhakrishnan
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12911
|
HDFS-10285
[SPS]: Modularize the SPS code and expose necessary interfaces for external/internal implementations.
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12790
|
HDFS-10285
[SPS]: Rebasing HDFS-10285 branch after HDFS-10467, HDFS-12599 and HDFS-11968 commits
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12570
|
HDFS-10285
[SPS]: Refactor Co-ordinator datanode logic to track the block storage movements
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12556
|
HDFS-10285
[SPS] : Block movement analysis should be done in read lock.
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12310
|
HDFS-10285
[SPS]: Provide an option to track the status of in progress requests
|
Surendra Singh Lilhore
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12291
|
HDFS-10285
[SPS]: Provide a mechanism to recursively iterate and satisfy storage policy of all the files under the given dir
|
Surendra Singh Lilhore
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12225
|
HDFS-10285
[SPS]: Optimize extended attributes for tracking SPS movements
|
Surendra Singh Lilhore
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12214
|
HDFS-10285
[SPS]: Fix review comments of StoragePolicySatisfier feature
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12152
|
HDFS-10285
[SPS]: Re-arrange StoragePolicySatisfyWorker stopping sequence to improve thread cleanup time
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12146
|
HDFS-10285
[SPS] : Fix TestStoragePolicySatisfierWithStripedFile#testSPSWhenFileHasLowRedundancyBlocks
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12141
|
HDFS-10285
[SPS]: Fix checkstyle warnings
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-12106
|
HDFS-10285
[SPS]: Improve storage policy satisfier configurations
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11966
|
HDFS-10285
[SPS] Correct the log in BlockStorageMovementAttemptedItems#blockStorageMovementResultCheck
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11965
|
HDFS-10285
[SPS]: Should give chance to satisfy the low redundant blocks before removing the xattr
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11883
|
HDFS-10285
[SPS] : Handle NPE in BlockStorageMovementTracker when dropSPSWork() called
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11874
|
HDFS-10285
[SPS]: Document the SPS feature
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11762
|
HDFS-10285
[SPS] : Empty files should be ignored in StoragePolicySatisfier.
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11726
|
HDFS-10285
[SPS] : StoragePolicySatisfier should not select same storage type as source and destination in same datanode.
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11695
|
HDFS-10285
[SPS]: Namenode failed to start while loading SPS xAttrs from the edits log.
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11670
|
HDFS-10285
[SPS]: Add CLI command for satisfy storage policy operations
|
Surendra Singh Lilhore
|
Surendra Singh Lilhore
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11572
|
HDFS-10285
[SPS]: SPS should clean Xattrs when no blocks required to satisfy for a file
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11338
|
HDFS-10285
[SPS]: Fix timeout issue in unit tests caused by longger NN down time
|
Rakesh Radhakrishnan
|
Wei Zhou
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11336
|
HDFS-10285
[SPS]: Remove xAttrs when movements done or SPS disabled
|
Yuanbo Liu
|
Yuanbo Liu
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11334
|
HDFS-10285
[SPS]: NN switch and rescheduling movements can lead to have more than one coordinator for same file blocks
|
Rakesh Radhakrishnan
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11309
|
HDFS-10285
[SPS]: chooseTargetTypeInSameNode should pass accurate block size to chooseStorage4Block while choosing target
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11293
|
HDFS-10285
[SPS]: Local DN should be given preference as source node, when target available in same node
|
Uma Maheswara Rao G
|
Yuanbo Liu
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11289
|
HDFS-10285
[SPS]: Make SPS movement monitor timeouts configurable
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11284
|
HDFS-10285
[SPS]: fix issue of moving blocks with satisfier while changing replication factor
|
Yuanbo Liu
|
Yuanbo Liu
|
|
Resolved |
Invalid
|
|
|
|
|
|
|
|
HDFS-11264
|
HDFS-10285
[SPS]: Double checks to ensure that SPS/Mover are not running together
|
Rakesh Radhakrishnan
|
Wei Zhou
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11248
|
HDFS-10285
[SPS]: Handle partial block location movements
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11243
|
HDFS-10285
[SPS]: Add a protocol command from NN to DN for dropping the SPS work and queues
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11239
|
HDFS-10285
[SPS]: Check Mover file ID lease also to determine whether Mover is running
|
Wei Zhou
|
Wei Zhou
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11193
|
HDFS-10285
[SPS]: Erasure coded files should be considered for satisfying storage policy
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11186
|
HDFS-10285
[SPS]: Daemon thread of SPS should start only in Active NN
|
Wei Zhou
|
Wei Zhou
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11151
|
HDFS-10285
[SPS]: StoragePolicySatisfier should gracefully handle when there is no target node with the required storage type
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11150
|
HDFS-10285
[SPS]: Provide persistence when satisfying storage policy.
|
Yuanbo Liu
|
Yuanbo Liu
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11123
|
HDFS-10285
[SPS] Make storage policy satisfier daemon work on/off dynamically
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11068
|
HDFS-10285
[SPS]: Provide unique trackID to track the block movement sends to coordinator
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11032
|
HDFS-10285
[SPS]: Handling of block movement failure at the coordinator datanode
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-11029
|
HDFS-10285
[SPS]:Provide retry mechanism for the blocks which were failed while moving its storage at DNs
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-10954
|
HDFS-10285
[SPS]: Provide mechanism to send blocks movement result back to NN from coordinator DN
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-10885
|
HDFS-10285
[SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on
|
Wei Zhou
|
Wei Zhou
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-10884
|
HDFS-10285
[SPS]: Add block movement tracker to track the completion of block movement future tasks at DN
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-10802
|
HDFS-10285
[SPS]: Add satisfyStoragePolicy API in HdfsAdmin
|
Yuanbo Liu
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-10801
|
HDFS-10285
[SPS]: Protocol buffer changes for sending storage movement commands from NN to DN
|
Rakesh Radhakrishnan
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-10800
|
HDFS-10285
[SPS]: Daemon thread in Namenode to find blocks placed in other storage than what the policy specifies
|
Uma Maheswara Rao G
|
Uma Maheswara Rao G
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
HDFS-10794
|
HDFS-10285
[SPS]: Provide storage policy satisfy worker at DN for co-ordinating the block storage movement work
|
Rakesh Radhakrishnan
|
Rakesh Radhakrishnan
|
|
Resolved |
Fixed
|
|
|
|
|