Bulk Operation

HDFS-7285 ErasureCodingWorker#processErasureCodingTasks should not fail to process remaining tasks due to one invalid ECTask

HDFS-7285 Erasure coding: update EC command "-s" flag to "-p" when specifying policy

HDFS-7285 Erasure Coding: Provide DistributedFilesystem API to getAllErasureCodingPolicies

HDFS-7285 Cleanup erasure coding documentation

HDFS-7285 Rename dfs.datanode.stripedread.threshold.millis to dfs.datanode.stripedread.timeout.millis

HDFS-7285 Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)

HDFS-7285 Erasure coding: MapReduce job failed when I set the / folder to the EC zone

HDFS-7285 Erasure coding: fix 2 failed tests of DFSStripedOutputStream

HDFS-7285 Erasure coding : Fix random failure in TestSafeModeWithStripedFile

HDFS-7285 Erasure coding: do not throw exception when setting replication factor on EC files

HDFS-7285 Erasure Coding: when recovering lost blocks, logs can be too verbose and hurt performance

HDFS-7285 Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

HDFS-7285 Erasure Coding: use threadpool for EC recovery tasks on DataNode

HDFS-7285 Erasure Coding: Use datablocks, parityblocks and cell size from ErasureCodingPolicy

HDFS-7285 Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure

HDFS-7285 Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

HDFS-7285 Erasure Coding: Provide ECSchema validation when setting EC policy

HDFS-7285 Erasure Coding: client occasionally gets less block locations when some datanodes fail

HDFS-7285 Erasure Coding: Tolerate datanode failures in DFSStripedOutputStream when the data length is small

HDFS-7285 Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

HDFS-7285 Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks

HDFS-7285 Erasure Coding: add test for namenode process over replicated striped block

HDFS-7285 Erasure Coding: Client no need to decode missing parity blocks

HDFS-7285 Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation

HDFS-7285 Erasure Coding: fix DFSStripedInputStream/DFSStripedOutputStream re-fetch token when expired

HDFS-7285 Erasure coding: rename BlockInfoContiguousUC and BlockInfoStripedUC to be consistent with trunk

HDFS-7285 Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block

HDFS-7285 Erasure Coding: unit test for SequentialBlockGroupIdGenerator

HDFS-7285 Erasure Coding: reuse BlockReader when reading the same block in pread

HDFS-7285 Erasure Coding: the number of chunks in packet is not updated when writing parity data

HDFS-7285 Erasure Coding: fix one cell need two packets

HDFS-7285 Erasure Coding: Fail to read a file with corrupted blocks

HDFS-7285 Erasure coding: For a small file missing and under replicated ec-block calculation is incorrect

HDFS-7285 Erasure coding: revisit and simplify BlockInfoStriped and INodeFile

HDFS-7285 Erasure Coding: client generates too many small packets when writing parity data

HDFS-7285 Erasure coding: update BlockManager.blockHasEnoughRacks(..) logic for striped block

HDFS-7285 DFSClient hang up when there are not sufficient DataNodes in EC cluster.

HDFS-7285 Erasure Coding: fix some block number calculation for striped block

HDFS-7285 Erasure Coding: handle missing internal block locations in DFSStripedInputStream

HDFS-7285 Erasure Coding: revisit replica counting for striped blocks

HDFS-7285 Erasure Coding: Client can't read(decode) the EC files which have corrupt blocks.

HDFS-7285 Erasure Coding: fix the copy constructor of BlockInfoStriped and BlockInfoContiguous

HDFS-7285 Erasure Coding: Remove dataBlockNum and parityBlockNum from StripedBlockProto

HDFS-7285 Erasure coding: Persist cellSize in BlockInfoStriped and StripedBlocksFeature

HDFS-7285 Fix TestErasureCodingCli test

HDFS-7285 Erasure Coding: SafeMode handles file smaller than a full stripe

HDFS-7285 Erasure Coding: fsck handles file smaller than a full stripe

HDFS-7285 Erasure Coding: fix non-protobuf fsimage for striped blocks

HDFS-7285 Allow to configure RS and XOR raw coders

HDFS-7285 Erasure Coding: Fix usage of 'createZone'

HDFS-7285 Erasure Coding: Fix FindBugs Multithreaded correctness Warning

HDFS-7285 Erasure Coding: processOverReplicatedBlock() handles striped block

HDFS-7285 Restore ECZone info inside FSImageLoader

HDFS-7285 Fix a decoding issue in stripped block recovering in client side

HDFS-7285 ErasureCodingWorker fails to do decode work

HDFS-7285 Erasure coding: Two contiguous blocks occupy IDs belong to same striped group

HDFS-7285 Erasure coding: test DataNode reporting bad/corrupted blocks which belongs to a striped block.

HDFS-7285 Erasure coding: remove workarounds in client side stripped blocks recovering

HDFS-7285 Erasure coding: fix striping related logic in FSDirWriteFileOp to sync with HDFS-8421

HDFS-7285 createErasureCodingZone should check whether cellSize is available

HDFS-7285 2 RPC calls for every file read in DFSClient#open(..) resulting in double Audit log entries

HDFS-7285 Refactor BlockInfoContiguous and fix NPE in TestBlockInfo#testCopyConstructor()

HDFS-7285 Erasure coding: fix priority level of UnderReplicatedBlocks for striped block

HDFS-7285 Erasure Coding: stateful read result doesn't match data occasionally because of flawed test

HDFS-7285 Erasure coding: properly handle start offset for internal blocks in a block group

HDFS-7285 Erasure Coding: Consolidate erasure coding zone related implementation into a single class

HDFS-7285 Erasure Coding: fix cannot rename a zone dir

HDFS-7285 Erasure Coding: make condition check earlier for setReplication

HDFS-7285 set blockToken in LocatedStripedBlock

HDFS-7285 Erasure Coding: Fix the NullPointerException when deleting file

HDFS-7285 Remove dataBlockNum and parityBlockNum from BlockInfoStriped

HDFS-7285 Erasure Coding: ECZoneManager#getECZoneInfo is not resolving the path properly if zone dir itself is the snapshottable dir

HDFS-7285 Fix the isNeededReplication calculation for Striped block in NN

HDFS-7285 Erasure Coding: Pread failed to read data starting from not-first stripe

HDFS-7285 Revisit and refactor ErasureCodingInfo

HDFS-7285 Erasure Coding: unit test the behaviour of BlockManager recovery work for the deleted blocks

HDFS-7285 NN should consider current EC tasks handling count from DN while assigning new tasks

HDFS-7285 Remove chunkSize and initialize from erasure coder

HDFS-7285 Fix issues like NPE in TestRecoverStripedFile

HDFS-7285 Erasure Coding: Few improvements for the erasure coding worker

HDFS-7285 Add cellSize as an XAttr to ECZone

HDFS-7285 Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327

HDFS-7285 Erasure Coding: TestRecoverStripedFile#testRecoverOneParityBlock is failing

HDFS-7285 Erasure Coding: DFS opening a non-existent file need to be handled properly

HDFS-7285 BlockInfoStriped uses EC schema

HDFS-7285 Erasure Coding: Make the timeout parameter of polling blocking queue configurable in DFSStripedOutputStream

HDFS-7285 Erasure Coding: Badly treated when short of Datanode in StripedDataStreamer

HDFS-7285 Erasure coding: fix some minor bugs in EC CLI

HDFS-7285 Erasure Coding: DFSStripedInputStream#seekToNewSource

HDFS-7285 Fix FindBugs issues introduced by erasure coding

HDFS-7285 Erasure Coding: Refactor BlockInfo and BlockInfoUnderConstruction

HDFS-7285 Erasure Coding: test webhdfs read write stripe file

HDFS-7285 Erasure Coding: whether to use the same chunkSize in decoding with the value in encoding

HDFS-7285 Erasure Coding: test failed in TestDFSStripedInputStream.testStatefulRead() when use ByteBuffer

HDFS-7285 Erasure Coding: test skip in TestDFSStripedInputStream

HDFS-7285 Erasure Coding: Badly treated when createBlockOutputStream failed in DataStreamer

HDFS-7285 Expose some administrative erasure coding operations to HdfsAdmin

HDFS-7285 Erasure coding: rename DFSStripedInputStream related test classes

HDFS-7285 Erasure coding: Rename Striped block recovery to reconstruction to eliminate confusion.

HDFS-7285 Follow-on to update decode for DataNode striped blocks reconstruction

HDFS-7285 Add trace info to DFSClient#getErasureCodingZoneInfo(..)

HDFS-7285 Bump GenerationStamp for write faliure in DFSStripedOutputStream

HDFS-7285 Erasure coding: consolidate striping-related terminologies

HDFS-7285 Erasure Coding: support decoding for stateful read

HDFS-7285 Erasure coding: refactor EC constants to be consistent with HDFS-8249

HDFS-7285 Erasure Coding: DFSStripedOutputStream#close throws NullPointerException exception in some cases

HDFS-7285 Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog

HDFS-7285 Erasure Coding: Fix Findbug warnings present in erasure coding

HDFS-7285 Erasure Coding: add ECSchema to HdfsFileStatus

HDFS-7285 Refactor DFSStripedOutputStream and StripedDataStreamer

HDFS-7285 Erasure coding: move striped reading logic to StripedBlockUtil

HDFS-7285 Erasure Coding: implement parallel stateful reading for striped layout

HDFS-7285 Erasure Coding: Implement batched listing of enrasure coding zones

HDFS-7285 Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read)

HDFS-7285 Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure

HDFS-7285 DFSStripedOutputStream.closeThreads releases cellBuffers multiple times

HDFS-7285 Erasure Coding: XML based end-to-end test for ECCli commands

HDFS-7285 Erasure coding: [bug] should always allocate unique striped block group IDs

HDFS-7285 Erasure Coding: Create DFSStripedInputStream in DFSClient#open

HDFS-7285 Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last stripe is at the block group boundary

HDFS-7285 Erasure Coding: Ignore DatanodeProtocol#DNA_ERASURE_CODING_RECOVERY commands from standbynode if any

HDFS-7285 Erasure Coding: SequentialBlockGroupIdGenerator#nextValue may cause block id conflicts

HDFS-7285 Should calculate checksum for parity blocks in DFSStripedOutputStream

HDFS-7285 Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't satisfy BlockGroupSize

HDFS-7285 TestDFSStripedOutputStream should use BlockReaderTestUtil to create BlockReader

HDFS-7285 DistributedFileSystem.createErasureCodingZone should pass schema in FileSystemLinkResolver

HDFS-7285 Erasure Coding: Seek and other Ops in DFSStripedInputStream.

HDFS-7285 Improve end to end striping file test to add erasure recovering test

HDFS-7285 Erasure coding: Fix file quota change when we complete/commit the striped blocks

HDFS-7285 StripedBlockUtil.getInternalBlockLength may have overflow error

HDFS-7285 ClientProtocol#createErasureCodingZone API was wrongly annotated as Idempotent

HDFS-7285 Erasure coding: refactor client-related code to sync with HDFS-8082 and HDFS-8169

HDFS-7285 Erasure coding: Make block placement policy for EC file configurable

HDFS-7285 Erasure Coding: Improve DFSStripedOutputStream closing of datastreamer threads

HDFS-7285 createErasureCodingZone sets retryCache state as false always

HDFS-7285 Failure handling: DFSStripedOutputStream continues writing with enough remaining datanodes

HDFS-7285 Erasure Coding: Keep default schema's name consistent

HDFS-7285 BlockManager.addBlockCollectionWithCheck should check if the block is a striped block

HDFS-7285 DFSStripedOutputStream should not create empty blocks

HDFS-7285 Add/implement necessary APIs even we just have the system default schema

HDFS-7285 Protobuf changes for BlockECRecoveryCommand and its fields for making it ready for transfer to DN

HDFS-7285 Fix the editlog corruption exposed by failed TestAddStripedBlocks

HDFS-7285 Send the EC schema to DataNode via EC encoding/recovering command

HDFS-7285 Client gets and uses EC schema when reads and writes a stripping file

HDFS-7285 Erasure Coding: Maintain consistent naming for Erasure Coding related classes - EC/ErasureCoding

HDFS-7285 DFSStripedInputStream fails to read data after one stripe

HDFS-7285 Erasure Coding: Better to move EC related proto messages to a separate erasurecoding proto file

HDFS-7285 Erasure Coding: Support specifying ECSchema during creation of ECZone

HDFS-7285 BlockManager treates good blocks in a block group as corrput

HDFS-7285 Erasure coding: created util class to analyze striped block groups

HDFS-7285 Erasure coding: Add auditlog FSNamesystem#createErasureCodingZone if this operation fails

HDFS-7285 Make hard-coded values consistent with the system default schema first before remove them

HDFS-7285 Erasure coding: fix bug in TestFSImage

HDFS-7285 Erasure Coding: Add RPC to client-namenode to list all ECSchemas loaded in Namenode.

HDFS-7285 Erasure coding: fix bug in EC zone and symlinks

HDFS-7285 Define a system-wide default EC schema

HDFS-7285 Erasure coding: Decommission handle for EC blocks.

HDFS-7285 Erasure coding: DataNode support for block recovery of striped block groups

HDFS-7285 Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

HDFS-7285 Erasure coding: stateful (non-positional) read from files in striped layout

HDFS-7285 Erasure Coding: Update CHANGES-HDFS-7285.txt with branch commits

HDFS-7285 Erasure Coding: ECworker frame, basics, bootstraping and configuration

HDFS-7285 Erasure Coding: retrieve eraure coding schema for a file from NameNode

HDFS-7285 Erasure coding: extend UnderReplicatedBlocks to accurately handle striped blocks

HDFS-7285 Erasure Coding: simplify striped block recovery work computation and add tests

HDFS-7285 Detect if resevered EC Block ID is already used

HDFS-7285 getStoragePolicy() regards HOT policy as EC policy

HDFS-7285 Erasure coding: NameNode support for lease recovery of striped block groups

HDFS-7285 WebImageViewer need support file size calculation with striped blocks

HDFS-7285 Erasure Coding: INodeFile quota computation unit tests

HDFS-7285 Erasure coding: resolving conflicts in the branch when merging trunk changes.

HDFS-7285 Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks

HDFS-7285 Erasure Coding: track invalid, corrupt, and under-recovery striped blocks in NameNode

HDFS-7285 Subclass DFSOutputStream to support writing striping layout files

HDFS-7285 Erasure Coding: INodeFile.dumpTreeRecursively() supports to print striped blocks

HDFS-7285 Erasure Coding: Update safemode calculation for striped blocks

HDFS-7285 Erasure coding: extend LocatedBlocks to support reading from striped files

HDFS-7285 Erasure coding: implement facilities in NameNode to create and manage EC zones

HDFS-7285 Erasure Coding: allocate and persist striped blocks in NameNode

HDFS-7285 Erasure Coding: support striped blocks in non-protobuf fsimage

HDFS-7285 Erasure Coding: Update INodeFile quota computation for striped blocks

HDFS-7285 Support appending to a striping layout file

HDFS-7285 Erasure coding: pread from files in striped layout

HDFS-7285 Erasure Coding: Add striped block support in INodeFile

HDFS-7285 Add logic to DFSOutputStream to support writing a file in striping layout

HDFS-7285 Erasure Coding: extend BlockInfo to handle EC info

HDFS-7285 Avoid Block movement in Balancer and Mover for the erasure encoded blocks

HDFS-7285 Add periodic checker to find the corrupted EC blocks/files

HDFS-7285 Client side api/config changes to support online encoding

HDFS-7285 Change fsck to support EC files

HDFS-7285 Erasure coding: DFSInputStream with decode functionality (pread)

HDFS-7285 Erasure Coding: consolidate streamer coordination logic and handle failure when writing striped blocks

HDFS-7285 Process block reports for erasure coded blocks

HDFS-7285 Erasure Coding: update the Balancer/Mover data migration logic

HDFS-7285 Change disk quota calculation for EC files

HDFS-7285 Add unit tests for editlog transactions for EC

HDFS-7285 Add a test for BlockGroup support in FSImage

HDFS-7285 Implement COMPLETE state of erasure coding block groups

HDFS-7285 [umbrella] Data striping support in HDFS client

HDFS-7285 Erasure coding: distribute recovery work for striped blocks to DataNode

HDFS-7285 Document the HDFS Erasure Coding feature

HDFS-7285 Support DFS command for the EC encoding

HDFS-7285 Erasure Coding: DataNode reconstruct striped blocks

HDFS-7285 Configurable erasure coding policy for individual files and directories

HDFS-7285 Representing striped block groups in NameNode with hierarchical naming protocol