Version 2.7.4

RELEASED

Start date not set

Released: 04/Aug/17

Release Notes

2.7.4 releaseShow more
PTKeySummaryAssigneeStatusDevelopment
BlockerBugHDFS-10991Export hdfsTruncateFile symbol in libhdfsSurendra Singh LilhoreResolved4 commits
BlockerBugHDFS-11180Intermittent deadlock in NameNode when failover happens.Akira AjisakaResolved5 commits
BlockerBugHDFS-11229HDFS-11056 failed to close meta fileWei-Chiu ChuangResolved4 commits
BlockerBugHDFS-11609Some blocks can be permanently lost if nodes are decommissioned while deadKihwal LeeResolved6 commits
BlockerBugHDFS-11742Improve balancer usability after HDFS-8818Kihwal LeeResolved4 commits
BlockerBugHDFS-11743Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7Zhe ZhangResolved1 commit
BlockerBugHDFS-11896Non-dfsUsed will be doubled on dead node re-registrationBrahma Reddy BattulaResolved5 commits
CriticalBugHDFS-5042Completed files lost after power failureVinayakumar BResolved6 commits
CriticalBugHDFS-8312Trash does not descent into child directories to check for permissionsWeiwei YangResolved5 commits
CriticalBugHDFS-8498Blocks can be committed with wrong sizeJing ZhaoResolved4 commits
CriticalSub-taskHDFS-9621getListing wrongly associates Erasure Coding policy to pre-existing replicated files under an EC directory Jing ZhaoResolved5 commits
CriticalBugHDFS-9696Garbage snapshot records lingering foreverKihwal LeeClosed6 commits
CriticalBugHDFS-10301BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of orderVinitha Reddy GankidiResolved13 commits
CriticalBugHDFS-10763Open files can leak permanently due to inconsistent lease updateKihwal LeeClosed7 commits
CriticalBugHDFS-10987Make Decommission less expensive when lot of blocks present.Brahma Reddy BattulaResolved4 commits
CriticalBugHDFS-11263ClassCastException when we use Bzipcodec for Fsimage compressionBrahma Reddy BattulaResolved4 commits
CriticalBugHDFS-11352Potential deadlock in NN when failing overErik KrogenResolved3 commits
CriticalBugHDFS-11373Backport HDFS-11258 and HDFS-11272 to branch-2.7Akira AjisakaResolved1 commit
CriticalBugHDFS-11379DFSInputStream may infinite loop requesting block locationsDaryn SharpResolved5 commits
CriticalBugHDFS-11445FSCK shows overall health stauts as corrupt even one replica is corruptBrahma Reddy BattulaResolved7 commits
CriticalBugHDFS-11472Fix inconsistent replica size after a data pipeline failureErik KrogenResolved4 commits
CriticalBugHDFS-11608HDFS write crashed with block size greater than 2 GBXiaobing ZhouResolved5 commits
CriticalBugHDFS-11674reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recoveredVinayakumar BResolved6 commits
CriticalBugHDFS-11708Positional read will fail if replicas moved to different DNs after stream is openedVinayakumar BResolved5 commits
CriticalBugHDFS-11709StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefullyErik KrogenResolved9 commits
CriticalBugHDFS-11714Newly added NN storage directory won't get initialized and cause space exhaustionKihwal LeeResolved6 commits
MajorBugHDFS-7847Modify NNThroughputBenchmark to be able to operate on a remote NameNodeCharles LambResolved4 commits
MajorImprovementHDFS-7933fsck should also report decommissioning replicas. Xiaoyu YaoResolved4 commits
MajorBugHDFS-7990IBR delete ack should not be delayedDaryn SharpResolved3 commits
MajorImprovementHDFS-8200Refactor FSDirStatAndListingOpHaohui MaiResolved3 commits
MajorBugHDFS-8224Schedule a block for scanning if its metadata file is corruptRushabh S ShahResolved4 commits
MajorImprovementHDFS-8549Abort the balancer if an upgrade is in progressAndrew WangResolved3 commits
MajorBugHDFS-8682Should not remove decommissioned node,while calculating the number of live/dead decommissioned node.J.AndreinaResolved3 commits
MajorSub-taskHDFS-8721Add a metric for number of encryption zonesRakesh RResolved3 commits
MajorBugHDFS-8780Fetching live/dead datanode list with arg true for removeDecommissionNode,returns list with decom node.J.AndreinaResolved4 commits
MajorSub-taskHDFS-8818Allow Balancer to run fasterTsz Wo Nicholas SzeResolved7 commits
MajorSub-taskHDFS-8824Do not use small blocks for balancing the clusterTsz Wo Nicholas SzeResolved3 commits
MajorImprovementHDFS-8873Allow the directoryScanner to be rate-limitedDaniel TempletonResolved3 commits
MajorImprovementHDFS-8883NameNode Metrics : Add FSNameSystem lock Queue LengthAnu EngineerResolved3 commits
MajorBugHDFS-9038DFS reserved space is erroneously counted towards non-DFS used.Brahma Reddy BattulaResolved4 commits
MajorBugHDFS-9124NullPointerException when underreplicated blocks are thereSyed AkramResolved
MajorImprovementHDFS-9145Tracking methods that hold FSNamesytemLock for too longMingliang LiuResolved3 commits
MajorImprovementHDFS-9412getBlocks occupies FSLock and takes too long to completeHe TianyiResolved4 commits
MajorSub-taskHDFS-9421NNThroughputBenchmark replication test NPE with -namenode optionMingliang LiuResolved3 commits
MajorBugHDFS-9444Add utility to find set of available ephemeral ports to ServerSocketUtilMasatake IwasakiResolved4 commits
MajorBugHDFS-9467Fix data race accessing writeLockHeldTimeStamp in FSNamesystemMingliang LiuResolved3 commits
MajorBugHDFS-9500datanodesSoftwareVersions map may counting wrong when rolling upgradeErik KrogenResolved4 commits
MajorSub-taskHDFS-9503Replace -namenode option with -fs for NNThroughputBenchmarkMingliang LiuResolved4 commits
MajorSub-taskHDFS-9601NNThroughputBenchmark.BlockReportStats should handle NotReplicatedYetException on adding blockMasatake IwasakiResolved6 commits
MajorImprovementHDFS-9710Change DN to send block receipt IBRs in batchesTsz Wo Nicholas SzeResolved5 commits
MajorNew FeatureHDFS-9804Allow long-running Balancer to login with keytabXiao ChenResolved4 commits
MajorImprovementHDFS-9902Support different values of dfs.datanode.du.reserved per storage typeBrahma Reddy BattulaResolved4 commits
MajorBugHDFS-10276HDFS should not expose path info that user has no permission to see.Yuanbo LiuResolved4 commits
MajorBugHDFS-10336TestBalancer failing intermittently because of not reseting UserGroupInformation completelyYiqun LinResolved4 commits
MajorBugHDFS-10396Using -diff option with DistCp may get "Comparison method violates its general contract" exceptionYongjun ZhangResolved4 commits
MajorBugHDFS-10512VolumeScanner may terminate due to NPE in DataNode.reportBadBlocksYiqun LinResolved4 commits
MajorImprovementHDFS-10534NameNode WebUI should display DataNode usage histogramKai SasakiResolved7 commits
MajorBugHDFS-10544Balancer doesn't work with IPFailoverProxyProviderZhe ZhangResolved5 commits
MajorBugHDFS-10609Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applicationsWei-Chiu ChuangResolved5 commits
MajorSub-taskHDFS-10619Cache path in InodesInPathDaryn SharpResolved4 commits
MajorImprovementHDFS-10625 VolumeScanner to report why a block is found badRushabh S ShahResolved4 commits
MajorBugHDFS-10627Volume Scanner marks a block as "suspect" even if the exception is network-relatedRushabh S ShahResolved4 commits
MajorSub-taskHDFS-10653Optimize conversion from path string to componentsDaryn SharpClosed5 commits
MajorSub-taskHDFS-10655Fix path related byte array conversion bugsDaryn SharpResolved4 commits
MajorSub-taskHDFS-10656Optimize conversion of byte arrays back to path stringDaryn SharpResolved4 commits
MajorSub-taskHDFS-10662Optimize UTF8 string/byte conversionsDaryn SharpResolved4 commits
MajorSub-taskHDFS-10673Optimize FSPermissionChecker's internal path usageDaryn SharpResolved4 commits
MajorSub-taskHDFS-10674Optimize creating a full path from an inodeDaryn SharpResolved4 commits
MajorBugHDFS-10691FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsExceptionYiqun LinResolved4 commits
MajorBugHDFS-10693metaSave should print blocks, not LightWeightHashSetYuanbo LiuResolved4 commits
MajorBugHDFS-10694BlockManager.processReport() should print blockReportId in each log message.Yuanbo LiuResolved4 commits
MajorBugHDFS-10712Fix TestDataNodeVolumeFailure on 2.* branches.Vinitha Reddy GankidiResolved4 commits
MajorBugHDFS-10713Throttle FsNameSystem lock warningsHanisha KoneruResolved4 commits
MajorBugHDFS-10715NPE when applying AvailableSpaceBlockPlacementPolicyGuangbin ZhuResolved4 commits
MajorBugHDFS-10733NameNode terminated after full GC thinking QJM is unresponsive.Vinitha Reddy GankidiResolved4 commits
MajorSub-taskHDFS-10744Internally optimize path component resolutionDaryn SharpResolved4 commits
MajorSub-taskHDFS-10745Directly resolve paths into INodesInPathDaryn SharpResolved4 commits
MajorImprovementHDFS-10798Make the threshold of reporting FSNamesystem lock contention configurableErik KrogenResolved4 commits
MajorBugHDFS-10809getNumEncryptionZones causes NPE in branch-2.7Vinitha Reddy GankidiResolved1 commit
MajorImprovementHDFS-10817Add Logging for Long-held NN Read LocksErik KrogenResolved7 commits
MajorBugHDFS-10843Update space quota when a UC block is completed rather than committed.Erik KrogenResolved5 commits
MajorSub-taskHDFS-10872Add MutableRate metrics for FSNamesystemLock operationsErik KrogenResolved6 commits
MajorBugHDFS-10878TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationExceptionRushabh S ShahResolved4 commits
MajorBugHDFS-10879TestEncryptionZonesWithKMS#testReadWrite fails intermittentlyXiao ChenResolved4 commits
MajorBugHDFS-10889Remove outdated Fault Injection Framework documentaionBrahma Reddy BattulaResolved4 commits
MajorSub-taskHDFS-10896Move lock logging logic from FSNamesystem into FSNamesystemLockErik KrogenResolved4 commits
MajorBugHDFS-10921TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe modeEric BadgerResolved4 commits
MajorImprovementHDFS-10941Improve BlockManager#processMisReplicatesAsync logChen LiangResolved4 commits
MajorBugHDFS-10966Enhance Dispatcher logic on deciding when to give up a source DataNodeMark WagnerResolved4 commits
MajorBugHDFS-11002Fix broken attr/getfattr/setfattr links in ExtendedAttributes.mdMingliang LiuResolved4 commits
MajorBugHDFS-11015Enforce timeout in balancerKihwal LeeResolved4 commits
MajorBugHDFS-11053Unnecessary superuser check in versionRequest()Kihwal LeeResolved4 commits
MajorBugHDFS-11056Concurrent append and read operations lead to checksum errorWei-Chiu ChuangResolved8 commits
MajorImprovementHDFS-11069Tighten the authorization of datanode RPCKihwal LeeResolved4 commits
MajorBugHDFS-11078Fix NPE in LazyPersistFileScrubberÍñigo GoiriResolved1 commit
MajorBugHDFS-11087NamenodeFsck should check if the output writer is still writable.Erik KrogenResolved4 commits
MajorBugHDFS-11160VolumeScanner reports write-in-progress replicas as corrupt incorrectlyWei-Chiu ChuangResolved4 commits
MajorBugHDFS-11280Allow WebHDFS to reuse HTTP connections to NNZheng ShaoResolved12 commits
MajorTestHDFS-11290TestFSNameSystemMBean should wait until JMX cache is clearedErik KrogenResolved5 commits
MajorBugHDFS-11377Balancer hung due to no available mover threadsyunjiong zhaoResolved5 commits
MajorImprovementHDFS-11384Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spikeKonstantin ShvachkoResolved6 commits
MajorImprovementHDFS-11466Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000msAndrew WangResolved4 commits
MajorBugHDFS-11486Client close() should not fail fast if the last block is being decommissionedWei-Chiu ChuangResolved4 commits
MajorBugHDFS-11499Decommissioning stuck because of failing recoveryLukas MajercakResolved4 commits
MajorBugHDFS-11583Parent spans are not initialized to NullScope for every DFSPacketMasatake IwasakiResolved1 commit
MajorBugHDFS-11615FSNamesystemLock metrics can be inaccurate due to millisecond precisionErik KrogenResolved5 commits
MajorImprovementHDFS-11628Clarify the behavior of HDFS Mover in documentationXiaobing ZhouResolved5 commits
MajorImprovementHDFS-11634Optimize BlockIterator when iterating starts in the middle.Konstantin ShvachkoResolved6 commits
MajorImprovementHDFS-11648Lazy construct the IIP pathname Daryn SharpResolved6 commits
MajorBugHDFS-11736OIV tests should not write outside 'target' directory.Yiqun LinResolved8 commits
MajorBugHDFS-11741Long running balancer may fail due to expired DataEncryptionKeyWei-Chiu ChuangResolved6 commits
MajorBugHDFS-11766Fix findbugs warning in branch-2.7Chen LiangResolved1 commit
MajorBugHDFS-11795Fix ASF Licence warnings in branch-2.7Yiqun LinResolved1 commit
MajorBugHDFS-11849JournalNode startup failure exception should be logged in log fileSurendra Singh LilhoreResolved8 commits
MajorBugHDFS-11856Ability to re-add Upgrading Nodes (remote) to pipeline for future pipeline updatesVinayakumar BResolved6 commits
MajorBugHDFS-11864Document Metrics to track usage of memory for writes Yiqun LinResolved7 commits
MajorBugHDFS-11893Fix TestDFSShell.testMoveWithTargetPortEmpty failure.Brahma Reddy BattulaResolved7 commits
MajorBugHDFS-12177NameNode exits due to setting BlockPlacementPolicy loglevel to DebugJiandan Yang Resolved3 commits
MinorBugHDFS-6291FSImage may be left unclosed in BootstrapStandby#doRun()Sanghyun YunResolved3 commits
MinorImprovementHDFS-8131Implement a space balanced block placement policyLiu ShaohuiResolved3 commits
MinorImprovementHDFS-8709Clarify automatic sync in FSEditLog#logEditAndrew WangResolved4 commits
MinorBugHDFS-8915TestFSNamesystem.testFSLockGetWaiterCount fails intermittently in jenkinsMasatake IwasakiResolved4 commits
MinorImprovementHDFS-9019Adding informative message to sticky bit permission denied exceptionXiaoyu YaoResolved3 commits
MinorTestHDFS-9333Some tests using MiniDFSCluster errored complaining port in useMasatake IwasakiResolved4 commits
MinorImprovementHDFS-9726Refactor IBR code to a new classTsz Wo Nicholas SzeResolved4 commits
MinorTestHDFS-9745TestSecureNNWithQJM#testSecureMode sometimes fails with timeoutsXiao ChenResolved4 commits
MinorBugHDFS-9885Correct the distcp counters name while displaying countersSurendra Singh LilhoreResolved5 commits
MinorTestHDFS-9888Allow reseting KerberosName in unit testsXiao ChenResolved4 commits
MinorBugHDFS-10270TestJMXGet:testNameNode() failsGergely NovákResolved4 commits
MinorBugHDFS-10455Logging the username when deny the setOwner operationRakesh RResolved10 commits
MinorBugHDFS-10716In Balancer, the target task should be removed when its size < 0.Yiqun LinResolved
MinorBugHDFS-10747o.a.h.hdfs.tools.DebugAdmin usage message is misleadingMingliang LiuResolved4 commits
MinorImprovementHDFS-10807Doc about upgrading to a version of HDFS with snapshots may be confusingMingliang LiuResolved4 commits
MinorBugHDFS-10915Fix time measurement bug in TestDatanodeRestart#testWaitForRegistrationOnRestartXiaobing ZhouResolved4 commits
MinorBugHDFS-11174Wrong HttpFS test command in docJohn ZhugeResolved4 commits
MinorImprovementHDFS-11333Print a user friendly error message when plugins are not foundWei-Chiu ChuangResolved4 commits
MinorImprovementHDFS-11345Document the configuration key for FSNamesystem lock fairnessErik KrogenResolved4 commits
MinorTaskHDFS-11717Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefullyErik KrogenResolved5 commits
TrivialBugHDFS-5802NameNode does not check for inode type before traversing down a pathXiao ChenResolved3 commits
TrivialBugHDFS-8307Spurious DNS Queries from hdfs shellAndres PerezResolved1 commit
1140 of 140