Version HA branch (HDFS-1623)

UNRELEASED

Start date not set

Release date not set

Release Notes

PTKeySummaryAssigneeStatusDevelopment
BlockerSub-taskHDFS-2767HA: ConfiguredFailoverProxyProvider should support NameNodeProtocolUma Maheswara Rao GResolved1 commit
BlockerSub-taskHDFS-2773HA: reading edit logs from an earlier version leaves blocks in under-construction stateTodd LipconResolved1 commit
CriticalSub-taskHDFS-2602NN should log newly-allocated blocks without losing BlockInfoAaron T. MyersResolved1 commit
CriticalSub-taskHDFS-2634Standby needs to ingest latest edit logs before transitioning to active`Todd LipconResolved1 commit
CriticalSub-taskHDFS-2671HA: NN should throw StandbyException in response to RPCs in STANDBY stateTodd LipconResolved1 commit
CriticalSub-taskHDFS-2683Authority-based lookup of proxy provider fails if path becomes canonicalizedTodd LipconResolved1 commit
CriticalSub-taskHDFS-2684Fix up some failing unit tests on HA branchTodd LipconResolved1 commit
CriticalSub-taskHDFS-2691HA: Tests and fixes for pipeline targets and replica recoveryTodd LipconResolved1 commit
CriticalSub-taskHDFS-2692HA: Bugs related to failover from/into safe-modeTodd LipconResolved1 commit
CriticalSub-taskHDFS-2693Synchronization issues around state transitionTodd LipconResolved1 commit
CriticalSub-taskHDFS-2709HA: Appropriately handle error conditions in EditLogTailerAaron T. MyersResolved1 commit
CriticalSub-taskHDFS-2795HA: Standby NN takes a long time to recover from a dead DN starting upTodd LipconResolved2 commits
CriticalSub-taskHDFS-2812When becoming active, NN should treat all leases as freshly renewedTodd LipconResolved1 commit
CriticalSub-taskHDFS-2841HA: HAAdmin does not work if security is enabledAaron T. MyersResolved2 commits
CriticalSub-taskHDFS-2853HA: NN fails to start if the shared edits dir is marked requiredAaron T. MyersResolved1 commit
CriticalSub-taskHDFS-2861HA: checkpointing should verify that the dfs.http.address has been configured to a non-loopback for peer NNTodd LipconResolved1 commit
CriticalSub-taskHDFS-2874HA: edit log should log to shared dirs before local dirsTodd LipconResolved1 commit
CriticalSub-taskHDFS-2904HA: Client support for getting delegation tokens to an HA clusterTodd LipconResolved1 commit
CriticalSub-taskHDFS-2924HA: Standby checkpointing fails to authenticate in secure clusterTodd LipconResolved1 commit
CriticalSub-taskHDFS-2929HA: stress test and fixes for block synchronizationTodd LipconResolved1 commit
CriticalBugHDFS-3019TestEditLogJournalFailures silently failingTodd LipconResolved1 commit
MajorSub-taskHDFS-1108Log newly allocated blocksTodd LipconClosed2 commits
MajorSub-taskHDFS-1971HA: Send block report from datanode to both active and standby namenodesSanjay RadiaResolved1 commit
MajorSub-taskHDFS-1972HA: Datanode fencing mechanismTodd LipconResolved1 commit
MajorSub-taskHDFS-1973HA: HDFS clients must handle namenode failover and switch over to the new active namenode.Aaron T. MyersResolved1 commit
MajorSub-taskHDFS-1975HA: Support for sharing the namenode state from active to standby.Jitendra Nath PandeyResolved1 commit
MajorImprovementHDFS-2197Refactor RPC call implementations out of NameNode classTodd LipconClosed4 commits
MajorSub-taskHDFS-2231Configuration changes for HA namenodeSuresh SrinivasResolved2 commits
MajorSub-taskHDFS-2291HA: Checkpointing in an HA setupTodd LipconResolved1 commit
MajorSub-taskHDFS-2301Start/stop appropriate namenode internal services during transition to active and standbySuresh SrinivasResolved1 commit
MajorSub-taskHDFS-2367HA: Enable the configuration of multiple HA cluster addressesAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2393Mark appropriate methods of ClientProtocol with the idempotent annotationAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2407getServerDefaults and getStats don't check operation categoryAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2418Change ConfiguredFailoverProxyProvider to take advantage of HDFS-2231Aaron T. MyersResolved1 commit
MajorSub-taskHDFS-2510Add HA-related metricsAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2577HA: NN fails to start since it tries to start secret manager in safemodeTodd LipconResolved1 commit
MajorSub-taskHDFS-2579Starting delegation token manager during safemode failsTodd LipconResolved1 commit
MajorSub-taskHDFS-2582Scope dfs.ha.namenodes config by nameserviceTodd LipconResolved1 commit
MajorSub-taskHDFS-2586Add protobuf service and implementation for HAServiceProtocolSuresh SrinivasResolved1 commit
MajorSub-taskHDFS-2591HA: MiniDFSCluster support to mix and match federation with HATodd LipconResolved1 commit
MajorSub-taskHDFS-2592HA: Balancer support for HA namenodesUma Maheswara Rao GResolved1 commit
MajorSub-taskHDFS-2612HA: handle refreshNameNodes in federated HA clustersTodd LipconResolved1 commit
MajorSub-taskHDFS-2616Change DatanodeProtocol#sendHeartbeat to return HeartbeatResponseSuresh SrinivasResolved2 commits
MajorSub-taskHDFS-2623HA: Add test case for hot standby capabilityTodd LipconResolved1 commit
MajorSub-taskHDFS-2624HA: ConfiguredFailoverProxyProvider doesn't correctly stop ProtocolTranslatorsTodd LipconResolved1 commit
MajorSub-taskHDFS-2626HA: BPOfferService.verifyAndSetNamespaceInfo needs to be synchronizedTodd LipconResolved1 commit
MajorSub-taskHDFS-2627HA: determine DN's view of which NN is active based on heartbeat responsesTodd LipconResolved1 commit
MajorSub-taskHDFS-2667HA: Fix NN Active->Standby transitionTodd LipconResolved1 commit
MajorSub-taskHDFS-2677HA: Web UI should indicate the NN stateEli CollinsResolved1 commit
MajorSub-taskHDFS-2678HA: When a FailoverProxyProvider is used, DFSClient should not retry connection ten times before failing overAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2679Add interface to query current state to HAServiceProtocol Eli CollinsResolved1 commit
MajorSub-taskHDFS-2682HA: When a FailoverProxyProvider is used, Client should not retry for 45 times(hard coded value) if it is timing out to connect to server.Uma Maheswara Rao GResolved1 commit
MajorSub-taskHDFS-2688HA: write tests for quota tracking and HATodd LipconResolved1 commit
MajorSub-taskHDFS-2689HA: BookKeeperEditLogInputStream doesn't implement isInProgress()Aaron T. MyersResolved1 commit
MajorSub-taskHDFS-2716HA: Configuration needs to allow different dfs.http.addresses for each HA NNTodd LipconResolved1 commit
MajorSub-taskHDFS-2720HA : TestStandbyIsHot is failing while copying in_use.lock file from NN1 nameSpaceDirs to NN2 nameSpaceDirs Uma Maheswara Rao GResolved1 commit
MajorSub-taskHDFS-2724NN web UI can throw NPE after startup, before standby state is enteredTodd LipconResolved1 commit
MajorSub-taskHDFS-2730HA: Refactor shared HA-related test code into HATestUtils classTodd LipconResolved1 commit
MajorSub-taskHDFS-2733Document HA configuration and CLIAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2737HA: Automatically trigger log rolls periodically on the active NNAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2747HA: entering safe mode after starting SBN can NPEUma Maheswara Rao GResolved1 commit
MajorSub-taskHDFS-2753Standby namenode stuck in safemode during a failoverHari MankudeResolved1 commit
MajorSub-taskHDFS-2762TestCheckpoint is timing outUma Maheswara Rao GResolved1 commit
MajorSub-taskHDFS-2772HA: On transition to active, standby should not swallow ELIEAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2775HA: TestStandbyCheckpoints.testBothNodesInStandbyState fails intermittentlyTodd LipconResolved1 commit
MajorSub-taskHDFS-2779HA: Add lease recovery handling to HASuresh SrinivasResolved
MajorSub-taskHDFS-2792HA: Make fsck workAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2794HA: Active NN may purge edit log files before standby NN has a chance to read themTodd LipconResolved1 commit
MajorSub-taskHDFS-2804SBN should not mark blocks under-replicated when exiting safemodeTodd LipconResolved1 commit
MajorSub-taskHDFS-2808HA: haadmin should use namenode idsEli CollinsResolved1 commit
MajorSub-taskHDFS-2824HA: failover does not succeed if prior NN died just after creating an edit log segmentAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2829HA: Fix ConfiguredFailoverProxyProvider for federationAaron T. MyersResolved
MajorSub-taskHDFS-2838HA: NPE in FSNamesystem when in safe modeGregory ChananResolved1 commit
MajorSub-taskHDFS-2845SBN should not allow browsing of the file system via web UIBikas SahaResolved1 commit
MajorSub-taskHDFS-2860HA: TestDFSRollback#testRollback is failingAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2863Failures observed if dfs.edits.dir and shared.edits.dir have same directories.Bikas SahaResolved1 commit
MajorSub-taskHDFS-2901HA: Improvements for SBN web UI - not show under-replicated/missing blocksBrandon LiResolved1 commit
MajorSub-taskHDFS-2912HA: Namenode not shutting down when shared edits dir is inaccessibleBikas SahaResolved1 commit
MajorSub-taskHDFS-2917HA: haadmin should not work if run by regular userEli CollinsResolved1 commit
MajorSub-taskHDFS-2920HA: fix remaining TODO itemsAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2922HA: close out operation categoriesEli CollinsResolved3 commits
MajorSub-taskHDFS-2935HA: Shared edits dir property should be suffixed with nameservice and namenodeIDTodd LipconResolved1 commit
MajorSub-taskHDFS-2937HA: TestDFSHAAdmin needs tests with MiniDFSClusterBrandon LiResolved1 commit
MajorSub-taskHDFS-2942HA: TestActiveStandbyElectorRealZK fails if build dir does not existAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2952HA: NN should not start with upgrade option or with a pending an unfinalized upgradeAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2955HA: IllegalStateException during standby startup in getCurSegmentTxIdHari MankudeResolved1 commit
MajorSub-taskHDFS-2958HA: Sweep for remaining proxy construction which doesn't go through failover pathAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2973HA: re-enable NO_ACK optimization for block deletionTodd LipconResolved1 commit
MajorSub-taskHDFS-2974HA: MiniDFSCluster does not delete standby NN name dirs during formatAaron T. MyersResolved1 commit
MajorSub-taskHDFS-2977HA: TestBalancerWithHANameNodes is failingAaron T. MyersResolved
MajorSub-taskHDFS-2979HA: Balancer should use logical uri for creating failover proxy with HA enabled.Aaron T. MyersResolved3 commits
MajorSub-taskHDFS-2993HA: BackupNode#checkOperation should permit CHECKPOINT operations Eli CollinsResolved1 commit
MajorSub-taskHDFS-3013HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configureMingjie LaiResolved1 commit
MajorImprovementHDFS-3023Optimize entries in edits log for persistBlocks callsTodd LipconResolved3 commits
MajorSub-taskHDFS-3027HA: Implement a simple NN health checkAaron T. MyersResolved1 commit
MajorSub-taskHDFS-3039HA: Address findbugs and javadoc warnings on branchTodd LipconResolved1 commit
MajorNew FeatureHDFS-4753GSOC 2013: Develop HLBS(HDFS Based Log-Structured Block Storage System) as a back-end storage driver for VMs like QEMU/XEN, etc.UnassignedOpen
MinorSub-taskHDFS-2622HA: fix TestDFSUpgrade on HA branchTodd LipconResolved1 commit
MinorSub-taskHDFS-2625HA: TestDfsOverAvroRpc failing after introduction of HeartbeatResponse typeTodd LipconResolved1 commit
MinorSub-taskHDFS-2680DFSClient should construct failover proxy with exponential backoffTodd LipconResolved1 commit
MinorSub-taskHDFS-2859LOCAL_ADDRESS_MATCHER.match has NPE when called from DFSUtil.getSuffixIDs when the host is incorrectBikas SahaResolved1 commit
MinorSub-taskHDFS-2890HA: DFSUtil#getSuffixIDs should skip unset configurationsAaron T. MyersResolved1 commit
MinorSub-taskHDFS-2915HA: TestFailureOfSharedDir.testFailureOfSharedDir() has race conditionBikas SahaResolved1 commit
MinorSub-taskHDFS-2928HA: ConfiguredFailoverProxyProvider should not create a NameNode proxy with an underlying retry proxyUma Maheswara Rao GResolved
MinorSub-taskHDFS-2934HA: Allow configs to be scoped to all NNs in the nameserviceTodd LipconResolved1 commit
MinorSub-taskHDFS-2939HA: TestHAStateTransitions fails on WindowsUma Maheswara Rao GResolved1 commit
MinorSub-taskHDFS-2947HA: On startup NN throws an NPE in the metrics systemAaron T. MyersResolved1 commit
MinorSub-taskHDFS-2948HA: NN throws NPE during shutdown if it fails to startupTodd LipconResolved2 commits
MinorSub-taskHDFS-2972HA: small optimization building incremental block reportTodd LipconResolved1 commit
MinorSub-taskHDFS-3035HA: fix failure of TestFileAppendRestart due to OP_UPDATE_BLOCKSTodd LipconResolved1 commit
TrivialSub-taskHDFS-2523NameNode needs to add HAServiceProtocol to its RPC ServerTodd LipconResolved1 commit
TrivialSub-taskHDFS-2714HA: Fix test cases which use standalone FSNamesystemsTodd LipconResolved1 commit
TrivialSub-taskHDFS-2820Add a simple sanity check for HA configTodd LipconResolved1 commit
TrivialSub-taskHDFS-2870HA: Remove some INFO level logging accidentally left aroundUnassignedResolved1 commit
1114 of 114