Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12615

Router-based HDFS federation phase 2

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      This umbrella JIRA tracks set of improvements over the Router-based HDFS federation (HDFS-10467).

        Attachments

          Issue Links

          1.
          RBF: Add WebHDFS Sub-task Resolved Wei Yan  
          2.
          RBF: Improve State Store FS implementation Sub-task Resolved Íñigo Goiri  
          3.
          RBF: Test Router-based federation using HDFSContract Sub-task Resolved Íñigo Goiri  
          4.
          RBF: Set MountTableResolver as default file resolver Sub-task Resolved Íñigo Goiri  
          5.
          RBF: Control MountTableResolver cache size Sub-task Resolved Íñigo Goiri  
          6.
          RBF: Fix Javadoc parameter errors Sub-task Resolved Wei Yan  
          7.
          RBF: Add router admin commands usage in HDFS commands reference doc Sub-task Resolved Yiqun Lin  
          8.
          RBF: Complete logic for -readonly option of dfsrouteradmin add command Sub-task Resolved Íñigo Goiri  
          9.
          RBF: Document Router and State Store metrics Sub-task Resolved Yiqun Lin  
          10.
          RBF: Add ACL support for mount table Sub-task Resolved Yiqun Lin  
          11.
          RBF: Add more unit tests for router admin commands Sub-task Resolved Yiqun Lin  
          12.
          RBF: Mount table entries not properly updated in the local cache Sub-task Resolved Íñigo Goiri  
          13.
          RBF: Support erasure coding methods in RouterRpcServer Sub-task Resolved Íñigo Goiri  
          14.
          RBF: Fix spurious TestRouterRpc#testProxyGetStats Sub-task Resolved Íñigo Goiri  
          15.
          RBF: Inconsistent Router OPTS config in branch-2 and branch-3 Sub-task Resolved Wei Yan  
          16.
          RBF: Fix doc error setting up client Sub-task Resolved tartarus

          0%

          Original Estimate - 24h
          Remaining Estimate - 24h
          17.
          RBF: Use the ZooKeeper as the default State Store Sub-task Resolved Yiqun Lin  
          18.
          RBF: Manage unavailable clusters Sub-task Resolved Yiqun Lin  
          19.
          RBF: Improve the unit test TestRouterRPCClientRetries Sub-task Resolved Yiqun Lin  
          20.
          RBF: Fix Routers information shown in the web UI Sub-task Resolved Wei Yan  
          21.
          RBF: Fix the hdfs router page missing label icon issue Sub-task Resolved maobaolong  
          22.
          RBF: Optimize name service safe mode icon Sub-task Resolved liuhongtong  
          23.
          RBF: Mount path not available after ADD-REMOVE-ADD Sub-task Resolved Wei Yan  
          24.
          RBF: Fix router location cache issue Sub-task Resolved Weiwei Wu  
          25.
          RBF: Complete document of Router configuration Sub-task Resolved Yiqun Lin  
          26.
          RBF: Move Router to its own module Sub-task Resolved Wei Yan  
          27.
          RBF: Resolvers to support mount points across multiple subclusters Sub-task Resolved Íñigo Goiri  
          28.
          RBF: Router to manage requests across multiple subclusters Sub-task Resolved Íñigo Goiri  
          29.
          RBF: Implement available space based OrderResolver Sub-task Resolved Yiqun Lin  
          30.
          [Documentation] RBF: Mount points across multiple subclusters Sub-task Patch Available Íñigo Goiri  
          31.
          RBF: Throw the exception if mount table entry validated failed Sub-task Resolved maobaolong  
          32.
          RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns Sub-task Resolved Chao Sun  
          33.
          RBF: ConnectionPool should return first usable connection Sub-task Resolved Ekanth S  
          34.
          RBF: Update some inaccurate document descriptions Sub-task Resolved Yiqun Lin  
          35.
          RBF: Improve error message returned from subcluster Sub-task Resolved Íñigo Goiri  
          36.
          RBF: The RouterRPCServer should transfer CallerContext and client ip to NamenodeRpcServer Sub-task Resolved Unassigned  
          37.
          RBF: MountTableResolver doesn't return the correct mount point of the given path Sub-task Resolved wangzhiyuan  
          38.
          RBF: Cannot mv/cp file cross namespace Sub-task Open Unassigned  
          39.
          RBF: Router audit logger Sub-task Open Unassigned  
          40.
          RBF: Namenode need to choose block location for the client Sub-task Open Íñigo Goiri  
          41.
          RBF: Extend RouterRpcClient to use multiple sockets Sub-task Open Íñigo Goiri  
          42.
          RBF: Fail when try to remove mount point paths Sub-task Open Unassigned  
          43.
          RBF: TestConnectionManager#testCleanup() test case need correction Sub-task Resolved Dibyendu Karmakar  
          44.
          RBF : Fix compilation error in branch-2 (TestMultipleDestinationResolver) Sub-task Resolved Brahma Reddy Battula  
          45.
          RBF: TestRouterAdminCLI#testCreateInvalidEntry fails on Windows Sub-task Resolved Íñigo Goiri  
          46.
          RBF: Fix FindBugs in hadoop-hdfs-rbf Sub-task Resolved Ekanth S  
          47.
          RBF: Improve the interfaces to modify and view mount tables Sub-task Resolved Gang Li  
          48.
          RBF: Cache datanode reports Sub-task Resolved Íñigo Goiri  
          49.
          RBF: Add xsl stylesheet for hdfs-rbf-default.xml Sub-task Resolved Takanobu Asanuma  
          50.
          RBF: TestRouterWebHDFSContractCreate failed Sub-task Resolved Takanobu Asanuma  
          51.
          RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails Sub-task Open Takanobu Asanuma  
          52.
          RBF: Support NamenodeProtocol in the Router Sub-task Resolved Íñigo Goiri  
          53.
          RBF: Improve timeout RPC call mechanism Sub-task Resolved Íñigo Goiri  
          54.
          RBF: Wrong date information in list file(-ls) result Sub-task Resolved Dibyendu Karmakar  
          55.
          RBF: Support federation with no subclusters Sub-task Resolved Íñigo Goiri  
          56.
          RBF: Fix java doc for StateStoreFileSystemImpl Sub-task Resolved Yiran Wu  
          57.
          RBF: Remove LinkedList From StateStoreFileImpl.java Sub-task Resolved BELUGA BEHR  
          58.
          RBF: Fix dead links in RBF document Sub-task Resolved Chetna Chaudhari  
          59.
          RBF: Improve the error loggings for printing the stack trace Sub-task Resolved Yiqun Lin  
          60.
          RBF: getMountPointDates should fetch latest subdir time/date when parent dir is not present but /parent/child dirs are present in mount table Sub-task Resolved Dibyendu Karmakar  
          61.
          RBF: Add more router-related information to the UI Sub-task Resolved Wei Yan  
          62.
          RBF: Disabled Nameservice store API Sub-task Resolved Íñigo Goiri  
          63.
          RBF: Disable Nameservices from the federation Sub-task Resolved Íñigo Goiri  
          64.
          RBF: Show disabled name services in the UI Sub-task Resolved Íñigo Goiri  
          65.
          RBF: Add unit test TestStateStoreDisabledNameservice Sub-task Resolved Yiqun Lin  
          66.
          RBF: Reject requests when a Router is overloaded Sub-task Resolved Íñigo Goiri  
          67.
          RBF: Fix setSafeMode in the Router Sub-task Resolved Íñigo Goiri  
          68.
          RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries Sub-task Resolved Ekanth S  
          69.
          RBF: State store DBMS implementation Sub-task Patch Available Yiran Wu  
          70.
          RBF: Correct the logic of mount validate to avoid the bad mountPoint Sub-task Open Unassigned  
          71.
          RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Sub-task Patch Available Mohammad Arshad  
          72.
          RBF: Adding trace support Sub-task Patch Available Íñigo Goiri  
          73.
          RBF: Support InodeID in the Router Sub-task Open Unassigned  
          74.
          RBF: Add Browse the Filesystem button to the UI Sub-task Patch Available Íñigo Goiri  
          75.
          RBF: Support Router Admin REST API Sub-task Open Mohammad Arshad  
          76.
          RBF: Cluster information on Router is not correct when the Federation shares datanodes Sub-task Open Unassigned  
          77.
          RBF: Create destination directory when adding mount entry using router admin cmds. Sub-task Open Dibyendu Karmakar  
          78.
          RBF: Remove update functionality from routeradmin's add cmd Sub-task Patch Available Gang Li  
          79.
          RBF: Add destination path length validation for add/update mount entry Sub-task Open Unassigned  
          80.
          RBF: Failed mount point operations, returns wrong exit code. Sub-task Open Dibyendu Karmakar  

            Activity

              People

              • Assignee:
                elgoiri Íñigo Goiri
                Reporter:
                elgoiri Íñigo Goiri
              • Votes:
                1 Vote for this issue
                Watchers:
                24 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 24h
                  24h
                  Remaining:
                  Remaining Estimate - 24h
                  24h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified