Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5257

Export namenode/datanode functionality through a pluggable RPC layer

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      New plugin facility for namenode and datanode instantiates classes named in new configuration properties dfs.datanode.plugins and dfs.namenode.plugins.

      Description

      Adding support for pluggable components would allow exporting DFS functionallity using arbitrary protocols, like Thirft or Protocol Buffers. I'm opening this issue on Dhruba's suggestion in HADOOP-4707.

      Plug-in implementations would extend this base class:

      abstract class Plugin {
      
          public abstract datanodeStarted(DataNode datanode);
      
          public abstract datanodeStopping();
      
          public abstract namenodeStarted(NameNode namenode);
      
          public abstract namenodeStopping();
      }

      Name node instances would then start the plug-ins according to a configuration object, and would also shut them down when the node goes down:

      public class NameNode {
      
          // [..]
      
          private void initialize(Configuration conf)
              // [...]
              for (Plugin p: PluginManager.loadPlugins(conf))
                p.namenodeStarted(this);
          }
      
          // [..]
      
          public void stop() {
              if (stopRequested)
                  return;
              stopRequested = true;
              for (Plugin p: plugins) 
                  p.namenodeStopping();
              // [..]
          }
      
          // [..]
      }

      Data nodes would do a similar thing in DataNode.startDatanode() and DataNode.shutdown

        Attachments

        1. HADOOP-5257-v8.patch
          11 kB
          Carlos Valiente
        2. HADOOP-5257-v7.patch
          11 kB
          Carlos Valiente
        3. HADOOP-5257-v6.patch
          11 kB
          Carlos Valiente
        4. HADOOP-5257-v5.patch
          11 kB
          Carlos Valiente
        5. HADOOP-5257-v4.patch
          15 kB
          Carlos Valiente
        6. HADOOP-5257-v3.patch
          16 kB
          Carlos Valiente
        7. HADOOP-5257-v2.patch
          17 kB
          Carlos Valiente
        8. HADOOP-5257.patch
          20 kB
          Carlos Valiente

          Issue Links

            Activity

              People

              • Assignee:
                carlos.valiente Carlos Valiente
                Reporter:
                carlos.valiente Carlos Valiente
              • Votes:
                1 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: