Hadoop Common
  1. Hadoop Common
  2. HADOOP-7150

Create a module system for adding extensions to Hadoop

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: util
    • Labels:
      None

      Description

      Currently adding extensions to Hadoop is difficult. I propose adding the concept of modules that can add jars and libraries to the system.

      1. modules.pdf
        100 kB
        Owen O'Malley
      2. modules.tex
        7 kB
        Owen O'Malley

        Activity

        Hide
        Philip Zeyliger added a comment -

        Owen,

        To clarify, are you referring about adding libraries for user MapReduce code (e.g., an image manipulation library) or more about adding plugins/modules to the core services (e.g., https://issues.apache.org/jira/browse/HADOOP-5257, "adding service plugins for namenode/datanode"; there's a similar MR ticket).

        Both are very handy; just curious which one you're thinking about.

        Thanks!

        Show
        Philip Zeyliger added a comment - Owen, To clarify, are you referring about adding libraries for user MapReduce code (e.g., an image manipulation library) or more about adding plugins/modules to the core services (e.g., https://issues.apache.org/jira/browse/HADOOP-5257 , "adding service plugins for namenode/datanode"; there's a similar MR ticket). Both are very handy; just curious which one you're thinking about. Thanks!
        Hide
        Owen O'Malley added a comment -

        a description of modules solution

        Show
        Owen O'Malley added a comment - a description of modules solution
        Hide
        Owen O'Malley added a comment -

        It is primarily for clients, like adding filesystems, codecs, serializations, etc. It can also be used in the servers for adding things like filesystems, compression codecs, and metrics plugins.

        Show
        Owen O'Malley added a comment - It is primarily for clients, like adding filesystems, codecs, serializations, etc. It can also be used in the servers for adding things like filesystems, compression codecs, and metrics plugins.
        Hide
        Owen O'Malley added a comment -

        The latex source code for the document

        Show
        Owen O'Malley added a comment - The latex source code for the document

          People

          • Assignee:
            Owen O'Malley
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

            • Created:
              Updated:

              Development