Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8574

Enable starting hadoop services from inside OSGi

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • None
    • None

    Description

      This JIRA captures the needed things in order to start hadoop services in OSGi.

      The main idea I used so far consists in:

      • using the OSGi ConfigAdmin to store the hadoop configuration
      • in that configuration, use a few boolean properties to determine which services should be started (nameNode, dataNode ...)
      • expose a configured url handler so that the whole OSGi runtime can use urls in hdfs:/xxx
      • the use of an OSGi ManagedService means that when the configuration changes, the services will be stopped and restarted with the new configuration

      Attachments

        Issue Links

          Activity

            There is already some Hadoop Jiras related to that (I will link to this one just for tracking).

            jbonofre Jean-Baptiste Onofré added a comment - There is already some Hadoop Jiras related to that (I will link to this one just for tracking).

            Possible patch to kick the discussion: https://github.com/gnodet/hadoop-common/commit/742ab08aa068424fc2292cf1cd2d64a345053173
            Though the OSGi metadata are not yet there, so this is not really testable yet (will upload a patch for that soon or JB).

            There is one possibly controversial change which is the one in the Configuration (see https://github.com/gnodet/hadoop-common/commit/742ab08aa068424fc2292cf1cd2d64a345053173#L3R207).
            The idea is that in OSGi, the whole configuration is controlled (at least the default) by ConfigAdmin. The benefit is that clients don't really have to deal with configuration.
            One thing I haven't really understood is why the configuration isn't a global singleton (at least the defaults), as the configuration files are being read multiple times (each time a new configuration is created).

            gnodet Guillaume Nodet added a comment - Possible patch to kick the discussion: https://github.com/gnodet/hadoop-common/commit/742ab08aa068424fc2292cf1cd2d64a345053173 Though the OSGi metadata are not yet there, so this is not really testable yet (will upload a patch for that soon or JB). There is one possibly controversial change which is the one in the Configuration (see https://github.com/gnodet/hadoop-common/commit/742ab08aa068424fc2292cf1cd2d64a345053173#L3R207 ). The idea is that in OSGi, the whole configuration is controlled (at least the default) by ConfigAdmin. The benefit is that clients don't really have to deal with configuration. One thing I haven't really understood is why the configuration isn't a global singleton (at least the defaults), as the configuration files are being read multiple times (each time a new configuration is created).

            Yep, I take the OSGi metadata and merge back the Karaf feature descriptor. It will be done over the week end.

            jbonofre Jean-Baptiste Onofré added a comment - Yep, I take the OSGi metadata and merge back the Karaf feature descriptor. It will be done over the week end.

            People

              Unassigned Unassigned
              gnodet Guillaume Nodet
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: