Geronimo
  1. Geronimo
  2. GERONIMO-6270

Re-enable multiple instances support in one installation

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0-beta-1
    • Fix Version/s: 3.0.0, 3.0-beta-2
    • Component/s: None
    • Security Level: public (Regular issues)
    • Labels:
      None

      Description

      We need to investigate how to re-enable the multiple instances support in one installation.

      Refer to
      https://issues.apache.org/jira/browse/GERONIMO-6175
      https://issues.apache.org/jira/browse/GERONIMO-6174
      https://issues.apache.org/jira/browse/GERONIMO-5987

        Issue Links

          Activity

          Hide
          Russell E Glaue added a comment -

          All subtasks are resolved.

          I am now able to successfully run multiple instances from a single Geronimo installation base.

          Accordingly, I have finished updating the related documentation to reflect all changes made under this jira.

          I think we can close this jira now.

          Thanks everyone!
          -RG

          Show
          Russell E Glaue added a comment - All subtasks are resolved. I am now able to successfully run multiple instances from a single Geronimo installation base. Accordingly, I have finished updating the related documentation to reflect all changes made under this jira. https://cwiki.apache.org/confluence/display/GMOxDOC30/Configuring+multiple+repositories https://cwiki.apache.org/confluence/display/GMOxDOC30/Running+multiple+Geronimo+instances I think we can close this jira now. Thanks everyone! -RG
          Hide
          Forrest Xia added a comment -

          Fixed the mvn url cannot resolved issue:
          3.0-beta@1291884
          trunk@1291886

          The fix assume to define a secondary geronimo repository in location newinstance/repository with a plan like this:

          <module xmlns="http://geronimo.apache.org/xml/ns/deployment-1.2">
          <environment>
          <moduleId>
          <groupId>org.example.configs</groupId>
          <artifactId>instance2_repo</artifactId>
          <version>1.0</version>
          <type>car</type>
          </moduleId>
          <dependencies>
          <dependency>
          <groupId>org.apache.geronimo.framework</groupId>
          <artifactId>j2ee-system</artifactId>
          <type>car</type>
          </dependency>
          </dependencies>
          <hidden-classes/>
          <non-overridable-classes/>
          </environment>
          <gbean name="Repo2" class="org.apache.geronimo.system.repository.Maven2Repository">
          <attribute name="root">repository/</attribute>
          <attribute name="resolveToServer">true</attribute>
          <reference name="ServerInfo">
          <name>ServerInfo</name>
          </reference>
          </gbean>
          <gbean name="Local2" class="org.apache.geronimo.system.configuration.RepositoryConfigurationStore">
          <reference name="Repository">
          <name>Repo2</name>
          </reference>
          </gbean>
          </module>

          Show
          Forrest Xia added a comment - Fixed the mvn url cannot resolved issue: 3.0-beta@1291884 trunk@1291886 The fix assume to define a secondary geronimo repository in location newinstance/repository with a plan like this: <module xmlns="http://geronimo.apache.org/xml/ns/deployment-1.2"> <environment> <moduleId> <groupId>org.example.configs</groupId> <artifactId>instance2_repo</artifactId> <version>1.0</version> <type>car</type> </moduleId> <dependencies> <dependency> <groupId>org.apache.geronimo.framework</groupId> <artifactId>j2ee-system</artifactId> <type>car</type> </dependency> </dependencies> <hidden-classes/> <non-overridable-classes/> </environment> <gbean name="Repo2" class="org.apache.geronimo.system.repository.Maven2Repository"> <attribute name="root">repository/</attribute> <attribute name="resolveToServer">true</attribute> <reference name="ServerInfo"> <name>ServerInfo</name> </reference> </gbean> <gbean name="Local2" class="org.apache.geronimo.system.configuration.RepositoryConfigurationStore"> <reference name="Repository"> <name>Repo2</name> </reference> </gbean> </module>
          Hide
          Russell E Glaue added a comment -
          Yi Xiao added a comment - 13/Feb/12 07:33
          Hi Russell,
          Your comments are reasonable, and need more discussion. I think if we indeed add
          the GERONIMO_SERVER, it will also affect GERONIMO-6175, GERONIMO-6174, so we'd
          better open an task to track the changes.
          

          Hi Yi Xiao,
          I opened the subtask GERONIMO-6275 for this and added a patch.
          The patch needs review. The Windows batch files should be tested.
          If it is okay, I need someone to commit it for me.

          There might be a desire to discuss the GERONIMO_TMPDIR variable. Currently without this patch, GERONIMO_TMPDIR is constructed as "$GERONIMO_HOME"/var/temp in some scripts, and just var/temp in other scripts.
          With my patch GERONIMO_TMPDIR is always constructed as "$GERONIMO_SERVER"/var/temp

          Show
          Russell E Glaue added a comment - Yi Xiao added a comment - 13/Feb/12 07:33 Hi Russell, Your comments are reasonable, and need more discussion. I think if we indeed add the GERONIMO_SERVER, it will also affect GERONIMO-6175, GERONIMO-6174, so we'd better open an task to track the changes. Hi Yi Xiao, I opened the subtask GERONIMO-6275 for this and added a patch. The patch needs review. The Windows batch files should be tested. If it is okay, I need someone to commit it for me. There might be a desire to discuss the GERONIMO_TMPDIR variable. Currently without this patch, GERONIMO_TMPDIR is constructed as "$GERONIMO_HOME"/var/temp in some scripts, and just var/temp in other scripts. With my patch GERONIMO_TMPDIR is always constructed as "$GERONIMO_SERVER"/var/temp
          Hide
          Russell E Glaue added a comment - - edited

          Here is a summary of the problems I am aware of in enabling multiple instances:

          1. SnapshotConfigXMLBuilder: (not reported) - Wants GERONIMO_HOME/var/monitoring to exist, and will complain if it does not, however, the geronimo instance will continue to start regardless. If it cannot create the directory, Geronimo will show the following error when "Loading agent-car-jmx": ERROR [SnapshotConfigXMLBuilder] Could not make the directory /opt/geronimo-tomcat7-javaee6-3.0-SNAPSHOT-20111220/var/monitoring/
          2. ActiveMQ: GERONIMO-5987 (reported in comment) - ActiveMQ lock directory is specified relative to the working directory the user starts up the geronimo instance. Also, if ActiveMQ cannot create var/activemq Geronimo will fail to start with the following error when "Loading activemq-broker-blueprint": ERROR [BrokerService] Failed to start ActiveMQ JMS Message Broker. Reason: java.io.IOException: Failed to create directory 'var/activemq'
          3. GERONIMO_TMPDIR: GERONIMO-6174 - The start script sets the temp directory relative to GERONIMO_HOME, but instead should be set relative to the Geronimo Instance directory
          4. Karaf: GERONIMO-6174 - Karaf expects to use {karaf.home}

            /etc/shell.init.script each time a shell session is started, but I think ideally that should be

            {karaf.base}

            /etc/shell.init.script

          5. Repository: GERONIMO-6175 - The repository will use one in {org.geronimo.server.dir}

            /repository if configured, but some Geronimo code still expects a writable repository to exist at GERONIMO_HOME/repository. If it does not exist, Geronimo will fail to start with the following errors when "Loading rmi-naming": (1) WARN [RMIRegistryService] RMI Registry failed, (2) ERROR [GBeanInstanceState] Error while starting; GBean is now in the FAILED state: abstractName="org.apache.geronimo.framework/rmi-naming/3.0-beta-1/car?ServiceModule=org.apache.geronimo.framework/rmi-naming/3.0-beta-1/car,j2eeType=GBean,name=RMIRegistry"

          To see some of the complaints, the var directory must be removed from GERONIMO_HOME, and in its place you must create a file named var. This will prevent Geronimo from creating GERONIMO_HOME/var when it discovers it does not exist.

          Here is the procedure to create two geronimo instances:

          1. Follow steps in https://cwiki.apache.org/confluence/display/GMOxDOC30/Running+multiple+Geronimo+instances
          2. Use the below start script
          3. create an empty file GERONIMO_HOME/var
          4. To expose some issues, comment out the work-arounds and restart.

          Here is a start script I use to workaround some, but not all, of the issues as we are able.

          #!/bin/bash
          # Geronimo Named Instance start script
          # instance: gserv1
          
          # Uncomment this to explicitly set Geronimo's runtime Java
          #JAVA_HOME=/usr/jdk1.6.0_25
          #PATH=${JAVA_HOME}/bin:${PATH}
          
          # Set the location of the Geronimo Installation, and Geronimo Instance
          GHOME=/opt/geronimo3
          GVIRT=gserv1
          GSERV=${GHOME}/${GVIRT}
          
          #
          # The Next section works around some known issues
          #
          
          # GERONIMO-5987
          # Must change to the server directory in order to work around ActiveMQ lock
          # file conflict issue reported in GERONIMO-5987.
          cd ${GSERV}
          
          # GERONIMO-6174
          # Uncomment for the Workaround of issues in GERONIMO-6174
          GERONIMO_TMPDIR=${GSERV}/var/temp
          GERONIMO_OPTS="${GERONIMO_OPTS} -Dkaraf.home=${GSERV}"
          
          #
          # This Section starts the Geronimo Instance
          #
          export GERONIMO_OPTS="${GERONIMO_OPTS} -Dorg.apache.geronimo.server.dir=${GSERV}"
          export GERONIMO_TMPDIR
          export GERONIMO_HOME=${GHOME}
          export GERONIMO_SERVER=${GSERV}
          #
          # Uncomment for Normal startup, and comment out the "Interactive startup"
          #${GHOME}/bin/startup
          #
          # Interactive startup
          ${GHOME}/bin/geronimo run
          
          Show
          Russell E Glaue added a comment - - edited Here is a summary of the problems I am aware of in enabling multiple instances: SnapshotConfigXMLBuilder: (not reported) - Wants GERONIMO_HOME/var/monitoring to exist, and will complain if it does not, however, the geronimo instance will continue to start regardless. If it cannot create the directory, Geronimo will show the following error when "Loading agent-car-jmx": ERROR [SnapshotConfigXMLBuilder] Could not make the directory /opt/geronimo-tomcat7-javaee6-3.0-SNAPSHOT-20111220/var/monitoring/ ActiveMQ: GERONIMO-5987 (reported in comment) - ActiveMQ lock directory is specified relative to the working directory the user starts up the geronimo instance. Also, if ActiveMQ cannot create var/activemq Geronimo will fail to start with the following error when "Loading activemq-broker-blueprint": ERROR [BrokerService] Failed to start ActiveMQ JMS Message Broker. Reason: java.io.IOException: Failed to create directory 'var/activemq' GERONIMO_TMPDIR: GERONIMO-6174 - The start script sets the temp directory relative to GERONIMO_HOME, but instead should be set relative to the Geronimo Instance directory Karaf: GERONIMO-6174 - Karaf expects to use {karaf.home} /etc/shell.init.script each time a shell session is started, but I think ideally that should be {karaf.base} /etc/shell.init.script Repository: GERONIMO-6175 - The repository will use one in {org.geronimo.server.dir} /repository if configured, but some Geronimo code still expects a writable repository to exist at GERONIMO_HOME/repository. If it does not exist, Geronimo will fail to start with the following errors when "Loading rmi-naming": (1) WARN [RMIRegistryService] RMI Registry failed, (2) ERROR [GBeanInstanceState] Error while starting; GBean is now in the FAILED state: abstractName="org.apache.geronimo.framework/rmi-naming/3.0-beta-1/car?ServiceModule=org.apache.geronimo.framework/rmi-naming/3.0-beta-1/car,j2eeType=GBean,name=RMIRegistry" To see some of the complaints, the var directory must be removed from GERONIMO_HOME, and in its place you must create a file named var. This will prevent Geronimo from creating GERONIMO_HOME/var when it discovers it does not exist. Here is the procedure to create two geronimo instances: Follow steps in https://cwiki.apache.org/confluence/display/GMOxDOC30/Running+multiple+Geronimo+instances Use the below start script create an empty file GERONIMO_HOME/var To expose some issues, comment out the work-arounds and restart. Here is a start script I use to workaround some, but not all, of the issues as we are able. #!/bin/bash # Geronimo Named Instance start script # instance: gserv1 # Uncomment this to explicitly set Geronimo's runtime Java #JAVA_HOME=/usr/jdk1.6.0_25 #PATH=${JAVA_HOME}/bin:${PATH} # Set the location of the Geronimo Installation, and Geronimo Instance GHOME=/opt/geronimo3 GVIRT=gserv1 GSERV=${GHOME}/${GVIRT} # # The Next section works around some known issues # # GERONIMO-5987 # Must change to the server directory in order to work around ActiveMQ lock # file conflict issue reported in GERONIMO-5987. cd ${GSERV} # GERONIMO-6174 # Uncomment for the Workaround of issues in GERONIMO-6174 GERONIMO_TMPDIR=${GSERV}/var/temp GERONIMO_OPTS="${GERONIMO_OPTS} -Dkaraf.home=${GSERV}" # # This Section starts the Geronimo Instance # export GERONIMO_OPTS="${GERONIMO_OPTS} -Dorg.apache.geronimo.server.dir=${GSERV}" export GERONIMO_TMPDIR export GERONIMO_HOME=${GHOME} export GERONIMO_SERVER=${GSERV} # # Uncomment for Normal startup, and comment out the "Interactive startup" #${GHOME}/bin/startup # # Interactive startup ${GHOME}/bin/geronimo run
          Hide
          Yi Xiao added a comment -

          Hi Russell,
          Your comments are reasonable, and need more discussion. I think if we indeed add the GERONIMO_SERVER, it will also affect GERONIMO-6175, GERONIMO-6174, so we'd better open an task to track the changes.

          Show
          Yi Xiao added a comment - Hi Russell, Your comments are reasonable, and need more discussion. I think if we indeed add the GERONIMO_SERVER, it will also affect GERONIMO-6175 , GERONIMO-6174 , so we'd better open an task to track the changes.
          Hide
          Yi Xiao added a comment -

          There are two problems when enable multiple instance:
          (1)When two or more server instances start up, the activeMQ will try to lock the var/activemq directory, we can specify the activemq.data in var/config/config-subsitutions.properties to resolve it.
          (2)When deploy a application in a new repository besides the default one using
          deploy.bat -port RMI_port deploy --targets %other_repo% path/to/application, the application could not be started by the "mvn:groupid/artifactId/..." could not be resolved error. I think it's the URL handler couldn't resolve the new repo, still under investigation.

          Show
          Yi Xiao added a comment - There are two problems when enable multiple instance: (1)When two or more server instances start up, the activeMQ will try to lock the var/activemq directory, we can specify the activemq.data in var/config/config-subsitutions.properties to resolve it. (2)When deploy a application in a new repository besides the default one using deploy.bat -port RMI_port deploy --targets %other_repo% path/to/application, the application could not be started by the "mvn:groupid/artifactId/..." could not be resolved error. I think it's the URL handler couldn't resolve the new repo, still under investigation.
          Hide
          Russell E Glaue added a comment -

          Related: GERONIMO-5164 "Incomplete feature for deploy:new-instance command in geronimo3.0"

          Show
          Russell E Glaue added a comment - Related: GERONIMO-5164 "Incomplete feature for deploy:new-instance command in geronimo3.0"
          Hide
          Russell E Glaue added a comment -

          It is my doing that killed the org.apache.geronimo.base.dir java property and GERONIMO_BASE shell variable back in GERONIMO-4229 . We should not resurrect them, but the idea of GERONIMO_BASE should be brought back from the dead. We need to have a complete path reference to the running Geronimo instance of interest. Then we make sure we consistantly use it, and document its intentional meaning.

          We now have these three run-time path properties in Geronimo
          1. org.apache.geronimo.home.dir
          2. org.apache.geronimo.server.name
          3. org.apache.geronimo.server.dir (which can be "org.apache.geronimo.home.dir/org.apache.geronimo.server.name", but can also be a complete path to some directory outside the Geronimo install base)

          Currently, the property org.apache.geronimo.server.dir essentially is the "base" directory of the Named Server Instance. If a user supplies the property org.apache.geronimo.server.name, then the value for the property org.apache.geronimo.server.dir is generated from it relative to org.apache.geronimo.home.dir .

          To help developers more easily keep things separated in the head, I recommend we follow two policies. One for shell coding, and one for java coding. I suggest a new variable 'GERONIMO_SERVER' to hold the full path to the named server instance.
          1. In shell code, ALWAYS use the GERONIMO_SERVER variable for run-time configuration files and read-write data that is created or modified during run-time (This includes the Geronimo repository). Only use GERONIMO_HOME to reference binary code, or any static data that is expected to be read-only (not supposed to be modified).
          2. In Java code, ALWAYS use the org.apache.geronimo.server.dir property for run-time configuration files and read-write data that is created or modified during run-time (This includes the Geronimo repository). Only use org.apache.geronimo.home.dir to reference binary code, or any data that is expected to be read-only.

          • GERONIMO_SERVER can be initialized by the user as a relative path, or a full path. If the path is relative, then the shell code should expand it by prepending the value of GERONIMO_HOME .
          • org.apache.geronimo.server.dir is the same as GERONIMO_SERVER. If it does not reference a full path, the java code will expand it by prepending the value of org.apache.geronimo.home.dir .
          • If we treat org.apache.geronimo.server.dir as descibed previously, we no longer need org.apache.geronimo.server.name . However, since there is no harm or confusion at this time, we should keep it for now and treat it as a relative path to the geronimo server instance from org.apache.geronimo.home.dir, but only if org.apache.geronimo.server.dir is not defined. So org.apache.geronimo.server.dir would have precedence over org.apache.geronimo.server.name .

          We should make a standardization like this, and preach it in the developer documentation (perhaps page 1). For the last 5 years I have been involved in Geronimo, it seems like this detail gets overlooked from time to time. I don't pass blame though, it has been confusing on what the proper usage is. But we should start by stamping out the confusion.

          Also we should outline what we intend by "home" versus "base"
          When karaf was added to Geronimo, we got two new java properties: karaf.home and karaf.base . It is already not being used consistently as intended. As described in GERONIMO-6174, Karaf is written to look for

          {karaf.home}

          /etc/shell.init.script each time a shell session is started. When instead it should look for

          {karaf.base}

          /etc/shell.init.script . (See: geronimo/server/trunk/framework/configs/karaf-framework/src/main/distribution/text/etc/system.properties).
          From what I read/interpret from the karaf code, "base" is intended to be the path to the named server instance.

          Show
          Russell E Glaue added a comment - It is my doing that killed the org.apache.geronimo.base.dir java property and GERONIMO_BASE shell variable back in GERONIMO-4229 . We should not resurrect them, but the idea of GERONIMO_BASE should be brought back from the dead. We need to have a complete path reference to the running Geronimo instance of interest. Then we make sure we consistantly use it, and document its intentional meaning. We now have these three run-time path properties in Geronimo 1. org.apache.geronimo.home.dir 2. org.apache.geronimo.server.name 3. org.apache.geronimo.server.dir (which can be "org.apache.geronimo.home.dir/org.apache.geronimo.server.name", but can also be a complete path to some directory outside the Geronimo install base) Currently, the property org.apache.geronimo.server.dir essentially is the "base" directory of the Named Server Instance. If a user supplies the property org.apache.geronimo.server.name, then the value for the property org.apache.geronimo.server.dir is generated from it relative to org.apache.geronimo.home.dir . To help developers more easily keep things separated in the head, I recommend we follow two policies. One for shell coding, and one for java coding. I suggest a new variable 'GERONIMO_SERVER' to hold the full path to the named server instance. 1. In shell code, ALWAYS use the GERONIMO_SERVER variable for run-time configuration files and read-write data that is created or modified during run-time (This includes the Geronimo repository). Only use GERONIMO_HOME to reference binary code, or any static data that is expected to be read-only (not supposed to be modified). 2. In Java code, ALWAYS use the org.apache.geronimo.server.dir property for run-time configuration files and read-write data that is created or modified during run-time (This includes the Geronimo repository). Only use org.apache.geronimo.home.dir to reference binary code, or any data that is expected to be read-only. GERONIMO_SERVER can be initialized by the user as a relative path, or a full path. If the path is relative, then the shell code should expand it by prepending the value of GERONIMO_HOME . org.apache.geronimo.server.dir is the same as GERONIMO_SERVER. If it does not reference a full path, the java code will expand it by prepending the value of org.apache.geronimo.home.dir . If we treat org.apache.geronimo.server.dir as descibed previously, we no longer need org.apache.geronimo.server.name . However, since there is no harm or confusion at this time, we should keep it for now and treat it as a relative path to the geronimo server instance from org.apache.geronimo.home.dir, but only if org.apache.geronimo.server.dir is not defined. So org.apache.geronimo.server.dir would have precedence over org.apache.geronimo.server.name . We should make a standardization like this, and preach it in the developer documentation (perhaps page 1). For the last 5 years I have been involved in Geronimo, it seems like this detail gets overlooked from time to time. I don't pass blame though, it has been confusing on what the proper usage is. But we should start by stamping out the confusion. Also we should outline what we intend by "home" versus "base" When karaf was added to Geronimo, we got two new java properties: karaf.home and karaf.base . It is already not being used consistently as intended. As described in GERONIMO-6174 , Karaf is written to look for {karaf.home} /etc/shell.init.script each time a shell session is started. When instead it should look for {karaf.base} /etc/shell.init.script . (See: geronimo/server/trunk/framework/configs/karaf-framework/src/main/distribution/text/etc/system.properties). From what I read/interpret from the karaf code, "base" is intended to be the path to the named server instance.
          Hide
          Russell E Glaue added a comment -

          I have been cataloging mail list discussions, JIRA issues, Documentation, and other links on this external page regarding this matter.
          http://russ.glaue.org/proj/geronimo-enterprise/

          Show
          Russell E Glaue added a comment - I have been cataloging mail list discussions, JIRA issues, Documentation, and other links on this external page regarding this matter. http://russ.glaue.org/proj/geronimo-enterprise/
          Show
          Forrest Xia added a comment - Refer to doc for expected behaviors: 1. https://cwiki.apache.org/GMOxDOC21/multiple-repositories.html 2. https://cwiki.apache.org/GMOxDOC21/running-multiple-instances-of-geronimo.html 3. https://cwiki.apache.org/GMOxDOC30/running-multiple-geronimo-instances.html

            People

            • Assignee:
              Yi Xiao
              Reporter:
              Forrest Xia
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development