Accumulo
  1. Accumulo
  2. ACCUMULO-1550

Create ACCUMULO_CONF_DIR environment variable

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.3, 1.5.0
    • Fix Version/s: 1.4.4, 1.5.1, 1.6.0
    • Component/s: scripts
    • Labels:
      None

      Description

      Hadoop (and many of the other projects in the hadoop stack) have a HADOOP_CONF_DIR (<project>_CONF_DIR) environment variable that is used to start daemons using the configuration specified.

      This is convenient because it decouples the configuration from the distributions – daemons can be run with different configs without having to perform stateful file operations.

      For example one could run accumulo like this:

      ACCUMULO_CONF_DIR=/etc/accumulo/conf accumulo ....
      

      or set env variables in .bashrc/jenkins scripts for different configurations

      export ACCUMULO_CONF_DIR=$ACCUMULO_HOME/conf/examples/1GB/standalone
      .. run test
      
      export ACCUMULO_CONF_DIR=$ACCUMULO_HOME/conf/examples/1GB/native-standalone
      .. run test
      
      1. accumulo-1550-trunk.patch
        27 kB
        Jonathan Hsieh
      2. accumulo-1550-1.4.patch
        38 kB
        Jonathan Hsieh

        Issue Links

          Activity

          Hide
          Jonathan Hsieh added a comment -

          All versions pass unit tests.

          The trunk version applies cleanly to 1.5 and to trunk.

          I've tested the 1.5 version by deploying sucessfully a using start-all.sh to start a 1.5 cluster using the ACCUMULO_CONF_DIR env variable where the accumulo/conf dir was renamed and not present. It started.

          I've been testing more rigorously with the 1.4 patch and have been running continuous ingets and randomwalks with a similar use the ACCUMULO_CONF_DIR env variable and rename the accumulo/conf.

          There are a few places we actually want to use ACCUMULO_HOME/conf like when we copy example dirs in testChangingDirectory, and in config.sh where we default to ACCUMULO_HOME/conf id no ACCUMULO_CONF_DIR is specified.

          Show
          Jonathan Hsieh added a comment - All versions pass unit tests. The trunk version applies cleanly to 1.5 and to trunk. I've tested the 1.5 version by deploying sucessfully a using start-all.sh to start a 1.5 cluster using the ACCUMULO_CONF_DIR env variable where the accumulo/conf dir was renamed and not present. It started. I've been testing more rigorously with the 1.4 patch and have been running continuous ingets and randomwalks with a similar use the ACCUMULO_CONF_DIR env variable and rename the accumulo/conf. There are a few places we actually want to use ACCUMULO_HOME/conf like when we copy example dirs in testChangingDirectory, and in config.sh where we default to ACCUMULO_HOME/conf id no ACCUMULO_CONF_DIR is specified.
          Hide
          Jonathan Hsieh added a comment -

          Summary of work:

          ACCUMULO-1550 Create ACCUMULO_CONF_DIR environment variable

          • ACCUMULO_CONF_DIR add for TestUtils.py
          • add comments at script header
          • Convert script/python ACCUMULO_HOME/conf into ACCUMULO_CONF_DIRs
          • Convert in java ACCUMULO_HOME + /conf in java code.
          • NOTE: not changed in python functional tests.
          Show
          Jonathan Hsieh added a comment - Summary of work: ACCUMULO-1550 Create ACCUMULO_CONF_DIR environment variable ACCUMULO_CONF_DIR add for TestUtils.py add comments at script header Convert script/python ACCUMULO_HOME/conf into ACCUMULO_CONF_DIRs Convert in java ACCUMULO_HOME + /conf in java code. NOTE: not changed in python functional tests.
          Hide
          ASF subversion and git services added a comment -

          Commit cd8ebfe895726695378644fad55e886022d67fc6 in branch refs/heads/1.4.4-SNAPSHOT from Eric Newton
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=cd8ebfe ]

          ACCUMULO-1550 committing Jonathan Hsieh's patch

          Show
          ASF subversion and git services added a comment - Commit cd8ebfe895726695378644fad55e886022d67fc6 in branch refs/heads/1.4.4-SNAPSHOT from Eric Newton [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=cd8ebfe ] ACCUMULO-1550 committing Jonathan Hsieh's patch
          Hide
          ASF subversion and git services added a comment -

          Commit c82c4316ad90ee7370c5919efb8ded65bb50b302 in branch refs/heads/1.5.1-SNAPSHOT from Eric Newton
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=c82c431 ]

          ACCUMULO-1550 committing Jonathan Hsieh's patch

          Show
          ASF subversion and git services added a comment - Commit c82c4316ad90ee7370c5919efb8ded65bb50b302 in branch refs/heads/1.5.1-SNAPSHOT from Eric Newton [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=c82c431 ] ACCUMULO-1550 committing Jonathan Hsieh's patch
          Hide
          ASF subversion and git services added a comment -

          Commit c82c4316ad90ee7370c5919efb8ded65bb50b302 in branch refs/heads/master from Eric Newton
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=c82c431 ]

          ACCUMULO-1550 committing Jonathan Hsieh's patch

          Show
          ASF subversion and git services added a comment - Commit c82c4316ad90ee7370c5919efb8ded65bb50b302 in branch refs/heads/master from Eric Newton [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=c82c431 ] ACCUMULO-1550 committing Jonathan Hsieh's patch
          Hide
          Eric Newton added a comment -

          Patched. Thanks!

          Show
          Eric Newton added a comment - Patched. Thanks!
          Hide
          Eric Newton added a comment -

          In 1.4.4rc4, test/system/continuous/start-stats.sh, there's a reference to ACCUMULO_CONF_DIR, and it is not defined anywhere.

          Show
          Eric Newton added a comment - In 1.4.4rc4, test/system/continuous/start-stats.sh, there's a reference to ACCUMULO_CONF_DIR, and it is not defined anywhere.
          Hide
          Sean Busbey added a comment -

          Eric Newton, updated patches against branches 1.4.4-SNAPSHOT, 1.5.1-SNAPSHOT, and master?

          Show
          Sean Busbey added a comment - Eric Newton , updated patches against branches 1.4.4-SNAPSHOT, 1.5.1-SNAPSHOT, and master?
          Hide
          Sean Busbey added a comment -

          To clarify: would you prefer this as a patch that just corrects the defect, or an update to the existing patch (which would require reverting the original application)?

          Show
          Sean Busbey added a comment - To clarify: would you prefer this as a patch that just corrects the defect, or an update to the existing patch (which would require reverting the original application)?
          Hide
          Mike Drob added a comment -

          I think new patches to correct the defects are appropriate, and created ACCUMULO-1659 for them.

          Show
          Mike Drob added a comment - I think new patches to correct the defects are appropriate, and created ACCUMULO-1659 for them.
          Hide
          Sean Busbey added a comment -

          Actually, it looks like this is a problem for all of the integration tests; either they ignore ACCUMULO_CONF_DIR or they presume it's externally defined without giving an example that sets it properly. opened ACCUMULO-1658.

          (Note that ACCUMULO-1584 is also a relevant missed bit from this patch)

          Show
          Sean Busbey added a comment - Actually, it looks like this is a problem for all of the integration tests; either they ignore ACCUMULO_CONF_DIR or they presume it's externally defined without giving an example that sets it properly. opened ACCUMULO-1658 . (Note that ACCUMULO-1584 is also a relevant missed bit from this patch)

            People

            • Assignee:
              Jonathan Hsieh
              Reporter:
              Jonathan Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development