Uploaded image for project: 'Accumulo'
  1. Accumulo
  2. ACCUMULO-1030

Create a Maven plugin to run MiniAccumuloCluster for integration testing

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: None
    • Labels:
      None

      Description

      It would be great for users to be able to start up MiniAccumuloCluster with a maven plugin, so that they can test their code against Accumulo during the integration test phase of the Maven lifecycle.

      MiniAccumuloCluster can be started during the pre-integration-test phase of the build lifecycle, then shut down during the post-integration-test phase.

      This would satisfy use cases similar to how one might start up the jetty-maven-plugin to test a webapp.

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 2bedf7bfecfd2d7945e1573d3f09451e85dc1e17 in branch refs/heads/master from Christopher Tubbs
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=2bedf7b ]

          Revert "ACCUMULO-1030 - accumulo-maven-plugin skipped with skipTests"

          This reverts commit 9d6da9698d7a8ae35fa72ce0b48caa0e8124c497.

          This commit was done to fix ACCUMULO-1675, but does not correctly do
          that. ACCUMULO-1675 was supposed to skip execution of the plugin on a
          project that uses it. It was NOT supposed to offer a way to avoid
          building the plugin itself. Even if we skip tests in the Accumulo build,
          we still need to build/package/install/deploy this plugin.

          To correctly fix ACCUMULO-1675, we need to modify the plugin itself to
          behave differently upon encountering -DskipTests (or another similar
          flag) on a project that uses this plugin for integration testing against
          Accumulo.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 2bedf7bfecfd2d7945e1573d3f09451e85dc1e17 in branch refs/heads/master from Christopher Tubbs [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=2bedf7b ] Revert " ACCUMULO-1030 - accumulo-maven-plugin skipped with skipTests" This reverts commit 9d6da9698d7a8ae35fa72ce0b48caa0e8124c497. This commit was done to fix ACCUMULO-1675 , but does not correctly do that. ACCUMULO-1675 was supposed to skip execution of the plugin on a project that uses it. It was NOT supposed to offer a way to avoid building the plugin itself. Even if we skip tests in the Accumulo build, we still need to build/package/install/deploy this plugin. To correctly fix ACCUMULO-1675 , we need to modify the plugin itself to behave differently upon encountering -DskipTests (or another similar flag) on a project that uses this plugin for integration testing against Accumulo.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 9d6da9698d7a8ae35fa72ce0b48caa0e8124c497 in branch refs/heads/master from John Vines
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=9d6da96 ]

          ACCUMULO-1030 - accumulo-maven-plugin skipped with skipTests

          Show
          jira-bot ASF subversion and git services added a comment - Commit 9d6da9698d7a8ae35fa72ce0b48caa0e8124c497 in branch refs/heads/master from John Vines [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=9d6da96 ] ACCUMULO-1030 - accumulo-maven-plugin skipped with skipTests
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 57afd7e2834173619840e18817983d4d1d500f5e in branch refs/heads/master from Christopher Tubbs
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=57afd7e ]

          ACCUMULO-1030 Fix classpath issues

          Fixed classpath issues introduced by ACCUMULO-1368 changes
          (741daecfde80e19ceaeb11f89124883f53effd59). MiniAccumuloConfig now has
          the ability to explicitly set classpath elements to be used when
          spawning processes.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 57afd7e2834173619840e18817983d4d1d500f5e in branch refs/heads/master from Christopher Tubbs [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=57afd7e ] ACCUMULO-1030 Fix classpath issues Fixed classpath issues introduced by ACCUMULO-1368 changes (741daecfde80e19ceaeb11f89124883f53effd59). MiniAccumuloConfig now has the ability to explicitly set classpath elements to be used when spawning processes.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 88c11e63c1c83b2d75e7a1da4b5fa90d97a38243 in branch refs/heads/master from Eric Newton
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=88c11e6 ]

          ACCUMULO-1030 clean up the directories left from any previous runs

          Show
          jira-bot ASF subversion and git services added a comment - Commit 88c11e63c1c83b2d75e7a1da4b5fa90d97a38243 in branch refs/heads/master from Eric Newton [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=88c11e6 ] ACCUMULO-1030 clean up the directories left from any previous runs
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit af06b162919fd10d9caca907c84e58c2a0a13247 in branch refs/heads/ACCUMULO-1000 from Christopher Tubbs
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=af06b16 ]

          ACCUMULO-1030 Shared mini for integration tests, fix plugin package

          Show
          jira-bot ASF subversion and git services added a comment - Commit af06b162919fd10d9caca907c84e58c2a0a13247 in branch refs/heads/ ACCUMULO-1000 from Christopher Tubbs [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=af06b16 ] ACCUMULO-1030 Shared mini for integration tests, fix plugin package
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit af06b162919fd10d9caca907c84e58c2a0a13247 in branch refs/heads/master from Christopher Tubbs
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=af06b16 ]

          ACCUMULO-1030 Shared mini for integration tests, fix plugin package

          Show
          jira-bot ASF subversion and git services added a comment - Commit af06b162919fd10d9caca907c84e58c2a0a13247 in branch refs/heads/master from Christopher Tubbs [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=af06b16 ] ACCUMULO-1030 Shared mini for integration tests, fix plugin package
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1502702 from Christopher Tubbs
          [ https://svn.apache.org/r1502702 ]

          ACCUMULO-1030 First attempt at a working accumulo-maven-plugin

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1502702 from Christopher Tubbs [ https://svn.apache.org/r1502702 ] ACCUMULO-1030 First attempt at a working accumulo-maven-plugin
          Hide
          ctubbsii Christopher Tubbs added a comment -

          created a configurable shutdown port on the MAC runner

          Excellent!

          Show
          ctubbsii Christopher Tubbs added a comment - created a configurable shutdown port on the MAC runner Excellent!
          Hide
          sonixbp Corey J. Nolet added a comment -

          I've created a configurable shutdown port on the MAC runner for ACCUMULO-1368. This should help when running the MAC in the background so that a plugin could do something like "mvn accumulo:start" and then "mvn accumulo:stop"

          Show
          sonixbp Corey J. Nolet added a comment - I've created a configurable shutdown port on the MAC runner for ACCUMULO-1368 . This should help when running the MAC in the background so that a plugin could do something like "mvn accumulo:start" and then "mvn accumulo:stop"
          Hide
          ctubbsii Christopher Tubbs added a comment -

          Josh Elser wrote:

          Do you have any ideas about how to isolate configuration changes, existing tables, etc from different integration tests?

          MiniAccumuloCluster could have options to generate table names or user names that are unique, so integration tests could leverage that (or one could use JUnit's ability to generate unique temporary file names).

          If you need to isolate the tests further, you could specify multiple executions of MiniAccumuloCluster in the pre-integration-test phase, with different configuration properties (from a properties file specified in the configuration of the plugin), and integration tests can be programmed to connect to one or another by reading its corresponding properties file.

          What I'm imagining is something like:

          <build>
            <plugins>
              <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <executions>
                  <execution>
                    <id>run-integration-tests</id>
                    <goals>
                      <goal>integration-test</goal>
                    </goals>
                    <phase>integration-test</phase>
                  </execution>
                  <execution>
                    <id>verify-integration-tests</id>
                    <goals>
                      <goal>verify</goal>
                    </goals>
                    <phase>verify</phase>
                  </execution>
                </executions>
              </plugin>
            </plugins>
          </build>
          <profiles>
            <profile>
              <id>integration-tests-with-accumulo</id>
              <activation>
                <property>
                  <name>!skipTests</name>
                </property>
              </activation>
              <build>
                <plugins>
                  <plugin>
                    <groupId>org.apache.accumulo</groupId>
                    <artifactId>accumulo-maven-plugin</artifactId>
                    <version>1.6.0</version>
                    <executions>
                      <execution>
                        <id>start-accumulo-1</id>
                        <goals>
                          <goal>start</goal>
                        </goals>
                        <phase>pre-integration-test</phase>
                        <configuration>
                          <file>src/test/resources/accumulo-it-1.properties</file>
                        </configuration>
                      </execution>
                      <execution>
                        <id>start-accumulo-2</id>
                        <goals>
                          <goal>start</goal>
                        </goals>
                        <phase>pre-integration-test</phase>
                        <configuration>
                          <file>src/test/resources/accumulo-it-2.properties</file>
                        </configuration>
                      </execution>
                      <execution>
                        <id>stop-accumulo-1</id>
                        <goals>
                          <goal>stop</goal>
                        </goals>
                        <phase>post-integration-test</phase>
                        <configuration>
                          <file>src/test/resources/accumulo-it-1.properties</file>
                        </configuration>
                      </execution>
                      <execution>
                        <id>stop-accumulo-2</id>
                        <goals>
                          <goal>stop</goal>
                        </goals>
                        <phase>post-integration-test</phase>
                        <configuration>
                          <file>src/test/resources/accumulo-it-2.properties</file>
                        </configuration>
                      </execution>
                    </executions>
                  </plugin>
                </plugins>
              </build>
            </profile>
          </profiles>
          

          These settings could be combined with other plugins to reserve network ports and write the properties files.

          Show
          ctubbsii Christopher Tubbs added a comment - Josh Elser wrote: Do you have any ideas about how to isolate configuration changes, existing tables, etc from different integration tests? MiniAccumuloCluster could have options to generate table names or user names that are unique, so integration tests could leverage that (or one could use JUnit's ability to generate unique temporary file names). If you need to isolate the tests further, you could specify multiple executions of MiniAccumuloCluster in the pre-integration-test phase, with different configuration properties (from a properties file specified in the configuration of the plugin), and integration tests can be programmed to connect to one or another by reading its corresponding properties file. What I'm imagining is something like: <build> <plugins> <plugin> <groupId> org.apache.maven.plugins </groupId> <artifactId> maven-failsafe-plugin </artifactId> <executions> <execution> <id> run-integration-tests </id> <goals> <goal> integration-test </goal> </goals> <phase> integration-test </phase> </execution> <execution> <id> verify-integration-tests </id> <goals> <goal> verify </goal> </goals> <phase> verify </phase> </execution> </executions> </plugin> </plugins> </build> <profiles> <profile> <id> integration-tests-with-accumulo </id> <activation> <property> <name> !skipTests </name> </property> </activation> <build> <plugins> <plugin> <groupId> org.apache.accumulo </groupId> <artifactId> accumulo-maven-plugin </artifactId> <version> 1.6.0 </version> <executions> <execution> <id> start-accumulo-1 </id> <goals> <goal> start </goal> </goals> <phase> pre-integration-test </phase> <configuration> <file> src/test/resources/accumulo-it-1.properties </file> </configuration> </execution> <execution> <id> start-accumulo-2 </id> <goals> <goal> start </goal> </goals> <phase> pre-integration-test </phase> <configuration> <file> src/test/resources/accumulo-it-2.properties </file> </configuration> </execution> <execution> <id> stop-accumulo-1 </id> <goals> <goal> stop </goal> </goals> <phase> post-integration-test </phase> <configuration> <file> src/test/resources/accumulo-it-1.properties </file> </configuration> </execution> <execution> <id> stop-accumulo-2 </id> <goals> <goal> stop </goal> </goals> <phase> post-integration-test </phase> <configuration> <file> src/test/resources/accumulo-it-2.properties </file> </configuration> </execution> </executions> </plugin> </plugins> </build> </profile> </profiles> These settings could be combined with other plugins to reserve network ports and write the properties files.
          Hide
          medined David Medinets added a comment -

          I was going to suggest a reset command for the MiniAccumuloCluster but then
          I thought able having tests run in parallel instead of sequentially. When I
          run tests, I frequently add a UUID to some base table name so that each
          test is isolated. In the @After method, I delete the test-specific table.

          Show
          medined David Medinets added a comment - I was going to suggest a reset command for the MiniAccumuloCluster but then I thought able having tests run in parallel instead of sequentially. When I run tests, I frequently add a UUID to some base table name so that each test is isolated. In the @After method, I delete the test-specific table.
          Hide
          elserj Josh Elser added a comment -

          I was actually just thinking about this today. Do you have any ideas about how to isolate configuration changes, existing tables, etc from different integration tests?

          It would be nice to spin up the single MAC instance in pre-integration-test, and not have to worry about what other integration tests are doing as I write a new test (I'm thinking of some sort of weird tests against !METADATA). Perhaps this is something that could be addressed down the road if it becomes an impediment. Regardless, I definitely like the general idea here.

          Show
          elserj Josh Elser added a comment - I was actually just thinking about this today. Do you have any ideas about how to isolate configuration changes, existing tables, etc from different integration tests? It would be nice to spin up the single MAC instance in pre-integration-test, and not have to worry about what other integration tests are doing as I write a new test (I'm thinking of some sort of weird tests against !METADATA). Perhaps this is something that could be addressed down the road if it becomes an impediment. Regardless, I definitely like the general idea here.

            People

            • Assignee:
              ctubbsii Christopher Tubbs
              Reporter:
              ctubbsii Christopher Tubbs
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development