Uploaded image for project: 'NiFi Registry'
  1. NiFi Registry
  2. NIFIREG-240

Add support for MariaDB database as configuration backend

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.4.0
    • Labels:
      None

      Description

      I have tried to use MariaDB 10.3 with NiFi Registry, with no luck. Although there should be a way to enable MariaDB as a backend service instead of H2.

       

      Environment:

      NiFi Registry 0.3.0 (inside Docker, but this does not really affect the output)

      MariaDB Java Client 2.3.0 - mariadb-java-client-2.3.0.jar (not 2.4.1 due to https://github.com/flyway/flyway/issues/2289#issuecomment-459481080)

      Configuration:

      nifi-registry.properties:

      nifi.registry.db.url=jdbc:mariadb://database:3306/db
      nifi.registry.db.driver.class=org.mariadb.jdbc.Driver
      nifi.registry.db.driver.directory=/opt/nifi-registry/libs/
      nifi.registry.db.username=root
      nifi.registry.db.password=myPassword

      The mariadb-java-client-2.3.0.jar was placed to

      /opt/nifi-registry/libs/

       

      Steps to reproduce:

      1. unpack nifi registry to /opt/nifi-registry/nifi-registry-0.3.0
      2. set the nifi-registry.properties as above
      3. add mariadb-java-client-2.3.0.jar to /opt/nifi-registry/libs/
      4. start

      Or alternatively, use the Docker image I was developping and during which I found this issue.

      The docker-compose file is https://github.com/michalklempa/docker-nifi-registry/blob/master/docker-compose.mariadb.yml

       

      Expected behavior: working Registry

      Actual behavior:

      nifi-registry_1  | 2019-03-24 17:55:39,771 WARN [main] o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'accessPolicyResource' defined in file [/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/api/AccessPolicyResource.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'authorizationService' defined in URL [jar:file:/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/lib/nifi-registry-framework-0.3.0.jar!/org/apache/nifi/registry/service/AuthorizationService.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'registryService' defined in URL [jar:file:/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/lib/nifi-registry-framework-0.3.0.jar!/org/apache/nifi/registry/service/RegistryService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'databaseMetadataService' defined in URL [jar:file:/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/lib/nifi-registry-framework-0.3.0.jar!/org/apache/nifi/registry/db/DatabaseMetadataService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateSqlException:
      nifi-registry_1  | Migration V2__Initial.sql failed
      nifi-registry_1  | --------------------------------
      nifi-registry_1  | SQL State  : 42000
      nifi-registry_1  | Error Code : 1071
      nifi-registry_1  | Message    : (conn=8) Specified key was too long; max key length is 3072 bytes
      nifi-registry_1  | Location   : db/migration/V2__Initial.sql (/opt/nifi-registry/nifi-registry-0.3.0/file:/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/lib/nifi-registry-framework-0.3.0.jar!/db/migration/V2__Initial.sql)
      nifi-registry_1  | Line       : 54
      nifi-registry_1  | Statement  : CREATE TABLE SIGNING_KEY (
      nifi-registry_1  |     ID VARCHAR(50) NOT NULL,
      nifi-registry_1  |     TENANT_IDENTITY VARCHAR(4096) NOT NULL,
      nifi-registry_1  |     KEY_VALUE VARCHAR(50) NOT NULL,
      nifi-registry_1  |     CONSTRAINT PK__SIGNING_KEY_ID PRIMARY KEY (ID),
      nifi-registry_1  |     CONSTRAINT UNIQUE__SIGNING_KEY_TENANT_IDENTITY UNIQUE (TENANT_IDENTITY)
      nifi-registry_1  | )
      nifi-registry_1  |
      nifi-registry_1  | 2019-03-24 17:55:39,772 INFO [main] o.a.nifi.registry.event.EventService Shutting down event consumer...

       

      There are various SQL dialect differences.

      I was able to shorten TENANT_IDENTITY varchar to 1096:

      SIGNING_KEY.TENANT_IDENTITY VARCHAR(1096)

      Then there was mariadb complaining about lenght of constraint name UNIQUE__BUNDLE_VERSION_DEPENDENCY_BUNDLE_ID_GROUP_ARTIFACT_VERSION

      and finally the DROP CONSTRAINT for FOREIGN KEY syntax is different.

      Reverse diff of my changes can be seen here:

      https://github.com/michalklempa/docker-nifi-registry/commit/2a78f5f6df0b00fa3011da9ee49c65a412012fcc

       

      I am unsure is such changes are compatible or not with the Registry codebase. Further work is required. And since I was creating the schema manually, NiFi Registry did not start up either - since it found existing schema without some required metadata.

        Attachments

          Activity

            People

            • Assignee:
              bbende Bryan Bende
              Reporter:
              michal.klempa Michal Klempa
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: