Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-21481

Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

    Details

      Description

      STR:

      • Install BigInsights 4.2.5 with Spark2
      • Upgrade to Ambari 2.5.2
      • Start Ambari, which will fail in the DB consistency check
      2017-07-07 01:07:01,881 ERROR - You have non selected configs: spark2-javaopts-properties for service SPARK2 from cluster c1!
      

      Looks like this config is never selected,

      ambari=> SELECT sc.service_name, sc.version, sc.user_name, TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
      c.version AS type_version, c.version_tag, c.type_name, c.stack_id, TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
      TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, ccm.selected, ccm.user_name
      FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = m.service_config_id
      JOIN clusterconfig c ON m.config_id = c.config_id
      JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND c.version_tag = ccm.version_tag
      WHERE sc.service_name = 'SPARK2' and c.type_name = 'spark2-javaopts-properties'
      ORDER BY sc.version, c.type_name;
       service_name | version |   user_name    |      to_timestamp      | stack_id |               note                | type_version | version_tag |         type_name          | stack_id | clusterconfig_created  | clusterconfigmapping_created | selected | user_name
      --------------+---------+----------------+------------------------+----------+-----------------------------------+--------------+-------------+----------------------------+----------+------------------------+------------------------------+----------+-----------
       SPARK2       |       1 | admin          | 2017-07-11 20:17:01+00 |        3 | Initial configurations for Spark2 |            1 | version1    | spark2-javaopts-properties |        3 | 2017-07-11 20:17:00+00 | 2017-07-11 20:17:01+00       |        0 | admin
       SPARK2       |       2 | ambari-upgrade | 2017-07-13 20:33:50+00 |        3 |                                   |            1 | version1    | spark2-javaopts-properties |        3 | 2017-07-11 20:17:00+00 | 2017-07-11 20:17:01+00       |        0 | admin
      (2 rows)
      

      This is because the config type has a single config with an empty value, so perhaps UI never selects it.

      <configuration supports_final="true">
        <property>
          <name>content</name>
          <description>Spark2-javaopts-properties</description>
          <value> </value>
          <on-ambari-upgrade add="true"/>
        </property>
      </configuration>
      

      Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each config type if none are selected.
      The only config type to hit this issue is spark2-javaopts-properties

      1. AMBARI-21481.patch
        10 kB
        Alejandro Fernandez

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12877423/AMBARI-21481.patch
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/11796//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12877423/AMBARI-21481.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/11796//console This message is automatically generated.
          Hide
          afernandez Alejandro Fernandez added a comment -

          Pushed to branch-2.5, commit 0ed09cd5342cfc4cac0d6061a7b7b9a3cef127c1

          Show
          afernandez Alejandro Fernandez added a comment - Pushed to branch-2.5, commit 0ed09cd5342cfc4cac0d6061a7b7b9a3cef127c1
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Ambari-branch-2.5 #1705 (See https://builds.apache.org/job/Ambari-branch-2.5/1705/)
          AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on (afernandez: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=0ed09cd5342cfc4cac0d6061a7b7b9a3cef127c1)

          • (edit) ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
          • (edit) ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
          • (edit) ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
          • (edit) ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
          • (edit) ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Ambari-branch-2.5 #1705 (See https://builds.apache.org/job/Ambari-branch-2.5/1705/ ) AMBARI-21481 . Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on (afernandez: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=0ed09cd5342cfc4cac0d6061a7b7b9a3cef127c1 ) (edit) ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml (edit) ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml (edit) ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml (edit) ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java (edit) ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml

            People

            • Assignee:
              afernandez Alejandro Fernandez
              Reporter:
              afernandez Alejandro Fernandez
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development