Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-25302

Migrate Ambari Metrics Data On Ambari Major Upgrade to ambari-2.7.x is Migrating on 1 months data

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.7.3
    • Fix Version/s: None
    • Component/s: ambari-metrics
    • Labels:

      Description

      I am upgrading from ambari-2.6.x to ambari-3.1.x (HDF3.1 to HDF - 3.3.1)

      after upgrade of HDF and ambari, I follow the command to Migrate the Ambari metrics data:

      /usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/
                  upgrade_start /etc/ambari-metrics-collector/conf/metrics_whitelist
      

      Now after migration i can see only last 30 days data , all the Metrics data prior to 30 days is lost.

      Analysis :
      Upon checking the code : https://github.com/apache/ambari/blob/release-2.7.3/ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/timeline/upgrade/core/MetricsDataMigrationLauncher.java#L285

      I found the Class MetricsDataMigrationLauncher.java Actually needs `startTime` as an argument which is the StartTime from which the migration need to be done, but we are not giving that anywhere in the code : https://github.com/apache/ambari-metrics/blob/dfaf6a4fcb78a6c0516a769b2fbe9cf449400ee2/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector#L333

      also we are having the Default value of start time as 1 month :
      https://github.com/apache/ambari/blob/release-2.7.3/ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/timeline/upgrade/core/MetricsDataMigrationLauncher.java#L72

        public static final long DEFAULT_START_TIME = System.currentTimeMillis() - ONE_MONTH_MILLIS; //Last month
      

      so inshort we are only migrating last one months data , but no where in docs we have mentioned that and it caused Data loss to Customer.

      reference : https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.3.1/ambari-managed-hdf-upgrade/content/hdf-migrate-ambari-metrics-data.html

        Attachments

          Activity

            People

            • Assignee:
              asnaik Akhil Naik
              Reporter:
              asnaik Akhil Naik
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: