Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1809

Remove gridgain-hadoop component once ignite-hadoop gets added

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.0.0, 1.1.0
    • Component/s: build
    • Labels:
      None

      Description

      GridGain Hadoop superseded by Ignite Hadoop.
      Once ignite-hadoop added to BigTop, gridgain should be removed from there.

      See https://issues.apache.org/jira/browse/BIGTOP-1806

      1. BIGTOP-1809.patch
        58 kB
        Konstantin Boudnik
      2. BIGTOP-1809--a.patch
        71 kB
        Ivan Veselovsky
      3. BIGTOP-1809--b.patch
        73 kB
        Ivan Veselovsky
      4. BIGTOP-1809--d.patch
        58 kB
        Ivan Veselovsky

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          Let's fix this ASAP

          Show
          cos Konstantin Boudnik added a comment - Let's fix this ASAP
          Hide
          iveselovskiy Ivan Veselovsky added a comment -

          sure, will provide the patch tomorrow.

          Show
          iveselovskiy Ivan Veselovsky added a comment - sure, will provide the patch tomorrow.
          Hide
          iveselovskiy Ivan Veselovsky added a comment - - edited

          Patch "a" simply removes all files related to gridgain integration.
          The following items are still pending and will be addressed separately:
          1) ignite-hadoop puppet integration (BIGTOP-1814);
          2) ignite-hadoop smoke tests (BIGTOP-1821);

          Show
          iveselovskiy Ivan Veselovsky added a comment - - edited Patch "a" simply removes all files related to gridgain integration. The following items are still pending and will be addressed separately: 1) ignite-hadoop puppet integration ( BIGTOP-1814 ); 2) ignite-hadoop smoke tests ( BIGTOP-1821 );
          Hide
          cos Konstantin Boudnik added a comment -

          Ivan Veselovsky instead of removing puppet and test code, I'd recommend to use git mv which will preserve the early history. That's what I have done in BIGTOP-1814. And in fact if we go with current patch then BIGTOP-1814 won't apply (and vice versa). Shall the scope of this patch be just limited to the removal of the code which has been already duplicated in BIGTOP-1806? And BIGTOP-1821 will deal with the smoke test and, perhaps, the package tests? Thoughts?

          Show
          cos Konstantin Boudnik added a comment - Ivan Veselovsky instead of removing puppet and test code, I'd recommend to use git mv which will preserve the early history. That's what I have done in BIGTOP-1814 . And in fact if we go with current patch then BIGTOP-1814 won't apply (and vice versa). Shall the scope of this patch be just limited to the removal of the code which has been already duplicated in BIGTOP-1806 ? And BIGTOP-1821 will deal with the smoke test and, perhaps, the package tests? Thoughts?
          Hide
          iveselovskiy Ivan Veselovsky added a comment -

          It's probably good idea to do the tasks by parts, not glueing 1809 1814 and 1821 into one patch.
          So, as you suggest, let's limit this one to removal of the code that was already ported in 1806.

          Show
          iveselovskiy Ivan Veselovsky added a comment - It's probably good idea to do the tasks by parts, not glueing 1809 1814 and 1821 into one patch. So, as you suggest, let's limit this one to removal of the code that was already ported in 1806.
          Hide
          cos Konstantin Boudnik added a comment -

          Any advances on this? Shall we at least remove the gridgain-hadoop component from BOM to avoid building an extra component?

          Show
          cos Konstantin Boudnik added a comment - Any advances on this? Shall we at least remove the gridgain-hadoop component from BOM to avoid building an extra component?
          Hide
          cos Konstantin Boudnik added a comment -

          Ivan Veselovsky - any progress here? Thanks!

          Show
          cos Konstantin Boudnik added a comment - Ivan Veselovsky - any progress here? Thanks!
          Hide
          iveselovskiy Ivan Veselovsky added a comment -

          Konstantin, sorry, not yet. I'm too busy with urgent Ignite tasks. Yes, we may make a small step forward just removing GG from bom.
          I'll provide patch for that.

          Show
          iveselovskiy Ivan Veselovsky added a comment - Konstantin, sorry, not yet. I'm too busy with urgent Ignite tasks. Yes, we may make a small step forward just removing GG from bom. I'll provide patch for that.
          Hide
          cos Konstantin Boudnik added a comment -

          Ok, I have fixed the BOM part in BIGTOP-1842

          Show
          cos Konstantin Boudnik added a comment - Ok, I have fixed the BOM part in BIGTOP-1842
          Hide
          cos Konstantin Boudnik added a comment -

          The patch is no longer valid - let's fix it.

          Show
          cos Konstantin Boudnik added a comment - The patch is no longer valid - let's fix it.
          Hide
          cos Konstantin Boudnik added a comment -

          Any progress here? We need to fix the tests at the very least. Are still working on this Ivan Veselovsky?

          Show
          cos Konstantin Boudnik added a comment - Any progress here? We need to fix the tests at the very least. Are still working on this Ivan Veselovsky ?
          Hide
          cos Konstantin Boudnik added a comment -

          I've looked into the TestGridGainHadoop.groovy and it seems that setting the classpath in the test is an overkill. We are preparing the environment, including the client side configurations, as a part of the cluster deployment; so the only thing the test needs to do is to make sure that HADOOP_CONF_DIR is pointing to Ignite-specific configs.

          Show
          cos Konstantin Boudnik added a comment - I've looked into the TestGridGainHadoop.groovy and it seems that setting the classpath in the test is an overkill. We are preparing the environment, including the client side configurations, as a part of the cluster deployment; so the only thing the test needs to do is to make sure that HADOOP_CONF_DIR is pointing to Ignite-specific configs.
          Hide
          iveselovskiy Ivan Veselovsky added a comment - - edited

          The patch "BIGTOP-1809--b.patch" removes GridGain and adds the smoke test for Ignite Hadoop.
          The .deb package tested by installing on Ubuntu 14.04 TLS and running Hadoop samples over "ignited" Hadoop.
          The smoke test is also checked by running it on the same environment.

          Notice that patch also advances the Ignite version to 1.3.0 (formerly it was 1.2.0).

          The patch should also address BIGTOP-1821.

          Show
          iveselovskiy Ivan Veselovsky added a comment - - edited The patch " BIGTOP-1809 --b.patch" removes GridGain and adds the smoke test for Ignite Hadoop. The .deb package tested by installing on Ubuntu 14.04 TLS and running Hadoop samples over "ignited" Hadoop. The smoke test is also checked by running it on the same environment. Notice that patch also advances the Ignite version to 1.3.0 (formerly it was 1.2.0). The patch should also address BIGTOP-1821 .
          Hide
          cos Konstantin Boudnik added a comment - - edited

          I'd prefer not to mix the component version bump and the removal of the unused one in one JIRA. Could you please open a separate ticket for 1.3 upgrade? Thank you very much!

          Also, if the new smoke test is added here - shall we close BIGTOP-1821 now?

          Show
          cos Konstantin Boudnik added a comment - - edited I'd prefer not to mix the component version bump and the removal of the unused one in one JIRA. Could you please open a separate ticket for 1.3 upgrade? Thank you very much! Also, if the new smoke test is added here - shall we close BIGTOP-1821 now?
          Hide
          iveselovskiy Ivan Veselovsky added a comment -

          yes.

          Show
          iveselovskiy Ivan Veselovsky added a comment - yes.
          Hide
          iveselovskiy Ivan Veselovsky added a comment -

          Created BIGTOP-1955 about the version upgrade.
          All the 3 issues are now addressed in one patch.
          Shall we separate the fixes into 3 separate patches?

          Show
          iveselovskiy Ivan Veselovsky added a comment - Created BIGTOP-1955 about the version upgrade. All the 3 issues are now addressed in one patch. Shall we separate the fixes into 3 separate patches?
          Hide
          cos Konstantin Boudnik added a comment -

          We have a strong expectation to keep unrelated commits in the separate JIRAs. Hence, if it isn't too much work - could you please do the split of the gridgain removal and the test? I can pick up the version bump unless you're ok doing it yourself. Thanks!

          Show
          cos Konstantin Boudnik added a comment - We have a strong expectation to keep unrelated commits in the separate JIRAs. Hence, if it isn't too much work - could you please do the split of the gridgain removal and the test? I can pick up the version bump unless you're ok doing it yourself. Thanks!
          Hide
          iveselovskiy Ivan Veselovsky added a comment -

          Please use patch "1809--d" – it only removes GridGain component.
          Other changes (BIGTOP-1821, BIGTOP-1955) are extracted into separate patches.

          Show
          iveselovskiy Ivan Veselovsky added a comment - Please use patch "1809--d" – it only removes GridGain component. Other changes ( BIGTOP-1821 , BIGTOP-1955 ) are extracted into separate patches.
          Hide
          iveselovskiy Ivan Veselovsky added a comment -

          done. Patches 1809, 1821, and 1955 are splitted and attached separately.

          Show
          iveselovskiy Ivan Veselovsky added a comment - done. Patches 1809, 1821, and 1955 are splitted and attached separately.
          Hide
          cos Konstantin Boudnik added a comment -

          +1 on the "-d" patch.
          I have changed it a tiny bit because there was a trailing white space introduced in one of the file. Will commit the updated version shortly. Thank you very much Ivan. I will look into the other two in a few minutes.

          Show
          cos Konstantin Boudnik added a comment - +1 on the "-d" patch. I have changed it a tiny bit because there was a trailing white space introduced in one of the file. Will commit the updated version shortly. Thank you very much Ivan. I will look into the other two in a few minutes.
          Hide
          cos Konstantin Boudnik added a comment -

          Committed and pushed

          Show
          cos Konstantin Boudnik added a comment - Committed and pushed

            People

            • Assignee:
              iveselovskiy Ivan Veselovsky
              Reporter:
              iveselovskiy Ivan Veselovsky
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development