Uploaded image for project: 'Metron'
  1. Metron
  2. METRON-1130

Updating README.md - Amazon EC2 deployment instructions

    Details

    • Type: Improvement
    • Status: To Do
    • Priority: Trivial
    • Resolution: Unresolved
    • Affects Version/s: 0.4.0
    • Fix Version/s: 0.4.0
    • Environment:

      Description

      PR is updating doc based on experience with installing 0.4.0 using a linux env (ClearOS release 7.4.0 - ClearOS is based on CentOS).

      https://github.com/apache/metron/pull/713

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user as22323 opened a pull request:

          https://github.com/apache/metron/pull/713

          METRON-1130 - Updating Amazon EC2 README.md

            1. Contributor Comments
              Hello, This is my first time submitting a pull request to the Apache Metron project. Feel free to point me in the right direction if I am not doing this right.

          The suggested updates were based on my experience of using ClearOS 7.4.0 (Distro is based on CentOS) as a host to install Apache Metron 0.4.0 in Amazon EC2.

          FYI:
          Here is a list of versions that seemed break/cause issues when installing:
          -Ansible 2.0.0.2
          -Maven 3.0.5

            1. Pull Request Checklist

          Thank you for submitting a contribution to Apache Metron.
          Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.
          Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.

          In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:

              1. For all changes:
              1. For code changes:
          • [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
          • [ ] Have you included steps or a guide to how the change may be verified and tested manually?
          • [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
            ```
            mvn -q clean integration-test install && build_utils/verify_licenses.sh
            ```
          • [ ] Have you written or updated unit tests and or integration tests to verify your changes?
          • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
          • [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
              1. For documentation related changes:
          • [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:

          ```
          cd site-book
          mvn site
          ```

                1. Note:
                  Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
                  It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/as22323/metron master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/metron/pull/713.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #713


          commit 095be23dce0f818a98a1fc587053d133ca542856
          Author: JonZeolla <zeolla@gmail.com>
          Date: 2017-06-28T18:04:39Z

          METRON-999 Add virtualization support checks to platform-info.sh (JonZeolla via jonzeolla) closes apache/metron#618

          commit df94ed40523db375b0bac4601a7efffa5a1fede5
          Author: justinleet <justinjleet@gmail.com>
          Date: 2017-06-30T13:21:17Z

          METRON-1004 Travis CI - Job Exceeded Maximum Time Limit (justinleet) closes apache/metron#624

          commit a5b13777a92418155da61864e98fc24cde866501
          Author: mattf-horton <mfoley@hortonworks.com>
          Date: 2017-07-02T22:38:14Z

          METRON-877 Extract core implementation and UDF support, create metron-stellar module (mattf-horton) closes apache/metron#616

          commit ab87bdf673e151da4b6352da6b0f4a6957f4777b
          Author: ottobackwards <ottobackwards@gmail.com>
          Date: 2017-07-04T11:47:43Z

          METRON-1007 Ship the metron-management jar as part of the mpack install (ottobackwards) closes apache/metron#627

          commit cc7bbc94870f0e0d5026c14ac710ca8882d0589b
          Author: JonZeolla <zeolla@gmail.com>
          Date: 2017-07-05T01:52:00Z

          METRON-508 Expand Elasticsearch templates to support the standard bro logs (JonZeolla) closes apache/metron#586

          commit 87f652215cbe9e611029f5269f6c2b30aaa8f229
          Author: merrimanr <merrimanr@gmail.com>
          Date: 2017-07-05T16:04:34Z

          METRON-990 Clean up and organize flux properties (merrimanr) closes apache/metron#613

          commit df270feeab5ca435ad32438351dfe56a310700c5
          Author: mattf-horton <mfoley@hortonworks.com>
          Date: 2017-07-05T16:16:53Z

          METRON-1014 StellarShell class name typo (mattf-horton) closes apache/metron#630

          commit f7a94f2e631227f035c63ad258c124c0cd582189
          Author: mattf-horton <mfoley@hortonworks.com>
          Date: 2017-07-05T17:47:59Z

          METRON-1012 Update Metron public web site for 0.4.0 release (mattf-horton) closes apache/metron#629

          commit 50e521b0077d098fdc89562d7aa9b95398ce7bb6
          Author: nickwallen <nick@nickallen.org>
          Date: 2017-07-07T16:27:27Z

          METRON-986 Enhance Fastcapa to Support Intel X520 (nickwallen) closes apache/metron#608

          commit a8f35363ffdd1e20bc7a64b7107d270f9667be3b
          Author: mattf-horton <mfoley@hortonworks.com>
          Date: 2017-07-07T19:20:33Z

          METRON-1021 increment metron version number to 0.4.1 in poms etc (mattf-horton) closes apache/metron#637

          commit fc6614b18c73d339293bb76e682569523feb041a
          Author: ottobackwards <ottobackwards@gmail.com>
          Date: 2017-07-09T13:48:45Z

          METRON-1025 Metron github pr template should not reference incubator (ottobackwards) closes apache/metron#640

          commit 7cd39316a4e7587c84efbfa2d8da503d1a6640a7
          Author: nickwallen <nick@nickallen.org>
          Date: 2017-07-11T14:10:24Z

          METRON-1027 Errant Log Message When No GeoIP Hit on Valid, External IPv4 Address (nickwallen) closes apache/metron#644

          commit dcec5a7cfb7f8b6544af8ab137dab3f75de9c49a
          Author: cstella <cestella@gmail.com>
          Date: 2017-07-11T17:25:37Z

          METRON-1001: Allow metron to ingest parser metadata along with data closes apache/incubator-metron#621

          commit cf165ff306764641a2c7991ff93cf48228d7910d
          Author: ottobackwards <ottobackwards@gmail.com>
          Date: 2017-07-11T19:11:12Z

          METRON-933 STELLAR support for setting fields to regex match captures (ottobackwards) closes apache/metron#638

          commit a9b01d1e373091662f4dff3ac3acf7cec181e103
          Author: merrimanr <merrimanr@gmail.com>
          Date: 2017-07-12T13:20:03Z

          METRON-1017 Ambari components should be separate (merrimanr) closes apache/metron#634

          commit bfc4786df55d522597d87c01872982bc9ec7923f
          Author: ottobackwards <ottobackwards@gmail.com>
          Date: 2017-07-12T14:04:20Z

          METRON-1013 StellarShell does not validate parameters (ottobackwards) closes apache/metron#639

          commit dcf768297cdc62be1400cf5dbc5db28820900a43
          Author: lvets <laurens.vets@sap.com>
          Date: 2017-07-12T14:31:31Z

          METRON-1019 Metron 0.4.0 manual installation guide for CentOS 6 (lvets via ottobackwards) closes apache/metron#631

          commit 886ed7a00501c7ccb98ed8296cab9ded3e0a62fd
          Author: simonellistonball <simon@simonellistonball.com>
          Date: 2017-07-13T13:23:07Z

          METRON-1033 Profiler example uses incorrect units for expires (simonellistonball via nickwallen) closes apache/metron#648

          commit 29aad8b4abc9993915616bfac3f18c739c2c7103
          Author: cestella <cestella@gmail.com>
          Date: 2017-07-13T13:35:25Z

          METRON-833 Update MaaS documentation to explain how it interacts with kerberos (cestella via justinleet) closes apache/metron#520

          commit eaa52f6231faf9445fdb49031d308609aa4db14c
          Author: nickwallen <nick@nickallen.org>
          Date: 2017-07-13T20:14:04Z

          METRON-1040 Create Installation Instructions for the Profiler (nickwallen) closes apache/metron#653

          commit b9125c8307b5e8f5b77eeb6f4b96004cf52dc4f9
          Author: bjigmp <bjigmp@users.noreply.github.com>
          Date: 2017-07-14T11:54:18Z

          METRON-1003 ParserUtil parses dates incorrect (bjigmp via justinleet) closes apache/metron#623

          commit afe6fd56ee727092dbc66b065df4ff6c7b467bb0
          Author: zezutom <zezulatomas@gmail.com>
          Date: 2017-07-14T12:00:17Z

          METRON-633 Create better logging for HbaseEnrichmentWriter (zezutom via justinleet) closes apache/metron#572

          commit 954b5ea13fc04cb8c9ae277d57a75cfb16956772
          Author: jjmeyer <jjmeyer0@gmail.com>
          Date: 2017-07-14T12:45:59Z

          METRON-1008 Migrate Travis build to use Trusty (jjmeyer via justinleet) closes apache/metron#633

          commit 57d2764c318c8ec7d37d8a3f11e4609e35078285
          Author: nickwallen <nick@nickallen.org>
          Date: 2017-07-17T19:42:40Z

          METRON-1031 Management UI Cannot Start Topologies in Kerberized Environment (nickwallen) closes apache/metron#647

          commit a8bcbf0b4e30cb5803d5bb99a30dd988d8375b32
          Author: merrimanr <merrimanr@gmail.com>
          Date: 2017-07-18T12:53:47Z

          METRON-1029 Topology worker opts are not being set when Kerberos is enabled (merrimanr) closes apache/metron#645

          commit 0d1923f83aee6630f1112cd7f9d20514726272f0
          Author: nickwallen <nick@nickallen.org>
          Date: 2017-07-19T17:19:07Z

          METRON-1048 Intermittent Test Failure for SimpleHBaseEnrichmentWriterLoggingTest (nickwallen) closes apache/metron#658

          commit cf7043c59b1091c59b06cb14ca9d332a727fc3ea
          Author: merrimanr <merrimanr@gmail.com>
          Date: 2017-07-20T09:32:45Z

          METRON-1022: Elasticsearch REST endpoint this closes apache/incubator-metron#636

          commit 3d6b049b8799235b580a275024c18f6092fc8942
          Author: merrimanr <merrimanr@gmail.com>
          Date: 2017-07-20T20:42:55Z

          METRON-1044 Disabled writers are not acking messages (merrimanr) closes apache/metron#654

          commit a0e756bf9a0f8dc4401a0dbac81b23ba87c16616
          Author: mmiklavc <michael.miklavcic@gmail.com>
          Date: 2017-07-21T17:19:11Z

          METRON-992: Create performance tuning guide (mmiklavc) closes apache/metron#614

          commit 309d8097a91cc40e46f187f672d651b00cdd90fa
          Author: nickwallen <nick@nickallen.org>
          Date: 2017-07-24T13:47:06Z

          METRON-1054 Ambari Mpack Attempts to Kill Topologies That Are Not Running (nickwallen) closes apache/metron#660


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user as22323 opened a pull request: https://github.com/apache/metron/pull/713 METRON-1130 - Updating Amazon EC2 README.md Contributor Comments Hello, This is my first time submitting a pull request to the Apache Metron project. Feel free to point me in the right direction if I am not doing this right. The suggested updates were based on my experience of using ClearOS 7.4.0 (Distro is based on CentOS) as a host to install Apache Metron 0.4.0 in Amazon EC2. FYI: Here is a list of versions that seemed break/cause issues when installing: -Ansible 2.0.0.2 -Maven 3.0.5 Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines] ( https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235 ) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines] ( https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview ) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: For all changes: [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira] ( https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel ). [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? <AS22323: README.md from branch "Metron_0.4.0" METRON-979 and the latest greatest of today METRON-979 were the same> For code changes: [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? [ ] Have you included steps or a guide to how the change may be verified and tested manually? [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && build_utils/verify_licenses.sh ``` [ ] Have you written or updated unit tests and or integration tests to verify your changes? [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0] ( http://www.apache.org/legal/resolved.html#category-a)? [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? For documentation related changes: [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci] ( https://travis-ci.org ) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/as22323/metron master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/713.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #713 commit 095be23dce0f818a98a1fc587053d133ca542856 Author: JonZeolla <zeolla@gmail.com> Date: 2017-06-28T18:04:39Z METRON-999 Add virtualization support checks to platform-info.sh (JonZeolla via jonzeolla) closes apache/metron#618 commit df94ed40523db375b0bac4601a7efffa5a1fede5 Author: justinleet <justinjleet@gmail.com> Date: 2017-06-30T13:21:17Z METRON-1004 Travis CI - Job Exceeded Maximum Time Limit (justinleet) closes apache/metron#624 commit a5b13777a92418155da61864e98fc24cde866501 Author: mattf-horton <mfoley@hortonworks.com> Date: 2017-07-02T22:38:14Z METRON-877 Extract core implementation and UDF support, create metron-stellar module (mattf-horton) closes apache/metron#616 commit ab87bdf673e151da4b6352da6b0f4a6957f4777b Author: ottobackwards <ottobackwards@gmail.com> Date: 2017-07-04T11:47:43Z METRON-1007 Ship the metron-management jar as part of the mpack install (ottobackwards) closes apache/metron#627 commit cc7bbc94870f0e0d5026c14ac710ca8882d0589b Author: JonZeolla <zeolla@gmail.com> Date: 2017-07-05T01:52:00Z METRON-508 Expand Elasticsearch templates to support the standard bro logs (JonZeolla) closes apache/metron#586 commit 87f652215cbe9e611029f5269f6c2b30aaa8f229 Author: merrimanr <merrimanr@gmail.com> Date: 2017-07-05T16:04:34Z METRON-990 Clean up and organize flux properties (merrimanr) closes apache/metron#613 commit df270feeab5ca435ad32438351dfe56a310700c5 Author: mattf-horton <mfoley@hortonworks.com> Date: 2017-07-05T16:16:53Z METRON-1014 StellarShell class name typo (mattf-horton) closes apache/metron#630 commit f7a94f2e631227f035c63ad258c124c0cd582189 Author: mattf-horton <mfoley@hortonworks.com> Date: 2017-07-05T17:47:59Z METRON-1012 Update Metron public web site for 0.4.0 release (mattf-horton) closes apache/metron#629 commit 50e521b0077d098fdc89562d7aa9b95398ce7bb6 Author: nickwallen <nick@nickallen.org> Date: 2017-07-07T16:27:27Z METRON-986 Enhance Fastcapa to Support Intel X520 (nickwallen) closes apache/metron#608 commit a8f35363ffdd1e20bc7a64b7107d270f9667be3b Author: mattf-horton <mfoley@hortonworks.com> Date: 2017-07-07T19:20:33Z METRON-1021 increment metron version number to 0.4.1 in poms etc (mattf-horton) closes apache/metron#637 commit fc6614b18c73d339293bb76e682569523feb041a Author: ottobackwards <ottobackwards@gmail.com> Date: 2017-07-09T13:48:45Z METRON-1025 Metron github pr template should not reference incubator (ottobackwards) closes apache/metron#640 commit 7cd39316a4e7587c84efbfa2d8da503d1a6640a7 Author: nickwallen <nick@nickallen.org> Date: 2017-07-11T14:10:24Z METRON-1027 Errant Log Message When No GeoIP Hit on Valid, External IPv4 Address (nickwallen) closes apache/metron#644 commit dcec5a7cfb7f8b6544af8ab137dab3f75de9c49a Author: cstella <cestella@gmail.com> Date: 2017-07-11T17:25:37Z METRON-1001 : Allow metron to ingest parser metadata along with data closes apache/incubator-metron#621 commit cf165ff306764641a2c7991ff93cf48228d7910d Author: ottobackwards <ottobackwards@gmail.com> Date: 2017-07-11T19:11:12Z METRON-933 STELLAR support for setting fields to regex match captures (ottobackwards) closes apache/metron#638 commit a9b01d1e373091662f4dff3ac3acf7cec181e103 Author: merrimanr <merrimanr@gmail.com> Date: 2017-07-12T13:20:03Z METRON-1017 Ambari components should be separate (merrimanr) closes apache/metron#634 commit bfc4786df55d522597d87c01872982bc9ec7923f Author: ottobackwards <ottobackwards@gmail.com> Date: 2017-07-12T14:04:20Z METRON-1013 StellarShell does not validate parameters (ottobackwards) closes apache/metron#639 commit dcf768297cdc62be1400cf5dbc5db28820900a43 Author: lvets <laurens.vets@sap.com> Date: 2017-07-12T14:31:31Z METRON-1019 Metron 0.4.0 manual installation guide for CentOS 6 (lvets via ottobackwards) closes apache/metron#631 commit 886ed7a00501c7ccb98ed8296cab9ded3e0a62fd Author: simonellistonball <simon@simonellistonball.com> Date: 2017-07-13T13:23:07Z METRON-1033 Profiler example uses incorrect units for expires (simonellistonball via nickwallen) closes apache/metron#648 commit 29aad8b4abc9993915616bfac3f18c739c2c7103 Author: cestella <cestella@gmail.com> Date: 2017-07-13T13:35:25Z METRON-833 Update MaaS documentation to explain how it interacts with kerberos (cestella via justinleet) closes apache/metron#520 commit eaa52f6231faf9445fdb49031d308609aa4db14c Author: nickwallen <nick@nickallen.org> Date: 2017-07-13T20:14:04Z METRON-1040 Create Installation Instructions for the Profiler (nickwallen) closes apache/metron#653 commit b9125c8307b5e8f5b77eeb6f4b96004cf52dc4f9 Author: bjigmp <bjigmp@users.noreply.github.com> Date: 2017-07-14T11:54:18Z METRON-1003 ParserUtil parses dates incorrect (bjigmp via justinleet) closes apache/metron#623 commit afe6fd56ee727092dbc66b065df4ff6c7b467bb0 Author: zezutom <zezulatomas@gmail.com> Date: 2017-07-14T12:00:17Z METRON-633 Create better logging for HbaseEnrichmentWriter (zezutom via justinleet) closes apache/metron#572 commit 954b5ea13fc04cb8c9ae277d57a75cfb16956772 Author: jjmeyer <jjmeyer0@gmail.com> Date: 2017-07-14T12:45:59Z METRON-1008 Migrate Travis build to use Trusty (jjmeyer via justinleet) closes apache/metron#633 commit 57d2764c318c8ec7d37d8a3f11e4609e35078285 Author: nickwallen <nick@nickallen.org> Date: 2017-07-17T19:42:40Z METRON-1031 Management UI Cannot Start Topologies in Kerberized Environment (nickwallen) closes apache/metron#647 commit a8bcbf0b4e30cb5803d5bb99a30dd988d8375b32 Author: merrimanr <merrimanr@gmail.com> Date: 2017-07-18T12:53:47Z METRON-1029 Topology worker opts are not being set when Kerberos is enabled (merrimanr) closes apache/metron#645 commit 0d1923f83aee6630f1112cd7f9d20514726272f0 Author: nickwallen <nick@nickallen.org> Date: 2017-07-19T17:19:07Z METRON-1048 Intermittent Test Failure for SimpleHBaseEnrichmentWriterLoggingTest (nickwallen) closes apache/metron#658 commit cf7043c59b1091c59b06cb14ca9d332a727fc3ea Author: merrimanr <merrimanr@gmail.com> Date: 2017-07-20T09:32:45Z METRON-1022 : Elasticsearch REST endpoint this closes apache/incubator-metron#636 commit 3d6b049b8799235b580a275024c18f6092fc8942 Author: merrimanr <merrimanr@gmail.com> Date: 2017-07-20T20:42:55Z METRON-1044 Disabled writers are not acking messages (merrimanr) closes apache/metron#654 commit a0e756bf9a0f8dc4401a0dbac81b23ba87c16616 Author: mmiklavc <michael.miklavcic@gmail.com> Date: 2017-07-21T17:19:11Z METRON-992 : Create performance tuning guide (mmiklavc) closes apache/metron#614 commit 309d8097a91cc40e46f187f672d651b00cdd90fa Author: nickwallen <nick@nickallen.org> Date: 2017-07-24T13:47:06Z METRON-1054 Ambari Mpack Attempts to Kill Topologies That Are Not Running (nickwallen) closes apache/metron#660
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user nickwallen commented on the issue:

          https://github.com/apache/metron/pull/713

          Hi @as22323 - Thanks for submitting a PR. You want the "base branch" of this PR to be "master". Right now it is set to "apache:Metron_0.4.0".

          Show
          githubbot ASF GitHub Bot added a comment - Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/713 Hi @as22323 - Thanks for submitting a PR. You want the "base branch" of this PR to be "master". Right now it is set to "apache:Metron_0.4.0".
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user as22323 commented on the issue:

          https://github.com/apache/metron/pull/713

          Thanks. I have updated to "master".
          According to the Metron Dev Guidelines I am suppose to assign the METRON JIRA ticket to my self. I'm not sure how to do that.

          Show
          githubbot ASF GitHub Bot added a comment - Github user as22323 commented on the issue: https://github.com/apache/metron/pull/713 Thanks. I have updated to "master". According to the Metron Dev Guidelines I am suppose to assign the METRON JIRA ticket to my self. I'm not sure how to do that.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cestella commented on the issue:

          https://github.com/apache/metron/pull/713

          Hm, yeah, I can't seem to assign it to you either @as22323

          Show
          githubbot ASF GitHub Bot added a comment - Github user cestella commented on the issue: https://github.com/apache/metron/pull/713 Hm, yeah, I can't seem to assign it to you either @as22323

            People

            • Assignee:
              AS3233 ashah
              Reporter:
              AS3233 ashah
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development