Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-682

New interpreter for Apache Beam (incubating)/DataFlow

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:

      Description

      Zeppelin has pluggagle intrepreter abstraction and multiple implementations for different data processing systems like Flink, Spark, Kylin, Ignite, Geode, etc

      Apache Beam (incubating) is a new promising project with unified stream\batch processing and it would be great to have an interpreter for it.

      This will require creation of Java interpreter first (i.e using existing REPL) as for now Beam only exposes Java API.

        Issue Links

          Activity

          Hide
          minudika001@gmail.com Minudika Malshan added a comment -

          Hi,

          I would like to working on this implementation in GSOC'16.
          Can you please provide some resources to get familiar with Apache Beam?
          I couldn't find a proper documentation to get start with Apache beam yet.

          Thanks a lot.
          Minudika

          Show
          minudika001@gmail.com Minudika Malshan added a comment - Hi, I would like to working on this implementation in GSOC'16. Can you please provide some resources to get familiar with Apache Beam? I couldn't find a proper documentation to get start with Apache beam yet. Thanks a lot. Minudika
          Hide
          wimag Mark Geller added a comment -

          Hi,

          My name is Mark Geller and I am first year masters student at Saint Petersburg State Academic University, Russia. Recently I found out about this project (Interpreter for Apache Beam). And I would like to know, what is considered as requirements for this project? (Apart from motivation to work on the project)

          Show
          wimag Mark Geller added a comment - Hi, My name is Mark Geller and I am first year masters student at Saint Petersburg State Academic University, Russia. Recently I found out about this project (Interpreter for Apache Beam). And I would like to know, what is considered as requirements for this project? (Apart from motivation to work on the project)
          Hide
          bzz Alexander Bezzubov added a comment - - edited

          It's great to see you guys interested in this project.

          Thank you for expressing it here and especially if you joined the community on public mailing list, made introduction or just read through MailingList archives with advices for the students and shared proposal draft!

          Hope you guys have submitted the proposals before the official deadline today!

          Show
          bzz Alexander Bezzubov added a comment - - edited It's great to see you guys interested in this project. Thank you for expressing it here and especially if you joined the community on public mailing list, made introduction or just read through MailingList archives with advices for the students and shared proposal draft! Hope you guys have submitted the proposals before the official deadline today!
          Hide
          iemejia Ismaël Mejía added a comment -

          Following our discussion at ApacheCon I just created a twin issue in the Apache Beam JIRA
          https://issues.apache.org/jira/browse/BEAM-290
          More details soon in the dev mailing list.

          Show
          iemejia Ismaël Mejía added a comment - Following our discussion at ApacheCon I just created a twin issue in the Apache Beam JIRA https://issues.apache.org/jira/browse/BEAM-290 More details soon in the dev mailing list.
          Hide
          bzz Alexander Bezzubov added a comment -

          Looks great, thank you for taking care Ismaël Mejía !

          Show
          bzz Alexander Bezzubov added a comment - Looks great, thank you for taking care Ismaël Mejía !
          Hide
          iemejia Ismaël Mejía added a comment -

          For info I just created a personal branch to start working around the integration, all contributions are welcomed.

          https://github.com/iemejia/incubator-zeppelin/tree/beam

          Show
          iemejia Ismaël Mejía added a comment - For info I just created a personal branch to start working around the integration, all contributions are welcomed. https://github.com/iemejia/incubator-zeppelin/tree/beam
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user mfelgamal opened a pull request:

          https://github.com/apache/zeppelin/pull/1334

          ZEPPELIN-682 New interpreter for Apache Beam (incubating)/DataFlow

            1. What is this PR for?
              The PR is a interpreter for [Apache Beam](http://beam.incubator.apache.org) which is an open source unified platform for data processing pipelines. A pipeline can be build using one of the Beam SDKs.
              The execution of the pipeline is done by different Runners . Currently, Beam supports Apache Flink Runner, Apache Spark Runner, and Google Dataflow Runner.
              1. What type of PR is it?
          • Feature
              1. Todos
          • Test case
          • Review Comments
          • Documentation
              1. What is the Jira issue?
              1. How should this be tested?
          • Start the Zeppelin server
          • The prefix of interpreter is `%beam` and then write your code with required imports and the runner
              1. Screenshots (if appropriate)
                ![](https://s9.postimg.org/s6eiwrbxb/beam_interpreter.png)
                ![](https://s9.postimg.org/eq3h8wsrz/visualisation_with_table.png)
              1. Questions:
          • Does the licenses files need update? no
          • Is there breaking changes for older versions? no
          • Does this needs documentation? yes

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/novelari/incubator-zeppelin beam-interpreter-static-repl-7

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1334.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1334


          commit 2586651ab9768e484e04a840978d6db8d5659f48
          Author: Fouad <fuad.assayadi@gmail.com>
          Date: 2016-05-29T14:39:08Z

          Merge pull request #2 from apache/master

          update from original

          commit 03a1e80f8f3115a75c42fa25fdaa06e7dd2cbc28
          Author: mfelgamal <mahmoudf.elgamal@gmail.com>
          Date: 2016-07-19T18:28:46Z

          Merge pull request #4 from apache/master

          merge master

          commit 1b42276c6799428ce84248ec58222ab4c9dfaf95
          Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
          Date: 2016-06-25T22:03:06Z

          beam interpreter

          commit 1c234caebfb5a05745f07e9536f94a8c2976474f
          Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
          Date: 2016-07-20T14:26:49Z

          Adding the beam to zeppelin 7

          commit aeedccee92513e9e81385537fcded508133b4181
          Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
          Date: 2016-08-01T08:53:12Z

          Refactoring of the code

          commit ae1053d0f983c812fcdbcdfc1e7ad43cc72950db
          Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
          Date: 2016-08-01T10:23:48Z

          Modifying pom file and Making documentation

          commit c2e206655a65a92ff67254f662de6c7d1bd5ccb8
          Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
          Date: 2016-08-03T22:55:32Z

          Modifying the documentation

          commit 0f7338771fb3789ed16d3b9e97def8034ef04d0f
          Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
          Date: 2016-08-08T08:37:52Z

          Adding some tests

          commit b88ff7537f6e9428076aab7461752e6ae006d1f3
          Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
          Date: 2016-08-08T13:10:27Z

          changing class name to StaticRepl and adding some modifications


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user mfelgamal opened a pull request: https://github.com/apache/zeppelin/pull/1334 ZEPPELIN-682 New interpreter for Apache Beam (incubating)/DataFlow What is this PR for? The PR is a interpreter for [Apache Beam] ( http://beam.incubator.apache.org ) which is an open source unified platform for data processing pipelines. A pipeline can be build using one of the Beam SDKs. The execution of the pipeline is done by different Runners . Currently, Beam supports Apache Flink Runner, Apache Spark Runner, and Google Dataflow Runner. What type of PR is it? Feature Todos Test case Review Comments Documentation What is the Jira issue? ZEPPELIN-682 How should this be tested? Start the Zeppelin server The prefix of interpreter is `%beam` and then write your code with required imports and the runner Screenshots (if appropriate) ![]( https://s9.postimg.org/s6eiwrbxb/beam_interpreter.png ) ![]( https://s9.postimg.org/eq3h8wsrz/visualisation_with_table.png ) Questions: Does the licenses files need update? no Is there breaking changes for older versions? no Does this needs documentation? yes You can merge this pull request into a Git repository by running: $ git pull https://github.com/novelari/incubator-zeppelin beam-interpreter-static-repl-7 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1334.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1334 commit 2586651ab9768e484e04a840978d6db8d5659f48 Author: Fouad <fuad.assayadi@gmail.com> Date: 2016-05-29T14:39:08Z Merge pull request #2 from apache/master update from original commit 03a1e80f8f3115a75c42fa25fdaa06e7dd2cbc28 Author: mfelgamal <mahmoudf.elgamal@gmail.com> Date: 2016-07-19T18:28:46Z Merge pull request #4 from apache/master merge master commit 1b42276c6799428ce84248ec58222ab4c9dfaf95 Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com> Date: 2016-06-25T22:03:06Z beam interpreter commit 1c234caebfb5a05745f07e9536f94a8c2976474f Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com> Date: 2016-07-20T14:26:49Z Adding the beam to zeppelin 7 commit aeedccee92513e9e81385537fcded508133b4181 Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com> Date: 2016-08-01T08:53:12Z Refactoring of the code commit ae1053d0f983c812fcdbcdfc1e7ad43cc72950db Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com> Date: 2016-08-01T10:23:48Z Modifying pom file and Making documentation commit c2e206655a65a92ff67254f662de6c7d1bd5ccb8 Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com> Date: 2016-08-03T22:55:32Z Modifying the documentation commit 0f7338771fb3789ed16d3b9e97def8034ef04d0f Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com> Date: 2016-08-08T08:37:52Z Adding some tests commit b88ff7537f6e9428076aab7461752e6ae006d1f3 Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com> Date: 2016-08-08T13:10:27Z changing class name to StaticRepl and adding some modifications
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Thank you for contributing @mfelgamal ! Apache Beam interpreter is a valuable contribution many people are looking for.

          There are few things that need to be done before we can merge it though:

          • this branch need to be rebased on top of the latest master (`git rebase master`) and conflicts must be resolved
          • establish the authorship of this work: this PR includes commits by you and @FouadMA
          • make sure that every file in this PR, as any other file in ASF project, contain Apache2.0 license header
          • make sure that every file is formatted according to [project's code conventions](https://github.com/apache/zeppelin/blob/master/CONTRIBUTING.md#code-convention): pom.xml, .jave, etc.
            From the quick glance at the Java code:
          • please make sure you follow the conventions that are used in Zeppelin projects: brackets, sl4j logging. Best way is to look at other interpreters code
          • classes should have comments explaining their purpose and extra empty lines as well as `@author` tags need to be removed
          • `zeppelin-distribution/src/bin_license` must be updated to include ALL dependencies, AND their transitive dependencies that were added in this PR. You can see all of them in `mvn dependency:tree`.

          Please, feel free to ping me after those issues are addressed and will be happy to look more into it and help you getting this merged!

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Thank you for contributing @mfelgamal ! Apache Beam interpreter is a valuable contribution many people are looking for. There are few things that need to be done before we can merge it though: this branch need to be rebased on top of the latest master (`git rebase master`) and conflicts must be resolved establish the authorship of this work: this PR includes commits by you and @FouadMA make sure that every file in this PR, as any other file in ASF project, contain Apache2.0 license header make sure that every file is formatted according to [project's code conventions] ( https://github.com/apache/zeppelin/blob/master/CONTRIBUTING.md#code-convention): pom.xml, .jave, etc. From the quick glance at the Java code: please make sure you follow the conventions that are used in Zeppelin projects: brackets, sl4j logging. Best way is to look at other interpreters code classes should have comments explaining their purpose and extra empty lines as well as `@author` tags need to be removed `zeppelin-distribution/src/bin_license` must be updated to include ALL dependencies, AND their transitive dependencies that were added in this PR. You can see all of them in `mvn dependency:tree`. Please, feel free to ping me after those issues are addressed and will be happy to look more into it and help you getting this merged!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz The required changes is done, in the recent commit.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz The required changes is done, in the recent commit.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Thank you for addressing the feedback promptly! Please let me take another pass on it and get back to you here.

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Thank you for addressing the feedback promptly! Please let me take another pass on it and get back to you here.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Before proceed with review - I have noticed that CI is failing.
          @mfelgamal could you please take a look and make sure that there are no Beam-related failures? Thanks!

          So far there is

          ```
          [INFO] Zeppelin: Beam interpreter ......................... FAILURE [ 6.632 s]
          [INFO] BUILD FAILURE
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 02:46 min
          [INFO] Finished at: 2016-08-16T23:45:31+00:00
          [INFO] Final Memory: 97M/414M
          [INFO] ------------------------------------------------------------------------
          ....
          [ERROR] Failed to execute goal on project zeppelin-beam: Could not resolve dependencies for project org.apache.zeppelin:zeppelin-beam:jar:0.7.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.beam:beam-runners-flink_2.11:jar:0.1.0-incubating, org.apache.beam:beam-runners-flink_2.11-examples:jar:0.1.0-incubating: Could not find artifact org.apache.beam:beam-runners-flink_2.11:jar:0.1.0-incubating in central (http://repo.maven.apache.org/maven2) -> [Help 1]
          ```

          Which includes dependencies that I can not find mentioned in `./zeppelin-distribution/src/bin-license/LICENSE`.

          > zeppelin-distribution/src/bin_license must be updated to include ALL dependencies, AND their transitive dependencies that were added in this PR

          Could you take a look into it one more time please?

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Before proceed with review - I have noticed that CI is failing. @mfelgamal could you please take a look and make sure that there are no Beam-related failures? Thanks! So far there is ``` [INFO] Zeppelin: Beam interpreter ......................... FAILURE [ 6.632 s] [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 02:46 min [INFO] Finished at: 2016-08-16T23:45:31+00:00 [INFO] Final Memory: 97M/414M [INFO] ------------------------------------------------------------------------ .... [ERROR] Failed to execute goal on project zeppelin-beam: Could not resolve dependencies for project org.apache.zeppelin:zeppelin-beam:jar:0.7.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.beam:beam-runners-flink_2.11:jar:0.1.0-incubating, org.apache.beam:beam-runners-flink_2.11-examples:jar:0.1.0-incubating: Could not find artifact org.apache.beam:beam-runners-flink_2.11:jar:0.1.0-incubating in central ( http://repo.maven.apache.org/maven2 ) -> [Help 1] ``` Which includes dependencies that I can not find mentioned in `./zeppelin-distribution/src/bin-license/LICENSE`. > zeppelin-distribution/src/bin_license must be updated to include ALL dependencies, AND their transitive dependencies that were added in this PR Could you take a look into it one more time please?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz Kindly be noted that there is no maven Scala 2.11 build for beam runner Flink. The only available build is beam-runners-flink_2.10 .
          https://mvnrepository.com/artifact/org.apache.beam/beam-runners-flink_2.10/0.1.0-incubating
          Our pom file :
          ```<groupId>org.apache.beam</groupId>
          <artifactId>beam-runners-flink_2.10</artifactId>
          <version>$

          {beam.beam.version}

          </version>
          ```
          That is why we hardcode it.
          While the failure logs claims that maven can not find `beam-runners-flink_2.11`
          It seems that your builder enforces the usage of Scala 2.11

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz Kindly be noted that there is no maven Scala 2.11 build for beam runner Flink. The only available build is beam-runners-flink_2.10 . https://mvnrepository.com/artifact/org.apache.beam/beam-runners-flink_2.10/0.1.0-incubating Our pom file : ```<groupId>org.apache.beam</groupId> <artifactId>beam-runners-flink_2.10</artifactId> <version>$ {beam.beam.version} </version> ``` That is why we hardcode it. While the failure logs claims that maven can not find `beam-runners-flink_2.11` It seems that your builder enforces the usage of Scala 2.11
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user iemejia commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Good work guys, probably this is the one other approach I missed in the JIRA, to have a static repl to compile the full class and then run it, this is nice because this can be reused for any full file Java case.

          Just two comments:
          I think that it is probably a good idea to move StaticRepl and BeamInterpreter to a more appropriate java package because this thing is a generic Java Repl/Interpreter (restricted to full class definitions). And I don't know if this is the case if it is worth dividing this into two PRs, one for the generic Java thing and other for the Beam specific part (notice that you can easily imagine reusing the java part e.g. to run a Spark or Flink note written in pure Java against a java ambient for example).

          However I don't know the details of how (if) this reuse is possible in zeppelin, any hints @bzz ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user iemejia commented on the issue: https://github.com/apache/zeppelin/pull/1334 Good work guys, probably this is the one other approach I missed in the JIRA, to have a static repl to compile the full class and then run it, this is nice because this can be reused for any full file Java case. Just two comments: I think that it is probably a good idea to move StaticRepl and BeamInterpreter to a more appropriate java package because this thing is a generic Java Repl/Interpreter (restricted to full class definitions). And I don't know if this is the case if it is worth dividing this into two PRs, one for the generic Java thing and other for the Beam specific part (notice that you can easily imagine reusing the java part e.g. to run a Spark or Flink note written in pure Java against a java ambient for example). However I don't know the details of how (if) this reuse is possible in zeppelin, any hints @bzz ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user iemejia commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Oups I forgot the second comment, I don't know if it is also worth to separate every runner as a different interpreter (leaving probably the DirectRunner as the default Beam one). And having the others: beam-spark, beam-flink, beam-cloud-dataflow, as different ones.

          Notice that this will reduce the size of the dependencies for casual use cases (that mostly can be run in local), but let open the option to run those in cluster mode extrictly when needed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user iemejia commented on the issue: https://github.com/apache/zeppelin/pull/1334 Oups I forgot the second comment, I don't know if it is also worth to separate every runner as a different interpreter (leaving probably the DirectRunner as the default Beam one). And having the others: beam-spark, beam-flink, beam-cloud-dataflow, as different ones. Notice that this will reduce the size of the dependencies for casual use cases (that mostly can be run in local), but let open the option to run those in cluster mode extrictly when needed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Let's make sure the new doc is added to the list in _navigation.html?
          https://zeppelin.apache.org/docs/0.6.1/development/writingzeppelininterpreter.html#contributing-a-new-interpreter-to-zeppelin-releases

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1334 Let's make sure the new doc is added to the list in _navigation.html? https://zeppelin.apache.org/docs/0.6.1/development/writingzeppelininterpreter.html#contributing-a-new-interpreter-to-zeppelin-releases
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @felixcheung the requied changes is done.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @felixcheung the requied changes is done.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          > It seems that your builder enforces the usage of Scala 2.11

          @mfelgamal you are right, `.travis.yml` contains Scala 2.11 as well as scala 2.10 profiles.
          In this case, could you please explicitly exclude `-pl "!beam"` from [all such CI build profiles](https://github.com/apache/zeppelin/blob/master/.travis.yml#L38) and put a comment\commit message like `there is no maven Scala 2.11 build for Beam runner on Flink`

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 > It seems that your builder enforces the usage of Scala 2.11 @mfelgamal you are right, `.travis.yml` contains Scala 2.11 as well as scala 2.10 profiles. In this case, could you please explicitly exclude `-pl "!beam"` from [all such CI build profiles] ( https://github.com/apache/zeppelin/blob/master/.travis.yml#L38 ) and put a comment\commit message like `there is no maven Scala 2.11 build for Beam runner on Flink`
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz I updated `.travis.yml` and there is an error while building `Could not find the selected project in the reactor: '!beam'` but the command `-pl '!beam'` run on local without any error.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz I updated `.travis.yml` and there is an error while building `Could not find the selected project in the reactor: '!beam'` but the command `-pl '!beam'` run on local without any error.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @AhyoungRyu the changes is done.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @AhyoungRyu the changes is done.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @mfelgamal Thank you! Please let me look into CI and get back to you

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 @mfelgamal Thank you! Please let me look into CI and get back to you
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz Gentle reminder!

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz Gentle reminder!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @mfelgamal thank you for kind reminder! Will try it more today and get back to you asap.

          Meanwhile, it looks like it needs to be rebased on latest master.

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 @mfelgamal thank you for kind reminder! Will try it more today and get back to you asap. Meanwhile, it looks like it needs to be rebased on latest master.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz rebased

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz rebased
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Few questions after another pass:

          1. Right now Beam interpreter uses *Beam version 0.1.0-incubating*, but didn't Beam project release 0.2.0-incubating on 2016-08-09? Do you think it would be useful to update it right here, before the merge or do you plan to do that in a separate PR later on?

          2. AFAIK there are multiple "runners" for Beam, and right only be reading the doc you can see how to switch them. Do you think it would fit Zeppelin use case better if there are 3 different interpreters `%beam.flink`, `%beam.spark`, etc each with it's own appropriate configuration options and a user could just skip creating a class and defining `main() {}` in every paragraph?

          3. Here is the list of dependencies from `mvn dependency:tree`. All of them will become part of the convenience binary for the next Zeppelin release. What is needed to be done, in order for us to merge it is:

          • make sure that ALL these dependencies
          • and ALL their transitive dependencies
          • have an appropriate licence, and are logged under the `./zeppelin-distribution/src/bin_license/LICENSE`
          • and for non-Apache license, full text of the license need to be added to ./zeppelin-distribution/src/license/<dependency-name>`

          For your convenience I have listed all current dependencies of `beam` module below, please feel free to verify and mark those that satisfy the above criterias.

          ```

          • [ ] org.apache.zeppelin:zeppelin-beam:jar:0.7.0-SNAPSHOT
          • [ ] +- org.apache.spark:spark-core_2.10:jar:1.4.1:compile
          • [ ] | +- com.twitter:chill_2.10:jar:0.5.0:compile
          • [ ] | | - com.esotericsoftware.kryo:kryo:jar:2.21:compile
          • [ ] | | +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:compile
          • [ ] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
          • [ ] | | - org.objenesis:objenesis:jar:1.2:compile
          • [ ] | +- com.twitter:chill-java:jar:0.5.0:compile
          • [ ] | +- org.apache.spark:spark-launcher_2.10:jar:1.4.1:compile
          • [ ] | +- org.apache.spark:spark-network-common_2.10:jar:1.4.1:compile
          • [ ] | +- org.apache.spark:spark-network-shuffle_2.10:jar:1.4.1:compile
          • [ ] | +- org.apache.spark:spark-unsafe_2.10:jar:1.4.1:compile
          • [ ] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile
          • [ ] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile
          • [ ] | | - org.apache.curator:curator-framework:jar:2.4.0:compile
          • [ ] | | - org.apache.curator:curator-client:jar:2.4.0:compile
          • [ ] | +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile
          • [ ] | +- org.apache.commons:commons-lang3:jar:3.3.2:compile
          • [ ] | +- org.apache.commons:commons-math3:jar:3.4.1:compile
          • [ ] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
          • [ ] | +- org.slf4j:slf4j-api:jar:1.7.10:compile
          • [ ] | +- org.slf4j:jul-to-slf4j:jar:1.7.10:compile
          • [ ] | +- org.slf4j:jcl-over-slf4j:jar:1.7.10:compile
          • [ ] | +- log4j:log4j:jar:1.2.17:compile
          • [ ] | +- com.ning:compress-lzf:jar:1.0.3:compile
          • [ ] | +- org.xerial.snappy:snappy-java:jar:1.1.1.7:compile
          • [ ] | +- net.jpountz.lz4:lz4:jar:1.2.0:compile
          • [ ] | +- org.roaringbitmap:RoaringBitmap:jar:0.4.5:compile
          • [ ] | +- commons-net:commons-net:jar:2.2:compile
          • [ ] | +- org.scala-lang:scala-library:jar:2.10.4:compile
          • [ ] | +- org.json4s:json4s-jackson_2.10:jar:3.2.10:compile
          • [ ] | | - org.json4s:json4s-core_2.10:jar:3.2.10:compile
          • [ ] | | +- org.json4s:json4s-ast_2.10:jar:3.2.10:compile
          • [ ] | | - org.scala-lang:scalap:jar:2.10.0:compile
          • [ ] | | - org.scala-lang:scala-compiler:jar:2.10.0:compile
          • [ ] | +- com.sun.jersey:jersey-server:jar:1.9:compile
          • [ ] | | - asm:asm:jar:3.1:compile
          • [ ] | +- com.sun.jersey:jersey-core:jar:1.9:compile
          • [ ] | +- org.apache.mesos:mesos:jar:shaded-protobuf:0.21.1:compile
          • [ ] | +- com.clearspring.analytics:stream:jar:2.7.0:compile
          • [ ] | +- io.dropwizard.metrics:metrics-core:jar:3.1.0:compile
          • [ ] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.0:compile
          • [ ] | +- io.dropwizard.metrics:metrics-json:jar:3.1.0:compile
          • [ ] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.0:compile
          • [ ] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile
          • [ ] | +- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.4.4:compile
          • [ ] | | +- org.scala-lang:scala-reflect:jar:2.10.4:compile
          • [ ] | | - com.thoughtworks.paranamer:paranamer:jar:2.6:compile
          • [ ] | +- org.apache.ivy:ivy:jar:2.4.0:compile
          • [ ] | +- oro:oro:jar:2.0.8:compile
          • [ ] | +- org.tachyonproject:tachyon-client:jar:0.6.4:compile
          • [ ] | | - org.tachyonproject:tachyon:jar:0.6.4:compile
          • [ ] | +- net.razorvine:pyrolite:jar:4.4:compile
          • [ ] | +- net.sf.py4j:py4j:jar:0.8.2.1:compile
          • [ ] | - org.spark-project.spark:unused:jar:1.0.0:compile
          • [ ] +- org.apache.spark:spark-streaming_2.10:jar:1.4.1:compile
          • [ ] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.3.0:compile
          • [ ] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
          • [ ] | +- org.apache.avro:avro:jar:1.7.4:compile
          • [ ] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile
          • [ ] | - io.netty:netty:jar:3.6.2.Final:compile
          • [ ] +- org.apache.hadoop:hadoop-common:jar:2.3.0:compile
          • [ ] | +- com.google.guava:guava:jar:15.0:compile
          • [ ] | +- commons-cli:commons-cli:jar:1.2:compile
          • [ ] | +- xmlenc:xmlenc:jar:0.52:compile
          • [ ] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
          • [ ] | +- commons-codec:commons-codec:jar:1.5:compile
          • [ ] | +- commons-io:commons-io:jar:2.4:compile
          • [ ] | +- commons-collections:commons-collections:jar:3.2.1:compile
          • [ ] | +- javax.servlet:servlet-api:jar:2.5:compile
          • [ ] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
          • [ ] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
          • [ ] | +- com.sun.jersey:jersey-json:jar:1.9:compile
          • [ ] | | +- org.codehaus.jettison:jettison:jar:1.1:compile
          • [ ] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
          • [ ] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile
          • [ ] | | - org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
          • [ ] | +- tomcat:jasper-compiler:jar:5.5.23:compile
          • [ ] | +- tomcat:jasper-runtime:jar:5.5.23:compile
          • [ ] | +- javax.servlet.jsp:jsp-api:jar:2.1:compile
          • [ ] | +- commons-el:commons-el:jar:1.0:compile
          • [ ] | +- commons-logging:commons-logging:jar:1.1.1:compile
          • [ ] | +- commons-lang:commons-lang:jar:2.5:compile
          • [ ] | +- commons-configuration:commons-configuration:jar:1.9:compile
          • [ ] | +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
          • [ ] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
          • [ ] | +- org.apache.hadoop:hadoop-auth:jar:2.3.0:compile
          • [ ] | | - org.apache.httpcomponents:httpclient:jar:4.3.6:compile
          • [ ] | | - org.apache.httpcomponents:httpcore:jar:4.3.3:compile
          • [ ] | +- com.jcraft:jsch:jar:0.1.42:compile
          • [ ] | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile
          • [ ] | - org.apache.commons:commons-compress:jar:1.4.1:compile
          • [ ] | - org.tukaani:xz:jar:1.0:compile
          • [ ] +- org.apache.hadoop:hadoop-hdfs:jar:2.3.0:compile
          • [ ] | - commons-daemon:commons-daemon:jar:1.0.13:compile
          • [ ] +- org.apache.hadoop:hadoop-client:jar:2.3.0:compile
          • [ ] | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.3.0:compile
          • [ ] | | - org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.3.0:compile
          • [ ] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.3.0:compile
          • [ ] | - org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.3.0:compile
          • [ ] +- org.apache.hadoop:hadoop-annotations:jar:2.3.0:compile
          • [ ] | - jdk.tools:jdk.tools:jar:1.7:system
          • [ ] +- org.apache.hadoop:hadoop-yarn-common:jar:2.3.0:compile
          • [ ] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile
          • [ ] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile
          • [ ] | | - javax.activation:activation:jar:1.1:compile
          • [ ] | +- com.google.inject:guice:jar:3.0:compile
          • [ ] | | +- javax.inject:javax.inject:jar:1:compile
          • [ ] | | - aopalliance:aopalliance:jar:1.0:compile
          • [ ] | - com.sun.jersey.contribs:jersey-guice:jar:1.9:compile
          • [ ] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.3.0:compile
          • [ ] | +- org.apache.hadoop:hadoop-yarn-client:jar:2.3.0:compile
          • [ ] | - org.apache.hadoop:hadoop-yarn-server-common:jar:2.3.0:compile
          • [ ] +- org.apache.hadoop:hadoop-core:jar:1.2.1:compile
          • [ ] | +- org.apache.commons:commons-math:jar:2.1:compile
          • [ ] | +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:compile
          • [ ] | | - org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:compile
          • [ ] | +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:compile
          • [ ] | | - ant:ant:jar:1.6.5:compile
          • [ ] | +- hsqldb:hsqldb:jar:1.8.0.10:compile
          • [ ] | - org.eclipse.jdt:core:jar:3.1.1:compile
          • [ ] +- com.thoughtworks.qdox:qdox:jar:2.0-M3:compile
          • [ ] +- org.apache.beam:beam-runners-parent:pom:0.1.0-incubating:compile
          • [ ] +- org.apache.beam:beam-runners-core-java:jar:0.1.0-incubating:compile
          • [ ] | - org.apache.beam:beam-sdks-java-core:jar:0.1.0-incubating:compile
          • [ ] | +- io.grpc:grpc-all:jar:0.12.0:compile
          • [ ] | | +- io.grpc:grpc-core:jar:0.12.0:compile
          • [ ] | | +- io.grpc:grpc-okhttp:jar:0.12.0:compile
          • [ ] | | | +- com.squareup.okio:okio:jar:1.6.0:compile
          • [ ] | | | - com.squareup.okhttp:okhttp:jar:2.5.0:compile
          • [ ] | | +- io.grpc:grpc-protobuf-nano:jar:0.12.0:compile
          • [ ] | | | - com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-4:compile
          • [ ] | | +- io.grpc:grpc-netty:jar:0.12.0:compile
          • [ ] | | | +- io.netty:netty-codec-http2:jar:4.1.0.Beta8:compile
          • [ ] | | | | - io.netty:netty-codec-http:jar:4.1.0.Beta8:compile
          • [ ] | | | - com.twitter:hpack:jar:0.10.1:compile
          • [ ] | | +- io.grpc:grpc-protobuf:jar:0.12.0:compile
          • [ ] | | +- io.grpc:grpc-stub:jar:0.12.0:compile
          • [ ] | | - io.grpc:grpc-auth:jar:0.12.0:compile
          • [ ] | +- com.google.auth:google-auth-library-oauth2-http:jar:0.4.0:compile
          • [ ] | | - com.google.auth:google-auth-library-credentials:jar:0.4.0:compile
          • [ ] | +- io.netty:netty-handler:jar:4.1.0.Beta8:compile
          • [ ] | | +- io.netty:netty-buffer:jar:4.1.0.Beta8:compile
          • [ ] | | | - io.netty:netty-common:jar:4.1.0.Beta8:compile
          • [ ] | | +- io.netty:netty-transport:jar:4.1.0.Beta8:compile
          • [ ] | | | - io.netty:netty-resolver:jar:4.1.0.Beta8:compile
          • [ ] | | - io.netty:netty-codec:jar:4.1.0.Beta8:compile
          • [ ] | +- com.google.api.grpc:grpc-pubsub-v1:jar:0.0.2:compile
          • [ ] | | - com.google.api.grpc:grpc-core-proto:jar:0.0.3:compile
          • [ ] | +- com.google.apis:google-api-services-pubsub:jar:v1-rev10-1.22.0:compile
          • [ ] | +- com.google.apis:google-api-services-storage:jar:v1-rev71-1.22.0:compile
          • [ ] | +- com.google.http-client:google-http-client-jackson:jar:1.22.0:runtime
          • [ ] | +- com.google.oauth-client:google-oauth-client-java6:jar:1.22.0:compile
          • [ ] | - com.google.cloud.bigdataoss:gcsio:jar:1.4.5:compile
          • [ ] +- org.apache.beam:beam-runners-direct-java:jar:0.1.0-incubating:compile
          • [ ] | +- com.google.http-client:google-http-client-protobuf:jar:1.22.0:runtime
          • [ ] | +- joda-time:joda-time:jar:2.4:compile
          • [ ] | - com.fasterxml.jackson.core:jackson-annotations:jar:2.7.2:compile
          • [ ] +- org.apache.beam:beam-runners-flink_2.10:jar:0.1.0-incubating:compile
          • [ ] | +- org.apache.flink:flink-streaming-java_2.10:jar:1.0.0:compile
          • [ ] | | +- org.apache.flink:flink-core:jar:1.0.0:compile
          • [ ] | | | - org.apache.flink:flink-annotations:jar:1.0.0:compile
          • [ ] | | +- org.apache.flink:flink-runtime_2.10:jar:1.0.0:compile
          • [ ] | | | +- org.javassist:javassist:jar:3.18.2-GA:compile
          • [ ] | | | +- com.typesafe.akka:akka-actor_2.10:jar:2.3.7:compile
          • [ ] | | | | - com.typesafe:config:jar:1.2.1:compile
          • [ ] | | | +- com.typesafe.akka:akka-remote_2.10:jar:2.3.7:compile
          • [ ] | | | | - org.uncommons.maths:uncommons-maths:jar:1.2.2a:compile
          • [ ] | | | +- com.typesafe.akka:akka-slf4j_2.10:jar:2.3.7:compile
          • [ ] | | | +- org.clapper:grizzled-slf4j_2.10:jar:1.0.2:compile
          • [ ] | | | - com.github.scopt:scopt_2.10:jar:3.2.0:compile
          • [ ] | | +- org.apache.sling:org.apache.sling.commons.json:jar:2.0.6:compile
          • [ ] | | - org.apache.flink:force-shading:jar:1.0.0:compile
          • [ ] | +- org.apache.flink:flink-java:jar:1.0.0:compile
          • [ ] | | - org.apache.flink:flink-shaded-hadoop2:jar:1.0.0:compile
          • [ ] | | +- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:compile
          • [ ] | | +- commons-digester:commons-digester:jar:1.8.1:compile
          • [ ] | | - commons-beanutils:commons-beanutils-bean-collections:jar:1.8.3:compile
          • [ ] | - org.apache.flink:flink-clients_2.10:jar:1.0.0:compile
          • [ ] | - org.apache.flink:flink-optimizer_2.10:jar:1.0.0:compile
          • [ ] +- org.apache.beam:beam-runners-flink_2.10-examples:jar:0.1.0-incubating:compile
          • [ ] | - org.apache.flink:flink-connector-kafka-0.8_2.10:jar:1.0.0:compile
          • [ ] | +- org.apache.flink:flink-connector-kafka-base_2.10:jar:1.0.0:compile
          • [ ] | | - com.101tec:zkclient:jar:0.7:compile
          • [ ] | - org.apache.kafka:kafka_2.10:jar:0.8.2.2:compile
          • [ ] | +- org.apache.kafka:kafka-clients:jar:0.8.2.2:compile
          • [ ] | - com.yammer.metrics:metrics-core:jar:2.2.0:compile
          • [ ] +- org.apache.beam:beam-runners-google-cloud-dataflow-java:jar:0.1.0-incubating:compile
          • [ ] | +- com.google.api-client:google-api-client:jar:1.22.0:compile
          • [ ] | +- com.google.apis:google-api-services-datastore-protobuf:jar:v1beta2-rev1-4.0.0:compile
          • [ ] | +- com.google.oauth-client:google-oauth-client:jar:1.22.0:compile
          • [ ] | +- com.google.http-client:google-http-client:jar:1.22.0:compile
          • [ ] | +- com.google.apis:google-api-services-dataflow:jar:v1b3-rev26-1.22.0:compile
          • [ ] | +- com.google.apis:google-api-services-clouddebugger:jar:v2-rev8-1.22.0:compile
          • [ ] | +- com.google.apis:google-api-services-bigquery:jar:v2-rev295-1.22.0:compile
          • [ ] | +- com.google.cloud.bigdataoss:util:jar:1.4.5:compile
          • [ ] | | +- com.google.api-client:google-api-client-java6:jar:1.20.0:compile
          • [ ] | | - com.google.api-client:google-api-client-jackson2:jar:1.20.0:compile
          • [ ] | - com.fasterxml.jackson.core:jackson-core:jar:2.7.2:compile
          • [ ] +- org.apache.beam:beam-runners-spark:jar:0.1.0-incubating:compile
          • [ ] | +- org.apache.beam:beam-examples-java:jar:0.1.0-incubating:compile
          • [ ] | | - javax.servlet:javax.servlet-api:jar:3.1.0:compile
          • [ ] | - org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
          • [ ] | +- org.apache.avro:avro-ipc:jar:1.7.7:compile
          • [ ] | | - org.apache.velocity:velocity:jar:1.7:compile
          • [ ] | - org.apache.avro:avro-ipc:jar:tests:1.7.7:compile
          • [ ] +- org.apache.commons:commons-exec:jar:1.3:compile
            ```

          Hope this helps, and please let me know if you have any questions!

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Few questions after another pass: 1. Right now Beam interpreter uses * Beam version 0.1.0-incubating *, but didn't Beam project release 0.2.0-incubating on 2016-08-09? Do you think it would be useful to update it right here, before the merge or do you plan to do that in a separate PR later on? 2. AFAIK there are multiple "runners" for Beam, and right only be reading the doc you can see how to switch them. Do you think it would fit Zeppelin use case better if there are 3 different interpreters `%beam.flink`, `%beam.spark`, etc each with it's own appropriate configuration options and a user could just skip creating a class and defining `main() {}` in every paragraph? 3. Here is the list of dependencies from `mvn dependency:tree`. All of them will become part of the convenience binary for the next Zeppelin release. What is needed to be done, in order for us to merge it is: make sure that ALL these dependencies and ALL their transitive dependencies have an appropriate licence, and are logged under the `./zeppelin-distribution/src/bin_license/LICENSE` and for non-Apache license, full text of the license need to be added to ./zeppelin-distribution/src/license/<dependency-name>` For your convenience I have listed all current dependencies of `beam` module below, please feel free to verify and mark those that satisfy the above criterias. ``` [ ] org.apache.zeppelin:zeppelin-beam:jar:0.7.0-SNAPSHOT [ ] +- org.apache.spark:spark-core_2.10:jar:1.4.1:compile [ ] | +- com.twitter:chill_2.10:jar:0.5.0:compile [ ] | | - com.esotericsoftware.kryo:kryo:jar:2.21:compile [ ] | | +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:compile [ ] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile [ ] | | - org.objenesis:objenesis:jar:1.2:compile [ ] | +- com.twitter:chill-java:jar:0.5.0:compile [ ] | +- org.apache.spark:spark-launcher_2.10:jar:1.4.1:compile [ ] | +- org.apache.spark:spark-network-common_2.10:jar:1.4.1:compile [ ] | +- org.apache.spark:spark-network-shuffle_2.10:jar:1.4.1:compile [ ] | +- org.apache.spark:spark-unsafe_2.10:jar:1.4.1:compile [ ] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile [ ] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile [ ] | | - org.apache.curator:curator-framework:jar:2.4.0:compile [ ] | | - org.apache.curator:curator-client:jar:2.4.0:compile [ ] | +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile [ ] | +- org.apache.commons:commons-lang3:jar:3.3.2:compile [ ] | +- org.apache.commons:commons-math3:jar:3.4.1:compile [ ] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile [ ] | +- org.slf4j:slf4j-api:jar:1.7.10:compile [ ] | +- org.slf4j:jul-to-slf4j:jar:1.7.10:compile [ ] | +- org.slf4j:jcl-over-slf4j:jar:1.7.10:compile [ ] | +- log4j:log4j:jar:1.2.17:compile [ ] | +- com.ning:compress-lzf:jar:1.0.3:compile [ ] | +- org.xerial.snappy:snappy-java:jar:1.1.1.7:compile [ ] | +- net.jpountz.lz4:lz4:jar:1.2.0:compile [ ] | +- org.roaringbitmap:RoaringBitmap:jar:0.4.5:compile [ ] | +- commons-net:commons-net:jar:2.2:compile [ ] | +- org.scala-lang:scala-library:jar:2.10.4:compile [ ] | +- org.json4s:json4s-jackson_2.10:jar:3.2.10:compile [ ] | | - org.json4s:json4s-core_2.10:jar:3.2.10:compile [ ] | | +- org.json4s:json4s-ast_2.10:jar:3.2.10:compile [ ] | | - org.scala-lang:scalap:jar:2.10.0:compile [ ] | | - org.scala-lang:scala-compiler:jar:2.10.0:compile [ ] | +- com.sun.jersey:jersey-server:jar:1.9:compile [ ] | | - asm:asm:jar:3.1:compile [ ] | +- com.sun.jersey:jersey-core:jar:1.9:compile [ ] | +- org.apache.mesos:mesos:jar:shaded-protobuf:0.21.1:compile [ ] | +- com.clearspring.analytics:stream:jar:2.7.0:compile [ ] | +- io.dropwizard.metrics:metrics-core:jar:3.1.0:compile [ ] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.0:compile [ ] | +- io.dropwizard.metrics:metrics-json:jar:3.1.0:compile [ ] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.0:compile [ ] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile [ ] | +- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.4.4:compile [ ] | | +- org.scala-lang:scala-reflect:jar:2.10.4:compile [ ] | | - com.thoughtworks.paranamer:paranamer:jar:2.6:compile [ ] | +- org.apache.ivy:ivy:jar:2.4.0:compile [ ] | +- oro:oro:jar:2.0.8:compile [ ] | +- org.tachyonproject:tachyon-client:jar:0.6.4:compile [ ] | | - org.tachyonproject:tachyon:jar:0.6.4:compile [ ] | +- net.razorvine:pyrolite:jar:4.4:compile [ ] | +- net.sf.py4j:py4j:jar:0.8.2.1:compile [ ] | - org.spark-project.spark:unused:jar:1.0.0:compile [ ] +- org.apache.spark:spark-streaming_2.10:jar:1.4.1:compile [ ] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.3.0:compile [ ] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile [ ] | +- org.apache.avro:avro:jar:1.7.4:compile [ ] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile [ ] | - io.netty:netty:jar:3.6.2.Final:compile [ ] +- org.apache.hadoop:hadoop-common:jar:2.3.0:compile [ ] | +- com.google.guava:guava:jar:15.0:compile [ ] | +- commons-cli:commons-cli:jar:1.2:compile [ ] | +- xmlenc:xmlenc:jar:0.52:compile [ ] | +- commons-httpclient:commons-httpclient:jar:3.1:compile [ ] | +- commons-codec:commons-codec:jar:1.5:compile [ ] | +- commons-io:commons-io:jar:2.4:compile [ ] | +- commons-collections:commons-collections:jar:3.2.1:compile [ ] | +- javax.servlet:servlet-api:jar:2.5:compile [ ] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile [ ] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile [ ] | +- com.sun.jersey:jersey-json:jar:1.9:compile [ ] | | +- org.codehaus.jettison:jettison:jar:1.1:compile [ ] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile [ ] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile [ ] | | - org.codehaus.jackson:jackson-xc:jar:1.8.3:compile [ ] | +- tomcat:jasper-compiler:jar:5.5.23:compile [ ] | +- tomcat:jasper-runtime:jar:5.5.23:compile [ ] | +- javax.servlet.jsp:jsp-api:jar:2.1:compile [ ] | +- commons-el:commons-el:jar:1.0:compile [ ] | +- commons-logging:commons-logging:jar:1.1.1:compile [ ] | +- commons-lang:commons-lang:jar:2.5:compile [ ] | +- commons-configuration:commons-configuration:jar:1.9:compile [ ] | +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile [ ] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile [ ] | +- org.apache.hadoop:hadoop-auth:jar:2.3.0:compile [ ] | | - org.apache.httpcomponents:httpclient:jar:4.3.6:compile [ ] | | - org.apache.httpcomponents:httpcore:jar:4.3.3:compile [ ] | +- com.jcraft:jsch:jar:0.1.42:compile [ ] | +- org.apache.zookeeper:zookeeper:jar:3.4.5:compile [ ] | - org.apache.commons:commons-compress:jar:1.4.1:compile [ ] | - org.tukaani:xz:jar:1.0:compile [ ] +- org.apache.hadoop:hadoop-hdfs:jar:2.3.0:compile [ ] | - commons-daemon:commons-daemon:jar:1.0.13:compile [ ] +- org.apache.hadoop:hadoop-client:jar:2.3.0:compile [ ] | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.3.0:compile [ ] | | - org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.3.0:compile [ ] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.3.0:compile [ ] | - org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.3.0:compile [ ] +- org.apache.hadoop:hadoop-annotations:jar:2.3.0:compile [ ] | - jdk.tools:jdk.tools:jar:1.7:system [ ] +- org.apache.hadoop:hadoop-yarn-common:jar:2.3.0:compile [ ] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile [ ] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile [ ] | | - javax.activation:activation:jar:1.1:compile [ ] | +- com.google.inject:guice:jar:3.0:compile [ ] | | +- javax.inject:javax.inject:jar:1:compile [ ] | | - aopalliance:aopalliance:jar:1.0:compile [ ] | - com.sun.jersey.contribs:jersey-guice:jar:1.9:compile [ ] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.3.0:compile [ ] | +- org.apache.hadoop:hadoop-yarn-client:jar:2.3.0:compile [ ] | - org.apache.hadoop:hadoop-yarn-server-common:jar:2.3.0:compile [ ] +- org.apache.hadoop:hadoop-core:jar:1.2.1:compile [ ] | +- org.apache.commons:commons-math:jar:2.1:compile [ ] | +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:compile [ ] | | - org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:compile [ ] | +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:compile [ ] | | - ant:ant:jar:1.6.5:compile [ ] | +- hsqldb:hsqldb:jar:1.8.0.10:compile [ ] | - org.eclipse.jdt:core:jar:3.1.1:compile [ ] +- com.thoughtworks.qdox:qdox:jar:2.0-M3:compile [ ] +- org.apache.beam:beam-runners-parent:pom:0.1.0-incubating:compile [ ] +- org.apache.beam:beam-runners-core-java:jar:0.1.0-incubating:compile [ ] | - org.apache.beam:beam-sdks-java-core:jar:0.1.0-incubating:compile [ ] | +- io.grpc:grpc-all:jar:0.12.0:compile [ ] | | +- io.grpc:grpc-core:jar:0.12.0:compile [ ] | | +- io.grpc:grpc-okhttp:jar:0.12.0:compile [ ] | | | +- com.squareup.okio:okio:jar:1.6.0:compile [ ] | | | - com.squareup.okhttp:okhttp:jar:2.5.0:compile [ ] | | +- io.grpc:grpc-protobuf-nano:jar:0.12.0:compile [ ] | | | - com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-4:compile [ ] | | +- io.grpc:grpc-netty:jar:0.12.0:compile [ ] | | | +- io.netty:netty-codec-http2:jar:4.1.0.Beta8:compile [ ] | | | | - io.netty:netty-codec-http:jar:4.1.0.Beta8:compile [ ] | | | - com.twitter:hpack:jar:0.10.1:compile [ ] | | +- io.grpc:grpc-protobuf:jar:0.12.0:compile [ ] | | +- io.grpc:grpc-stub:jar:0.12.0:compile [ ] | | - io.grpc:grpc-auth:jar:0.12.0:compile [ ] | +- com.google.auth:google-auth-library-oauth2-http:jar:0.4.0:compile [ ] | | - com.google.auth:google-auth-library-credentials:jar:0.4.0:compile [ ] | +- io.netty:netty-handler:jar:4.1.0.Beta8:compile [ ] | | +- io.netty:netty-buffer:jar:4.1.0.Beta8:compile [ ] | | | - io.netty:netty-common:jar:4.1.0.Beta8:compile [ ] | | +- io.netty:netty-transport:jar:4.1.0.Beta8:compile [ ] | | | - io.netty:netty-resolver:jar:4.1.0.Beta8:compile [ ] | | - io.netty:netty-codec:jar:4.1.0.Beta8:compile [ ] | +- com.google.api.grpc:grpc-pubsub-v1:jar:0.0.2:compile [ ] | | - com.google.api.grpc:grpc-core-proto:jar:0.0.3:compile [ ] | +- com.google.apis:google-api-services-pubsub:jar:v1-rev10-1.22.0:compile [ ] | +- com.google.apis:google-api-services-storage:jar:v1-rev71-1.22.0:compile [ ] | +- com.google.http-client:google-http-client-jackson:jar:1.22.0:runtime [ ] | +- com.google.oauth-client:google-oauth-client-java6:jar:1.22.0:compile [ ] | - com.google.cloud.bigdataoss:gcsio:jar:1.4.5:compile [ ] +- org.apache.beam:beam-runners-direct-java:jar:0.1.0-incubating:compile [ ] | +- com.google.http-client:google-http-client-protobuf:jar:1.22.0:runtime [ ] | +- joda-time:joda-time:jar:2.4:compile [ ] | - com.fasterxml.jackson.core:jackson-annotations:jar:2.7.2:compile [ ] +- org.apache.beam:beam-runners-flink_2.10:jar:0.1.0-incubating:compile [ ] | +- org.apache.flink:flink-streaming-java_2.10:jar:1.0.0:compile [ ] | | +- org.apache.flink:flink-core:jar:1.0.0:compile [ ] | | | - org.apache.flink:flink-annotations:jar:1.0.0:compile [ ] | | +- org.apache.flink:flink-runtime_2.10:jar:1.0.0:compile [ ] | | | +- org.javassist:javassist:jar:3.18.2-GA:compile [ ] | | | +- com.typesafe.akka:akka-actor_2.10:jar:2.3.7:compile [ ] | | | | - com.typesafe:config:jar:1.2.1:compile [ ] | | | +- com.typesafe.akka:akka-remote_2.10:jar:2.3.7:compile [ ] | | | | - org.uncommons.maths:uncommons-maths:jar:1.2.2a:compile [ ] | | | +- com.typesafe.akka:akka-slf4j_2.10:jar:2.3.7:compile [ ] | | | +- org.clapper:grizzled-slf4j_2.10:jar:1.0.2:compile [ ] | | | - com.github.scopt:scopt_2.10:jar:3.2.0:compile [ ] | | +- org.apache.sling:org.apache.sling.commons.json:jar:2.0.6:compile [ ] | | - org.apache.flink:force-shading:jar:1.0.0:compile [ ] | +- org.apache.flink:flink-java:jar:1.0.0:compile [ ] | | - org.apache.flink:flink-shaded-hadoop2:jar:1.0.0:compile [ ] | | +- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:compile [ ] | | +- commons-digester:commons-digester:jar:1.8.1:compile [ ] | | - commons-beanutils:commons-beanutils-bean-collections:jar:1.8.3:compile [ ] | - org.apache.flink:flink-clients_2.10:jar:1.0.0:compile [ ] | - org.apache.flink:flink-optimizer_2.10:jar:1.0.0:compile [ ] +- org.apache.beam:beam-runners-flink_2.10-examples:jar:0.1.0-incubating:compile [ ] | - org.apache.flink:flink-connector-kafka-0.8_2.10:jar:1.0.0:compile [ ] | +- org.apache.flink:flink-connector-kafka-base_2.10:jar:1.0.0:compile [ ] | | - com.101tec:zkclient:jar:0.7:compile [ ] | - org.apache.kafka:kafka_2.10:jar:0.8.2.2:compile [ ] | +- org.apache.kafka:kafka-clients:jar:0.8.2.2:compile [ ] | - com.yammer.metrics:metrics-core:jar:2.2.0:compile [ ] +- org.apache.beam:beam-runners-google-cloud-dataflow-java:jar:0.1.0-incubating:compile [ ] | +- com.google.api-client:google-api-client:jar:1.22.0:compile [ ] | +- com.google.apis:google-api-services-datastore-protobuf:jar:v1beta2-rev1-4.0.0:compile [ ] | +- com.google.oauth-client:google-oauth-client:jar:1.22.0:compile [ ] | +- com.google.http-client:google-http-client:jar:1.22.0:compile [ ] | +- com.google.apis:google-api-services-dataflow:jar:v1b3-rev26-1.22.0:compile [ ] | +- com.google.apis:google-api-services-clouddebugger:jar:v2-rev8-1.22.0:compile [ ] | +- com.google.apis:google-api-services-bigquery:jar:v2-rev295-1.22.0:compile [ ] | +- com.google.cloud.bigdataoss:util:jar:1.4.5:compile [ ] | | +- com.google.api-client:google-api-client-java6:jar:1.20.0:compile [ ] | | - com.google.api-client:google-api-client-jackson2:jar:1.20.0:compile [ ] | - com.fasterxml.jackson.core:jackson-core:jar:2.7.2:compile [ ] +- org.apache.beam:beam-runners-spark:jar:0.1.0-incubating:compile [ ] | +- org.apache.beam:beam-examples-java:jar:0.1.0-incubating:compile [ ] | | - javax.servlet:javax.servlet-api:jar:3.1.0:compile [ ] | - org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile [ ] | +- org.apache.avro:avro-ipc:jar:1.7.7:compile [ ] | | - org.apache.velocity:velocity:jar:1.7:compile [ ] | - org.apache.avro:avro-ipc:jar:tests:1.7.7:compile [ ] +- org.apache.commons:commons-exec:jar:1.3:compile ``` Hope this helps, and please let me know if you have any questions!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz kindly be noted for point # 2:
          Making user do minimal amount of code is good but the only cons is that the user will be given the default implementations and configurations and can not change it. This constraint is because we are using a very basic java REPL implementation. If this is fine, we can proceed and implement it!

          What do you think about that?

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz kindly be noted for point # 2: Making user do minimal amount of code is good but the only cons is that the user will be given the default implementations and configurations and can not change it. This constraint is because we are using a very basic java REPL implementation. If this is fine, we can proceed and implement it! What do you think about that?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz The following updates are made:
          1 - Updating beam version from 0.1 to 0.2
          2 - License report is updated for all dependencies

          Regarding point #2 in the previous message, do you have any further discussion on it. I think we should make it as is for now.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz The following updates are made: 1 - Updating beam version from 0.1 to 0.2 2 - License report is updated for all dependencies Regarding point #2 in the previous message, do you have any further discussion on it. I think we should make it as is for now.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          I think having it as it is now sounds as a good start. It can be always enhanced it later on, based on user feedback.

          Changes look great to me, thank you!

          If that is ok - please let me test it more over the weekend and merge it early next week, if there is no further discussion!

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 I think having it as it is now sounds as a good start. It can be always enhanced it later on, based on user feedback. Changes look great to me, thank you! If that is ok - please let me test it more over the weekend and merge it early next week, if there is no further discussion!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          After running it locally, I think it is great but the last thing that would be very nice to have, is to expand the example in the docs to be self-contained, meaning easily runnable by the the first-time Beam user.

          Also rebase this branch on the latest master is needed (I guess `.travis.yaml` was changed a bit by skipping RAT in all default profiles)

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 After running it locally, I think it is great but the last thing that would be very nice to have, is to expand the example in the docs to be self-contained, meaning easily runnable by the the first-time Beam user. Also rebase this branch on the latest master is needed (I guess `.travis.yaml` was changed a bit by skipping RAT in all default profiles)
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz all required changes are done.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz all required changes are done.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Looks great to me, thank you @mfelgamal !

          I think it's ready to be merged.

          The only minor thing that looks a bit off - re-formatted `pom.xml` is not consistent with the other modules, where just two spaces indentation is used, see [root pom.xml](https://github.com/apache/zeppelin/blob/master/pom.xml) as an example.

          If that is not hard - could you format pom.xml again please, for maintainability, with 2 white space indentation settings?

          I'll be happy to merge it right after!

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Looks great to me, thank you @mfelgamal ! I think it's ready to be merged. The only minor thing that looks a bit off - re-formatted `pom.xml` is not consistent with the other modules, where just two spaces indentation is used, see [root pom.xml] ( https://github.com/apache/zeppelin/blob/master/pom.xml ) as an example. If that is not hard - could you format pom.xml again please, for maintainability, with 2 white space indentation settings? I'll be happy to merge it right after!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @mfelgamal to address 1-2 can you please try to update `.travis.yml`: `-pl !beam` -> `-pl !beam` ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 @mfelgamal to address 1-2 can you please try to update `.travis.yml`: `-pl !beam` -> `-pl !beam` ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz the changes are done, could you take a look at pom file and CI ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz the changes are done, could you take a look at pom file and CI ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          CI is still failing.

          Sorry for confusion, after a bit more thinking it looks like simplest way here actually will be - to follow the same convention as with any other interpreters, like `geode`, `sparkr`, `pyspark ` or `scalding`:

          • create a `beam` profile in root `./pom.xml` that brings Beam module. See i.e [scalding](https://github.com/apache/zeppelin/blob/master/pom.xml#L578)
          • explicitly add `-Pbeam` only to CI envs lines in `.travis.yml`, that already have scala 2.10
          • create a `./beam/README.md` file and add some implementation details about interpreter impl there i.e it's a good place to mention that right now only Scala 2.10 is supported due to Flink runner.

          This will allow avoid any possible CI issues in future and keep build modular.

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 CI is still failing. Sorry for confusion, after a bit more thinking it looks like simplest way here actually will be - to follow the same convention as with any other interpreters, like `geode`, `sparkr`, `pyspark ` or `scalding`: create a `beam` profile in root `./pom.xml` that brings Beam module. See i.e [scalding] ( https://github.com/apache/zeppelin/blob/master/pom.xml#L578 ) explicitly add `-Pbeam` only to CI envs lines in `.travis.yml`, that already have scala 2.10 create a `./beam/README.md` file and add some implementation details about interpreter impl there i.e it's a good place to mention that right now only Scala 2.10 is supported due to Flink runner. This will allow avoid any possible CI issues in future and keep build modular.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz done.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz done.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz Gentle reminder!

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz Gentle reminder!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @felixcheung required changes are done.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @felixcheung required changes are done.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @felixcheung done, and now it checks if the main method is static or not.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @felixcheung done, and now it checks if the main method is static or not.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Last 2 profile in CI are failing

          ```
          [INFO] Zeppelin: Display system apis ...................... FAILURE [ 2.537 s]

          - should provide onclick method *** FAILED ***
           The code passed to eventually never returned normally. Attempted 1 times over 193.154233 milliseconds. Last failure message: 0 was not equal to 1. (AbstractAngularElemTest.scala:72)
          AngularElem
          ```

          and

          ```
          [INFO] Zeppelin: Server ................................... FAILURE [07:56 min]

          Tests in error:
          AuthenticationIT.testGroupPermission:179->AbstractZeppelinIT.pollingWait:96 » Timeout

          Tests run: 15, Failures: 0, Errors: 1, Skipped: 0
          ```

          which seems not related to changes.

          Let's wait if @felixcheung have more feedback, and merge to master if all issue are addressed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Last 2 profile in CI are failing ``` [INFO] Zeppelin: Display system apis ...................... FAILURE [ 2.537 s] - should provide onclick method *** FAILED ***  The code passed to eventually never returned normally. Attempted 1 times over 193.154233 milliseconds. Last failure message: 0 was not equal to 1. (AbstractAngularElemTest.scala:72) AngularElem ``` and ``` [INFO] Zeppelin: Server ................................... FAILURE [07:56 min] Tests in error: AuthenticationIT.testGroupPermission:179->AbstractZeppelinIT.pollingWait:96 » Timeout Tests run: 15, Failures: 0, Errors: 1, Skipped: 0 ``` which seems not related to changes. Let's wait if @felixcheung have more feedback, and merge to master if all issue are addressed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          LGTM other than the minor comment on static check

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1334 LGTM other than the minor comment on static check
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz @felixcheung Thank you for your reviews.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz @felixcheung Thank you for your reviews.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Great, thank you @mfelgamal for great job laying foundation for Apache Beam integration! I hate to ask but, could you please rebase it on latest master one more time? Looks like it have diverged a bit..

          I will be happy to merge it to master ASAP right after that.

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Great, thank you @mfelgamal for great job laying foundation for Apache Beam integration! I hate to ask but, could you please rebase it on latest master one more time? Looks like it have diverged a bit.. I will be happy to merge it to master ASAP right after that.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz rebased.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz rebased.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Thank you @mfelgamal !
          Merging to master, if there is no further discussion

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Thank you @mfelgamal ! Merging to master, if there is no further discussion
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mfelgamal commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @bzz For me, no further discussion. looking forward to seeing the PR merged.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1334 @bzz For me, no further discussion. looking forward to seeing the PR merged.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/zeppelin/pull/1334

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/1334
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Thank you so much @mfelgamal !

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/1334 Thank you so much @mfelgamal !
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user iemejia commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Congratulations guys, excellent work !

          Show
          githubbot ASF GitHub Bot added a comment - Github user iemejia commented on the issue: https://github.com/apache/zeppelin/pull/1334 Congratulations guys, excellent work !
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user iemejia commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          @mfelgamal or @bzz you must announce this milestone in the beam mailing list too.

          Show
          githubbot ASF GitHub Bot added a comment - Github user iemejia commented on the issue: https://github.com/apache/zeppelin/pull/1334 @mfelgamal or @bzz you must announce this milestone in the beam mailing list too.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user kennknowles commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          :+1:!

          Show
          githubbot ASF GitHub Bot added a comment - Github user kennknowles commented on the issue: https://github.com/apache/zeppelin/pull/1334 :+1:!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babupe commented on the issue:

          https://github.com/apache/zeppelin/pull/1334

          Great work guys!

          Show
          githubbot ASF GitHub Bot added a comment - Github user babupe commented on the issue: https://github.com/apache/zeppelin/pull/1334 Great work guys!

            People

            • Assignee:
              Unassigned
              Reporter:
              bzz Alexander Bezzubov
            • Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development