Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.8.0
    • Component/s: general
    • Labels:
      None

      Activity

      Konstantin Boudnik created issue -
      Hide
      Konstantin Boudnik added a comment - - edited

      New build dependencies were introduced:

      error: Failed build dependencies:
              python-devel is needed by hue-3.5.0-1.el6.x86_64
              python-setuptools is needed by hue-3.5.0-1.el6.x86_64
              libxml2-devel is needed by hue-3.5.0-1.el6.x86_64
              libxslt-devel is needed by hue-3.5.0-1.el6.x86_64
              cyrus-sasl-devel is needed by hue-3.5.0-1.el6.x86_64
              sqlite-devel is needed by hue-3.5.0-1.el6.x86_64
              openldap-devel is needed by hue-3.5.0-1.el6.x86_64
              mysql-devel is needed by hue-3.5.0-1.el6.x86_64
      

      I guess we need to update bigtop_toolchain. Or per Mark, RPM expresses biuld time deps. (not sure about DEB) and rpmbuild should be pulling in everything automatically. However, it doesn't happen to me - not on CentOS6.4 at least.

      Show
      Konstantin Boudnik added a comment - - edited New build dependencies were introduced: error: Failed build dependencies: python-devel is needed by hue-3.5.0-1.el6.x86_64 python-setuptools is needed by hue-3.5.0-1.el6.x86_64 libxml2-devel is needed by hue-3.5.0-1.el6.x86_64 libxslt-devel is needed by hue-3.5.0-1.el6.x86_64 cyrus-sasl-devel is needed by hue-3.5.0-1.el6.x86_64 sqlite-devel is needed by hue-3.5.0-1.el6.x86_64 openldap-devel is needed by hue-3.5.0-1.el6.x86_64 mysql-devel is needed by hue-3.5.0-1.el6.x86_64 I guess we need to update bigtop_toolchain. Or per Mark, RPM expresses biuld time deps. (not sure about DEB) and rpmbuild should be pulling in everything automatically. However, it doesn't happen to me - not on CentOS6.4 at least.
      Hide
      Romain Rigaux added a comment - - edited

      The main change since Hue 2.x is that Hue requires Python 2.6+. Except this it should be all the same.

      These packages have been there since Hue 2: https://github.com/cloudera/hue#development-prerequisites

      FYI Hue 3.6 is out now: http://gethue.com/hadoop-ui-hue-3-6-and-the-search-dashboards-are-out/

      Show
      Romain Rigaux added a comment - - edited The main change since Hue 2.x is that Hue requires Python 2.6+. Except this it should be all the same. These packages have been there since Hue 2: https://github.com/cloudera/hue#development-prerequisites FYI Hue 3.6 is out now: http://gethue.com/hadoop-ui-hue-3-6-and-the-search-dashboards-are-out/
      Hide
      Roman Shaposhnik added a comment -

      Thanks Romain Rigaux! I'll be working on Hue over the weekend.

      Show
      Roman Shaposhnik added a comment - Thanks Romain Rigaux ! I'll be working on Hue over the weekend.
      Roman Shaposhnik made changes -
      Field Original Value New Value
      Summary bump Hue version to 3.5.0 bump Hue version to 3.6.0
      Hide
      Roman Shaposhnik added a comment -

      Please review.

      Show
      Roman Shaposhnik added a comment - Please review.
      Roman Shaposhnik made changes -
      Roman Shaposhnik made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      Hide
      Peter Linnell added a comment -

      +1 Nice add with Spark and ZK.

      Show
      Peter Linnell added a comment - +1 Nice add with Spark and ZK.
      Roman Shaposhnik made changes -
      Status Patch Available [ 10002 ] Resolved [ 5 ]
      Resolution Fixed [ 1 ]
      Hide
      Gordon Wang added a comment -

      +1 nice to have Spark.

      Show
      Gordon Wang added a comment - +1 nice to have Spark.
      Hide
      Sean Mackrory added a comment -

      Note that the Zookeeper app requires the ZK REST server to be run, which you currently have to build and run from source yourself. There's a patch for packaging here: BIGTOP-1267. The Spark app also relies on the Spark Job Server, and last I checked it was not being accepted upstream very well, so it's not something we can include in our existing spark packages very easily - it would need to be a new thing entirely...

      Show
      Sean Mackrory added a comment - Note that the Zookeeper app requires the ZK REST server to be run, which you currently have to build and run from source yourself. There's a patch for packaging here: BIGTOP-1267 . The Spark app also relies on the Spark Job Server, and last I checked it was not being accepted upstream very well, so it's not something we can include in our existing spark packages very easily - it would need to be a new thing entirely...
      Hide
      Roman Shaposhnik added a comment -

      Sean Mackrory thanks for the heads up. Do you have pointers to the Spark Job Server code (perhaps on JIRAs, etc.) that we can consider including into Bigtop?

      Show
      Roman Shaposhnik added a comment - Sean Mackrory thanks for the heads up. Do you have pointers to the Spark Job Server code (perhaps on JIRAs, etc.) that we can consider including into Bigtop?
      Hide
      Sean Mackrory added a comment -

      There's a walkthrough of what you have to do to get the Spark app working here: http://gethue.com/a-new-spark-web-ui-spark-app/. Specifically, the Job Server code is here: https://github.com/ooyala/spark-jobserver

      Show
      Sean Mackrory added a comment - There's a walkthrough of what you have to do to get the Spark app working here: http://gethue.com/a-new-spark-web-ui-spark-app/ . Specifically, the Job Server code is here: https://github.com/ooyala/spark-jobserver
      Hide
      Roman Shaposhnik added a comment -

      Sean Mackrory interesting! Do you have any insights into this being accepted by upstream Spark community?

      Show
      Roman Shaposhnik added a comment - Sean Mackrory interesting! Do you have any insights into this being accepted by upstream Spark community?
      Hide
      Romain Rigaux added a comment -

      It is seen as a 'contrib' project for now. It should move forward in Q3-Q4.

      Show
      Romain Rigaux added a comment - It is seen as a 'contrib' project for now. It should move forward in Q3-Q4.
      Hide
      Roman Shaposhnik added a comment -

      Romain Rigaux what do you mean by 'contrib' project? The Github repo looks totally separate from Apache Spark. Have they move the code to contrib under Spark at some point?

      Show
      Roman Shaposhnik added a comment - Romain Rigaux what do you mean by 'contrib' project? The Github repo looks totally separate from Apache Spark. Have they move the code to contrib under Spark at some point?
      Hide
      Romain Rigaux added a comment -

      For now we don't know if it will be accepted into the Spark repo for real (it is a bit too early too).

      Show
      Romain Rigaux added a comment - For now we don't know if it will be accepted into the Spark repo for real (it is a bit too early too).
      Transition Time In Source Status Execution Times Last Executer Last Execution Date
      Open Open Patch Available Patch Available
      21d 22h 44m 1 Roman Shaposhnik 28/Jun/14 19:25
      Patch Available Patch Available Resolved Resolved
      32m 10s 1 Roman Shaposhnik 28/Jun/14 19:57

        People

        • Assignee:
          Roman Shaposhnik
          Reporter:
          Konstantin Boudnik
        • Votes:
          0 Vote for this issue
          Watchers:
          6 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development