Details

    • Type: Sub-task Sub-task
    • Status: Patch Available
    • Priority: Blocker Blocker
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: debian, rpm
    • Labels:
      None
    1. 0001-Avro-packaging.patch
      16 kB
      Sean Mackrory
    2. 0001-BIGTOP-39.-Include-Apache-Avro-in-Bigtop.patch
      15 kB
      Sean Mackrory
    3. 0001-BIGTOP-39.-Include-Apache-Avro-in-Bigtop.patch
      15 kB
      Sean Mackrory
    4. 0001-BIGTOP-39.-Include-Apache-Avro-in-Bigtop.patch
      17 kB
      Sean Mackrory
    5. avro.spec
      4 kB
      Bruno Mahé

      Activity

      Hide
      YoungWoo Kim added a comment -

      Sean Mackrory It seems like the attached patch need to be rebased.

      Show
      YoungWoo Kim added a comment - Sean Mackrory It seems like the attached patch need to be rebased.
      Hide
      Sean Mackrory added a comment -

      As long as there's someone interested in reviewing the patch. I'll re-test before committing, but as any changes will likely be very trivial, I'd rather wait until there's a +1 on the meat of the patch before going through a full round of testing with it again.

      Show
      Sean Mackrory added a comment - As long as there's someone interested in reviewing the patch. I'll re-test before committing, but as any changes will likely be very trivial, I'd rather wait until there's a +1 on the meat of the patch before going through a full round of testing with it again.
      Hide
      Konstantin Boudnik added a comment -

      Is it good to go/shall be carried on in 0.8.0?

      Show
      Konstantin Boudnik added a comment - Is it good to go/shall be carried on in 0.8.0?
      Hide
      Sean Mackrory added a comment -

      And one more round of small fixes - at least they're getting smaller Corrected the RELNOTES_NAME that was copy/pasted, and removed the -javadocs and -sources JARs that started getting packaged after I fixed the doc package.

      Show
      Sean Mackrory added a comment - And one more round of small fixes - at least they're getting smaller Corrected the RELNOTES_NAME that was copy/pasted, and removed the -javadocs and -sources JARs that started getting packaged after I fixed the doc package.
      Hide
      Sean Mackrory added a comment -

      Actually got all the docs working. It now includes all the introductory documents and the combined Java API reference. Also did a bit of general clean-up...

      Show
      Sean Mackrory added a comment - Actually got all the docs working. It now includes all the introductory documents and the combined Java API reference. Also did a bit of general clean-up...
      Hide
      Sean Mackrory added a comment -

      Here is a more serious proposal. Given the dependencies a lot of distros can't satify out-of-the-box and our general focus on JVM languages, I think we should omit the bindings for native and scripting languages for now.

      I've done more testing of "avro-tools" on the command-line, and added a package of documentation. I'd love to replace the JavaDocs for individual packages with the contents of http://avro.apache.org/docs/current/, but I'm still trying to get that to work. Because Forrest.

      Show
      Sean Mackrory added a comment - Here is a more serious proposal. Given the dependencies a lot of distros can't satify out-of-the-box and our general focus on JVM languages, I think we should omit the bindings for native and scripting languages for now. I've done more testing of "avro-tools" on the command-line, and added a package of documentation. I'd love to replace the JavaDocs for individual packages with the contents of http://avro.apache.org/docs/current/ , but I'm still trying to get that to work. Because Forrest.
      Hide
      Sean Mackrory added a comment - - edited

      Like Bruno's attachment, this is not something I want to propose for a commit just yet. It has some known flaws but I think it's a good starting point to packaging Avro. Some points to consider:

      • Fedora already has a package named "avro", so I think we should avoid using that name. On a similar note, should we generally isolate our package namespace from that of the distribution? I don't want to prefix everything with "bigtop-" but it seems that as Linux distributions ship their own Hadoop packages (that may or may not be based on Bigtop's), we're going to eventually have some collisions...
      • I've done very basic testing that this installs "avro-tools" for use on the command-line.
      • I've also included all the other JARs and other language libraries. Which bindings do we want to include? Another JIRA (BIGTOP-898) recently requested including avro-mapred in Hadoop's classpath, and it was suggested maybe it's time we packaged Avro. Without harmonizing the Avro version across components, it seems like this isn't the solution either... Thoughts?
      • The paranemer maven plugin was being troublesome and I"m not very familiar with it. I just disabled it with a patch for now. One of the things I intend to fix before seriously submitting this for review...
      • The C/C++ libraries are also troublesome in that they require a newer version of CMake than some of our supported distributions have, and there's /usr/local is affected by some Debian packaging policies that I'm not yet familiar with.
      Show
      Sean Mackrory added a comment - - edited Like Bruno's attachment, this is not something I want to propose for a commit just yet. It has some known flaws but I think it's a good starting point to packaging Avro. Some points to consider: Fedora already has a package named "avro", so I think we should avoid using that name. On a similar note, should we generally isolate our package namespace from that of the distribution? I don't want to prefix everything with "bigtop-" but it seems that as Linux distributions ship their own Hadoop packages (that may or may not be based on Bigtop's), we're going to eventually have some collisions... I've done very basic testing that this installs "avro-tools" for use on the command-line. I've also included all the other JARs and other language libraries. Which bindings do we want to include? Another JIRA ( BIGTOP-898 ) recently requested including avro-mapred in Hadoop's classpath, and it was suggested maybe it's time we packaged Avro. Without harmonizing the Avro version across components, it seems like this isn't the solution either... Thoughts? The paranemer maven plugin was being troublesome and I"m not very familiar with it. I just disabled it with a patch for now. One of the things I intend to fix before seriously submitting this for review... The C/C++ libraries are also troublesome in that they require a newer version of CMake than some of our supported distributions have, and there's /usr/local is affected by some Debian packaging policies that I'm not yet familiar with.
      Hide
      Bruno Mahé added a comment -

      To be clear, this is not meant to be reviewed for commit. This is just a potential starting point to whoever wants to do the job.

      Show
      Bruno Mahé added a comment - To be clear, this is not meant to be reviewed for commit. This is just a potential starting point to whoever wants to do the job.
      Hide
      Bruno Mahé added a comment -

      Here is an avro spec I had laying around.
      It used to work for 1.3.3. Not sure it would work with latest version.

      Show
      Bruno Mahé added a comment - Here is an avro spec I had laying around. It used to work for 1.3.3. Not sure it would work with latest version.
      Hide
      Bruno Mahé added a comment -

      We can only do 1).
      2) will have to be limited to engage discussions for future releases of each component

      Show
      Bruno Mahé added a comment - We can only do 1). 2) will have to be limited to engage discussions for future releases of each component
      Hide
      Tom White added a comment -

      I think there are two parts to this: i) packaging Avro tools and ii) harmonizing the version of Avro used by components in Bigtop.

      Show
      Tom White added a comment - I think there are two parts to this: i) packaging Avro tools and ii) harmonizing the version of Avro used by components in Bigtop.

        People

        • Assignee:
          Peter Linnell
          Reporter:
          Jeff Hammerbacher
        • Votes:
          1 Vote for this issue
          Watchers:
          8 Start watching this issue

          Dates

          • Created:
            Updated:

            Development