Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2.0
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      DataFormat for Google Protocol Buffers aka protobuf

      http://code.google.com/apis/protocolbuffers/
      http://code.google.com/p/protobuf/

      The Google project uses 'New BSD License'. Is this a compatible license? Camel Extra seems to just be for GPL style licenses.

      1. camel-protobuf.zip
        11 kB
        Martin Gilday

        Activity

        Hide
        Martin Gilday added a comment -

        Need to decide which exception should be thrown when unmarshalling a payload which is incomplete/missing required fields.
        Need a way to test that exception is thrown correctly. The generated builders by protobuf don't allow creation of incomplete instances.

        Show
        Martin Gilday added a comment - Need to decide which exception should be thrown when unmarshalling a payload which is incomplete/missing required fields. Need a way to test that exception is thrown correctly. The generated builders by protobuf don't allow creation of incomplete instances.
        Hide
        Claus Ibsen added a comment -

        The licenses is listed here
        http://www.apache.org/legal/3party.html

        Show
        Claus Ibsen added a comment - The licenses is listed here http://www.apache.org/legal/3party.html
        Hide
        Claus Ibsen added a comment -

        That code generator thing? How does it work with unit testing?

        Its preferred if its able to be unit tested completely from a mvn clean install cmd.

        Show
        Claus Ibsen added a comment - That code generator thing? How does it work with unit testing? Its preferred if its able to be unit tested completely from a mvn clean install cmd.
        Hide
        Martin Gilday added a comment -

        The code generator is something you only run once during the design phase of your project to create the message classes (well once per language). I've already run it an placed the generated file under src/test/java.
        If you want to run it every time then we could hook it up to the generate-resources Maven phase and invoke it using the Ant plugin. However the code generator is a platform dependant executable. Do you know how this is worked around with the graphviz testing?

        Do you think we need to run the code generator at all?

        Show
        Martin Gilday added a comment - The code generator is something you only run once during the design phase of your project to create the message classes (well once per language). I've already run it an placed the generated file under src/test/java. If you want to run it every time then we could hook it up to the generate-resources Maven phase and invoke it using the Ant plugin. However the code generator is a platform dependant executable. Do you know how this is worked around with the graphviz testing? Do you think we need to run the code generator at all?
        Hide
        Claus Ibsen added a comment -

        No I think its fine if the code is generated once and thus not needed to be every time.

        Show
        Claus Ibsen added a comment - No I think its fine if the code is generated once and thus not needed to be every time.
        Hide
        Claus Ibsen added a comment -

        Martin can you help with providing documentation for this new data format?

        Show
        Claus Ibsen added a comment - Martin can you help with providing documentation for this new data format?
        Hide
        Martin Gilday added a comment -

        Yes I can write the docs for the wiki. Do you plan to add it to the DSL, as that would affect the usage?

        Show
        Martin Gilday added a comment - Yes I can write the docs for the wiki. Do you plan to add it to the DSL, as that would affect the usage?
        Hide
        Claus Ibsen added a comment -

        Yeah we do have the data formats in the DSL.

        Often there are a few options you can configure etc. We can start without the DSL and then add that a bit later.

        Show
        Claus Ibsen added a comment - Yeah we do have the data formats in the DSL. Often there are a few options you can configure etc. We can start without the DSL and then add that a bit later.
        Hide
        Willem Jiang added a comment -

        Don't worry, I'm doing the DSL part now

        Show
        Willem Jiang added a comment - Don't worry, I'm doing the DSL part now
        Hide
        Willem Jiang added a comment -

        Hi Martin,

        Can your generate the AddressBookProtos with this package org.apache.camel.dataformat.protobuf.generated again?
        As I did some refactoring on the package name and move the generated file there to avoid the CheckStyle checking.

        Here is another thing, it look like the ProtobufDataFormat can only unmashal the input stream with a defaultInstance, is there a more generic way to do it without specify the defaultInstance?

        Show
        Willem Jiang added a comment - Hi Martin, Can your generate the AddressBookProtos with this package org.apache.camel.dataformat.protobuf.generated again? As I did some refactoring on the package name and move the generated file there to avoid the CheckStyle checking. Here is another thing, it look like the ProtobufDataFormat can only unmashal the input stream with a defaultInstance, is there a more generic way to do it without specify the defaultInstance?
        Hide
        Martin Gilday added a comment -

        Willem,
        I can run it again this evening.
        I asked in the protobuf mailing list but the response was that you need to tell it the type of class to unmarshal too

        http://groups.google.com/group/protobuf/browse_frm/thread/74260cf14746b961?tvc=1&q=camel

        Show
        Martin Gilday added a comment - Willem, I can run it again this evening. I asked in the protobuf mailing list but the response was that you need to tell it the type of class to unmarshal too http://groups.google.com/group/protobuf/browse_frm/thread/74260cf14746b961?tvc=1&q=camel
        Hide
        Willem Jiang added a comment -

        Hi Martin,
        I found the way to generate the AddressBookProtos myself, so you don't need to regenerate the file.

        Show
        Willem Jiang added a comment - Hi Martin, I found the way to generate the AddressBookProtos myself, so you don't need to regenerate the file.
        Hide
        Willem Jiang added a comment -

        Committed the patch of Martin with thank, some package refactoring , coding formatting.
        I also added the DSL for protobuf, like this

        from("direct:marshal").marshal().protobuf();
                        from("direct:unmarshal").unmarshal().protobuf("org.apache.camel.dataformat.protobuf.generated.AddressBookProtos$Person").to("mock:reverse");
        

        http://svn.apache.org/viewvc?rev=892109&view=rev

        TODO update the wiki page

        Show
        Willem Jiang added a comment - Committed the patch of Martin with thank, some package refactoring , coding formatting. I also added the DSL for protobuf, like this from( "direct:marshal" ).marshal().protobuf(); from( "direct:unmarshal" ).unmarshal().protobuf( "org.apache.camel.dataformat.protobuf.generated.AddressBookProtos$Person" ).to( "mock:reverse" ); http://svn.apache.org/viewvc?rev=892109&view=rev TODO update the wiki page
        Hide
        Claus Ibsen added a comment -

        I noticed the pom.xml was set to jar packaging and not bundle.

        And I think the code that does ObjectHelper.loadClass should use camelContext.resolveClass instead as its OSGi friendly.
        See for example camel-bindy which has such a feature to load classes at runtime that works with OSGi also.

        Show
        Claus Ibsen added a comment - I noticed the pom.xml was set to jar packaging and not bundle. And I think the code that does ObjectHelper.loadClass should use camelContext.resolveClass instead as its OSGi friendly. See for example camel-bindy which has such a feature to load classes at runtime that works with OSGi also.
        Hide
        Claus Ibsen added a comment -

        And there should be a entry added to the features as well

        I wonder if the google .jar is OSGi bundle already or do we need to provide our own wrapped in SMX repo?

        Show
        Claus Ibsen added a comment - And there should be a entry added to the features as well I wonder if the google .jar is OSGi bundle already or do we need to provide our own wrapped in SMX repo?
        Hide
        Willem Jiang added a comment -

        yeah, using CamleContext.resolveClass is a better idea
        Current the google jar is not OSGI ready, we need to wrap it in the SMX repo.

        Show
        Willem Jiang added a comment - yeah, using CamleContext.resolveClass is a better idea Current the google jar is not OSGI ready, we need to wrap it in the SMX repo.
        Hide
        Martin Gilday added a comment -

        Thanks Willem for applying this. Is it possible for the DSL to also allow you to pass in the Class as I had it originally, as an overload to the String version? Then we get the option of type checking in the IDE.

        Show
        Martin Gilday added a comment - Thanks Willem for applying this. Is it possible for the DSL to also allow you to pass in the Class as I had it originally, as an overload to the String version? Then we get the option of type checking in the IDE.
        Hide
        Willem Jiang added a comment -

        Hi Martin,

        It's not easy to pass a Message instance into an Spring version of DSL, so I introduced the InstanceClass in ProtobufDataFormat.
        But I think I can add a Java DSL for setting the Message instance into the ProtobufDataFormat.

        Show
        Willem Jiang added a comment - Hi Martin, It's not easy to pass a Message instance into an Spring version of DSL, so I introduced the InstanceClass in ProtobufDataFormat. But I think I can add a Java DSL for setting the Message instance into the ProtobufDataFormat.
        Hide
        Willem Jiang added a comment -

        OH, I remember why I don't add the .protobuf(Message defaultInstance) into the DataFormatClause.
        It is because I don't want to introduce a new dependency (google protobuf) into camel-core.

        Show
        Willem Jiang added a comment - OH, I remember why I don't add the .protobuf(Message defaultInstance) into the DataFormatClause. It is because I don't want to introduce a new dependency (google protobuf) into camel-core.
        Hide
        Claus Ibsen added a comment -

        You can use a (Object instance) method instead? And then do some validation in camel-protobof if that provided instance is an instanceof Message

        Show
        Claus Ibsen added a comment - You can use a (Object instance) method instead? And then do some validation in camel-protobof if that provided instance is an instanceof Message
        Hide
        Martin Gilday added a comment -

        That sounds like a good idea

        Show
        Martin Gilday added a comment - That sounds like a good idea
        Hide
        Martin Gilday added a comment -

        Started the documentation http://cwiki.apache.org/confluence/display/CAMEL/Protobuf
        Just requires the DSL sections completing once we know the options we will provide.

        Show
        Martin Gilday added a comment - Started the documentation http://cwiki.apache.org/confluence/display/CAMEL/Protobuf Just requires the DSL sections completing once we know the options we will provide.
        Hide
        Willem Jiang added a comment -

        update the wiki page for the Java DSL and Spring configuration example.

        Show
        Willem Jiang added a comment - update the wiki page for the Java DSL and Spring configuration example.

          People

          • Assignee:
            Willem Jiang
            Reporter:
            Martin Gilday
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development