Avro
  1. Avro
  2. AVRO-494

the genavro format should support defaults

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.3.2
    • Fix Version/s: 1.4.0
    • Component/s: java
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      It would be great if the genavro format also supported assigning default values.

        Activity

        Hide
        Doug Cutting added a comment -

        I committed this, updating the documentation too.

        I filed AVRO-590 for the issue of specifying ordering.

        Show
        Doug Cutting added a comment - I committed this, updating the documentation too. I filed AVRO-590 for the issue of specifying ordering.
        Hide
        Eric Evans added a comment -

        If it were me, I'd commit what you have and do the rest in another issue. The only reason that ordering ever made it into this one is because the annotation/decorator syntax was supposed to support that as well, (and didn't).

        Show
        Eric Evans added a comment - If it were me, I'd commit what you have and do the rest in another issue. The only reason that ordering ever made it into this one is because the annotation/decorator syntax was supposed to support that as well, (and didn't).
        Hide
        Doug Cutting added a comment -

        This adds more tests and fixes some problems they identified.

        Should I also add "increasing", "decreasing" and "unordered" keywords before I commit this, or should we commit this as-is and address ordering in a separate issue?

        Show
        Doug Cutting added a comment - This adds more tests and fixes some problems they identified. Should I also add "increasing", "decreasing" and "unordered" keywords before I commit this, or should we commit this as-is and address ordering in a separate issue?
        Hide
        Eric Evans added a comment -

        I like it. +1

        Show
        Eric Evans added a comment - I like it. +1
        Hide
        Todd Lipcon added a comment -

        Patch looks good to me. Consider adding a test case that tests the more complicated JSON? (eg a default for a record, and array, not just a string)

        Show
        Todd Lipcon added a comment - Patch looks good to me. Consider adding a test case that tests the more complicated JSON? (eg a default for a record, and array, not just a string)
        Hide
        Doug Cutting added a comment -

        Here's a patch that implements my suggestion above. Arbitrary JSON can be used to specify default values for fields.

        Show
        Doug Cutting added a comment - Here's a patch that implements my suggestion above. Arbitrary JSON can be used to specify default values for fields.
        Hide
        Hudson added a comment -

        Integrated in Cassandra #474 (See http://hudson.zones.apache.org/hudson/job/Cassandra/474/)
        custom avro build (genavro patched for defaults)

        See AVRO-494

        Patch by eevans

        Show
        Hudson added a comment - Integrated in Cassandra #474 (See http://hudson.zones.apache.org/hudson/job/Cassandra/474/ ) custom avro build (genavro patched for defaults) See AVRO-494 Patch by eevans
        Hide
        Doug Cutting added a comment -

        > it's actually adding them as properties to the attributes type object.

        If we want to use attributes, then we'd need to special-case the type attributes "order" and "default", which would mean that you couldn't also have a type attribute named "order" or "default".

        Another syntax to consider for default values might be something like:

        string foo = "foo";

        Ordering might be done with keywords in the record field definitition, like:

        decreasing string foo = "foo";
        unordered string bar = "bar";

        As for including JSON, I think we need to add grammar rules for Json to the parser. Todd, is that what you tried?

        Show
        Doug Cutting added a comment - > it's actually adding them as properties to the attributes type object. If we want to use attributes, then we'd need to special-case the type attributes "order" and "default", which would mean that you couldn't also have a type attribute named "order" or "default". Another syntax to consider for default values might be something like: string foo = "foo"; Ordering might be done with keywords in the record field definitition, like: decreasing string foo = "foo"; unordered string bar = "bar"; As for including JSON, I think we need to add grammar rules for Json to the parser. Todd, is that what you tried?
        Hide
        Todd Lipcon added a comment -

        I spent a few hours trying to figure out how to use Jackson and JavaCC together in such a way that I could put inline JSON in certain places in genavro. Unfortunately it turned out to be really rather difficult because of lookahead, etc. Any JavaCC or Jackson gurus out there?

        Show
        Todd Lipcon added a comment - I spent a few hours trying to figure out how to use Jackson and JavaCC together in such a way that I could put inline JSON in certain places in genavro. Unfortunately it turned out to be really rather difficult because of lookahead, etc. Any JavaCC or Jackson gurus out there?
        Hide
        Eric Evans added a comment -

        The parser was written to handle arbitrary attributes using the @identifer annotation, but this isn't working as expected; it's actually adding them as properties to the attributes type object.

        So in other words you end up with:

        {name: aname, type: {type: string, order: ignore}}
        

        ... instead of:

        {name: aname, type: string, order: ignore}
        

        The patch attached here parses the attributes for default and order into JsonNode and Field.Order instances respectively and passes them into the Field constructor. There is probably a better way. For example, this requires you to supply an attribute that parses as valid JSON. For example:

        record FooRecord {
          @default(false)
          boolean yesno;
        
          @default(100)
          int number;
          
          /* Fails; parses to the bareword ignore */
          @order("ignore")
          string badstring;
        
          /* Succeeds; parses to "ignore" */
          @order("\"ignore\"")
          string goodstring;
        }
        

        Thoughts?

        Show
        Eric Evans added a comment - The parser was written to handle arbitrary attributes using the @identifer annotation, but this isn't working as expected; it's actually adding them as properties to the attributes type object. So in other words you end up with: {name: aname, type: {type: string, order: ignore}} ... instead of: {name: aname, type: string, order: ignore} The patch attached here parses the attributes for default and order into JsonNode and Field.Order instances respectively and passes them into the Field constructor. There is probably a better way. For example, this requires you to supply an attribute that parses as valid JSON. For example: record FooRecord { @default(false) boolean yesno; @default(100) int number; /* Fails; parses to the bareword ignore */ @order("ignore") string badstring; /* Succeeds; parses to "ignore" */ @order("\"ignore\"") string goodstring; } Thoughts?
        Hide
        Hudson added a comment -

        Integrated in Cassandra #399 (See http://hudson.zones.apache.org/hudson/job/Cassandra/399/)
        genavro schema file (not to be used until )

        Patch by eevans

        Show
        Hudson added a comment - Integrated in Cassandra #399 (See http://hudson.zones.apache.org/hudson/job/Cassandra/399/ ) genavro schema file (not to be used until ) Patch by eevans

          People

          • Assignee:
            Doug Cutting
            Reporter:
            Eric Evans
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development