Avro
  1. Avro
  2. AVRO-328

Performance improvements Validating encoder/decoder for nested records

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: java
    • Labels:
      None

      Description

      With nested records, Parser.pushProduction() gets called multiple times (once per record nesting). Since we do not have any events for beginning and end of records, it enhances the performance if we flatten the grammar, wherever possible.

      The forthcoming patch does just that.

      Try org.apache.avro.io.Perf -N

      On my machine (HP laptop with Cygwin) the improvement is about 20%.

      1. AVRO-328.patch
        8 kB
        Thiruvalluvan M. G.
      2. AVRO-328-test.patch
        2 kB
        Thiruvalluvan M. G.

        Activity

        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Doug Cutting made changes -
        Fix Version/s 1.3.0 [ 12314318 ]
        Thiruvalluvan M. G. made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Thiruvalluvan M. G. added a comment -

        Committed revision 899888.

        Show
        Thiruvalluvan M. G. added a comment - Committed revision 899888.
        Thiruvalluvan M. G. made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Thiruvalluvan M. G. made changes -
        Attachment AVRO-328.patch [ 12430342 ]
        Thiruvalluvan M. G. made changes -
        Attachment AVRO-328-test.patch [ 12430264 ]
        Thiruvalluvan M. G. made changes -
        Attachment AVRO-328.patch [ 12430413 ]
        Attachment AVRO-328-test.patch [ 12430414 ]
        Thiruvalluvan M. G. made changes -
        Attachment AVRO-328.patch [ 12430263 ]
        Thiruvalluvan M. G. made changes -
        Attachment AVRO-328.patch [ 12430342 ]
        Thiruvalluvan M. G. made changes -
        Field Original Value New Value
        Attachment AVRO-328.patch [ 12430263 ]
        Attachment AVRO-328-test.patch [ 12430264 ]
        Hide
        Thiruvalluvan M. G. added a comment -

        Improved Perf to check if it reads back what was written.

        Show
        Thiruvalluvan M. G. added a comment - Improved Perf to check if it reads back what was written.
        Thiruvalluvan M. G. created issue -

          People

          • Assignee:
            Thiruvalluvan M. G.
            Reporter:
            Thiruvalluvan M. G.
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development