Avro
  1. Avro
  2. AVRO-711

JSON encoder and decoder for C++

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.5.0
    • Component/s: c++
    • Labels:
      None
    • Environment:

      Linux, Cygwin

      Description

      Forthcoming patch implements JSON encoder and decoder for C++.

      1. AVRO-711.patch
        198 kB
        Thiruvalluvan M. G.
      2. AVRO-711.patch
        170 kB
        Thiruvalluvan M. G.
      3. AVRO-711.patch
        169 kB
        Thiruvalluvan M. G.
      4. AVRO-711.patch
        122 kB
        Thiruvalluvan M. G.

        Activity

        Hide
        Thiruvalluvan M. G. added a comment - - edited

        This patch implements the JSON encoder and decoder for C++. The design is very similar to that of Java implementation. The patch is incomplete in two respects:

        • The decoder does not resolve schema. The Java implementation extends validating decoder to support resolution. I'll submit that as a separate patch.
        • The decoder and encoder do not handle recursive schema. They throw when a recursive schema is encountered. Again, I'll address this separately.

        Instead of modifying the existing interface, I added two new interfaces Encoder and Decoder in the same lines as Java. So none of the existing applications will have trouble.

        The new implementation uses the buffers and validSchema which we already have.

        As a side benefit there are binary encoder/decoder and validating encoder/decoder using the new interface.

        The new implementation is rigorously tested using the same test cases as that of Java.

        This patch may have conflicts with AVRO-700. I'll take care of them once AVRO-700 is done.

        Show
        Thiruvalluvan M. G. added a comment - - edited This patch implements the JSON encoder and decoder for C++. The design is very similar to that of Java implementation. The patch is incomplete in two respects: The decoder does not resolve schema. The Java implementation extends validating decoder to support resolution. I'll submit that as a separate patch. The decoder and encoder do not handle recursive schema. They throw when a recursive schema is encountered. Again, I'll address this separately. Instead of modifying the existing interface, I added two new interfaces Encoder and Decoder in the same lines as Java. So none of the existing applications will have trouble. The new implementation uses the buffers and validSchema which we already have. As a side benefit there are binary encoder/decoder and validating encoder/decoder using the new interface. The new implementation is rigorously tested using the same test cases as that of Java. This patch may have conflicts with AVRO-700 . I'll take care of them once AVRO-700 is done.
        Hide
        Scott Banachowski added a comment -

        Patch works OK for me. Would you like me to check it in now?

        Show
        Scott Banachowski added a comment - Patch works OK for me. Would you like me to check it in now?
        Hide
        Thiruvalluvan M. G. added a comment -

        This new patch handles recursive schemas and resolution. The only issue pending is that it does not insert default values for the fields that reader's schema has but writer's schema lacks.
        ValidSchema class does not yet have support for default values. We'll address these two together.

        Show
        Thiruvalluvan M. G. added a comment - This new patch handles recursive schemas and resolution. The only issue pending is that it does not insert default values for the fields that reader's schema has but writer's schema lacks. ValidSchema class does not yet have support for default values. We'll address these two together.
        Hide
        Thiruvalluvan M. G. added a comment -

        Same as the previous patch. But this is tested on the following:

        • RHEL 4.7 with CMake 2.6, Boost 1.41.0
        • Ubuntu 10.04 with CMake 2.8.0, Boost 1.38.0
        • Cygwin 1.7 with CMake 2.8.2, Boost 1.43.0
        Show
        Thiruvalluvan M. G. added a comment - Same as the previous patch. But this is tested on the following: RHEL 4.7 with CMake 2.6, Boost 1.41.0 Ubuntu 10.04 with CMake 2.8.0, Boost 1.38.0 Cygwin 1.7 with CMake 2.8.2, Boost 1.43.0
        Hide
        Thiruvalluvan M. G. added a comment -

        Slightly better than the previous one. There was a CMake-2.6-specific variation. Removed it.

        Show
        Thiruvalluvan M. G. added a comment - Slightly better than the previous one. There was a CMake-2.6-specific variation. Removed it.
        Hide
        Thiruvalluvan M. G. added a comment - - edited

        This is functionally same as the previous patch. But it uses IO streams instead of buffers. This improves performance and paves the way to implement Avro container files.

        This patch has been tested on

        • Ubuntu 10.04
        • RHEL 4.8
        • Cygwin
        Show
        Thiruvalluvan M. G. added a comment - - edited This is functionally same as the previous patch. But it uses IO streams instead of buffers. This improves performance and paves the way to implement Avro container files. This patch has been tested on Ubuntu 10.04 RHEL 4.8 Cygwin
        Hide
        Thiruvalluvan M. G. added a comment -

        Committed revision 1072677 after adjusting fot AVRO-700.

        Show
        Thiruvalluvan M. G. added a comment - Committed revision 1072677 after adjusting fot AVRO-700 .

          People

          • Assignee:
            Thiruvalluvan M. G.
            Reporter:
            Thiruvalluvan M. G.
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development