Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-1724

avrogen.exe "import" handling

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: csharp
    • Labels:
      None

      Description

      I have two simple JSON Schema in SAMEfolder:

      Content in "simpleA.avsc" as follows:
      {
      "namespace": "com.bearstearns",
      "type": "record",
      "name": "simpleA",
      "imports": ["simpleB.avsc"],
      "fields":
      [

      {"name":"appName","type":"string"}

      ,

      {"name":"userId","type":"string"}

      ,
      {"name":"nestedB","type":[

      {"type":"array","items":"simpleB"}

      ,"null"]}
      ]
      }

      Content in "simpleB.avsc" as follows:
      {"namespace": "com.bearstearns",
      "type": "record",
      "name": "simpleB",
      "fields":
      [

      {"name":"ID","type":"string"}

      ,

      {"name":"SimpleName","type":"string"}

      ]
      }

      Avrogen can be downloaded from http://apache.communilink.net/avro/avro-1.7.7/avro-csharp-1.7.7.tar.gz

      After download, from command prompt set PATH environment variable:
      set PATH=%PATH%;C:\...\avro\codegen\Release

      From Command Prompt,
      avrogen -s simpleA.avsc outputDir

      The error message is:
      "Exception occurred. Undefined name: simpleB"

      By the way, you cannot get around this by putting two schema in one file: Only one on top will be generated. The one in bottom won't. For example below if you put simpleB top, comma, then simpleA bottom. Only simpleB.cs will be generated (simpleA.cs won't)
      {"namespace": "com.bearstearns",
      "type": "record",
      "name": "simpleB",
      "fields":
      [

      {"name":"ID","type":"string"}

      ,

      {"name":"SimpleName","type":"string"}

      ]
      },
      {"namespace": "com.bearstearns",
      "type": "record",
      "name": "simpleA",
      "fields":
      [

      {"name":"appName","type":"string"}

      ,

      {"name":"userId","type":"string"}

      ,
      {"name":"nestedB","type":[

      {"type":"array","items":"simpleB"}

      ,"null"]}
      ]
      }

      However, a workaround is to nest simpleA and simpleB in same avsc - this will work:
      {
      "namespace": "com.bearstearns",
      "type": "record",
      "name": "simpleA",
      "fields":
      [

      {"name":"appName","type":"string"}

      ,

      {"name":"userId","type":"string"}

      ,
      {"name":"nestedB","type":{"type":"array","items":{
      "type": "record",
      "name": "simpleB",
      "fields":
      [

      {"name":"ID","type":"string"}

      ,

      {"name":"SimpleName","type":"string"}

      ]
      }
      }}
      ]
      }

      fyi, I also try using avrogen -p switch in combination with protocol file, no joy.
      For this method, I first define the protocol file "bearstearns.avdl"
      @namespace("com.bearstearns")
      protocol BearStearns {
      import schema "simpleA.avsc";
      import schema "simpleB.avsc";
      }

      Then from command prompt:
      avrogen -p bearstearns.avdl outputDir

      This time, the error message is,
      Exception occurred. Invalid JSON format: @namespace("com.bearstearns")
      protocol BearStearns {
      import schema "simpleA.avsc";
      import schema "simpleB.avsc";
      }

      Actually is Avro support for dotnet ready yet? Avro official doc for C# is completely empty: http://avro.apache.org/docs/current/api/csharp/index.html
      Their guys even created a ticket for this in DEC 2013. https://issues.apache.org/jira/browse/AVRO-1420
      As of August 2015, Avro doc for C# remains empty. And not much foot print in Internet (C# and Avro I meant).

      References:
      • Official: https://avro.apache.org/docs/1.7.7/index.html
      • Download utility avrogen.exe: http://apache.communilink.net/avro/avro-1.7.7/
      http://www.tutorialspoint.com/avro/serialization_by_generating_class.htm
      http://www.michael-noll.com/blog/2013/03/17/reading-and-writing-avro-files-from-the-command-line/

        Attachments

        1. simpleTest.7z
          3 kB
          Norman Fung JP
        2. avro-csharp-1.7.7.tar.gz
          449 kB
          Norman Fung JP

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              normanfungjp Norman Fung JP
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: