Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-2910

Random untion(__, null) data doesn't conform to schema

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Not A Problem
    • None
    • None
    • tools
    • None

    Description

      The random Avro data generator tool doesn't generate data that conforms to a schema containing nullable fields.

      For example, the following test.avdl can be used to define a schema for single `test` record, where some of the fields are nullable based on `union`. There doesn't seem to be a difference between setting null first or last.

       

      Using the `idl2schemata` tool, the attached .avdl file generates the attached (.avsc) schema

       

      Using the `random` tool and piping the result into `tojson`, you can see that the generated data (attached sample.json) doesn't conform to the schema:

       

      $ avro-tools random --count 10 --schema-file test.avsc - | avro-tools tojson --reader-schema-file test.avsc - > sample.json

       

      Note how the non-null random values are unnecessarily nested in the following examples, where the generated value is nested within an object with the data type as a key (e.g. "null_num2":{"int":1517937645}` instead of `"null_num2":1517937645)

       

       

      Attachments

        1. sample.json
          2 kB
          Chris Coutinho
        2. test.avdl
          0.3 kB
          Chris Coutinho
        3. test.avsc
          0.4 kB
          Chris Coutinho

        Activity

          People

            Unassigned Unassigned
            cbcoutinho Chris Coutinho
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: