Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1917

[parquet-proto] default values are stored in oneOf fields that aren't set

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.12.0
    • 1.12.0
    • parquet-protobuf
    • None

    Description

      SCHEMA
      --------

      message Person {
        int32 foo = 1;
        oneof optional_bar {
          int32 bar_int = 200;
          int32 bar_int2 = 201;
          string bar_string = 300;
        }
      }

       
      CODE
      --------
      I set values for foo and bar_string
       

      for (int i = 0; i < 3; i += 1) {
                      com.etsy.grpcparquet.Person message = Person.newBuilder()
                              .setFoo(i)
                              .setBarString("hello world")
                              .build();
                      message.writeDelimitedTo(out);
                  }

      And then I write the protobuf file out to parquet.
       
      RESULT
      -----------

      $ parquet-tools show example.parquet                                                                                                                        
      +-------+-----------+------------+--------------+
      |   foo |   bar_int |   bar_int2 | bar_string   |
      |-------+-----------+------------+--------------|
      |     0 |         0 |          0 | hello world  |
      |     1 |         0 |          0 | hello world  |
      |     2 |         0 |          0 | hello world  |
      +-------+-----------+------------+--------------+

       
      bar_int and bar_int2 should be EMPTY for all three rows since only bar_string is set in the oneof.  0 is the default value for int, but it should not be stored.

      Attachments

        Activity

          People

            dossett Aaron Blake Niskode-Dossett
            dossett Aaron Blake Niskode-Dossett
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: