Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4545

[C#] Extend Append/AppendRange in BinaryArray to support building rows

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: C#
    • Labels:
      None

      Description

      This is a proposal to extend BinaryArray to provide the ability to call Append/AppendRange to grow list elements when building arrays, and to expose values in ArrowBuffer.Builder<T> through a property to facilitate algorithms that require introspecting the buffer data (sorting, filtering) when building an array.

      Example:

      var builder = new BinaryArray.Builder()
           .Append(10, false)     
           .Append(20, false)
           .Mark();
      builder.Append(builder.Values[0], true);
      var array = builder.Build();
      
      // General idea:
      //
      // 1. Append byte (10) to current element (0)
      // 2. Append byte (20) to current element (0)
      // 3. Mark end of the row 
      // 4. Append byte (10) to current element (1)
      
      // Constructs a binary array with 2 elements:
      //
      // [0] 10, 20
      // [1] 10

       

      This proposed change would add the concept of "current element" to the builder, which in the specification are separated by recording the value offset. Append(true) appends one or more bytes to the current element and then marks the element as completed. Append(false) appends one or more bytes to the current element; Mark is required to signal to the builder that the current element is complete.

      Alternative Concept

      An alternative concept could be to add a new overload for Append that accepts another builder type with the purpose of building individual list elements. This would keep symmetry with the primitive array types in that the growth functions always append a single element to the array (where an element is an entire list, for list arrays). Essentially the additional builder type's purpose is to build a list of a specific primitive. 

      Example:

      var builder = new BinaryArray.Builder()
          .Append(e => e.Append(10).Append(20))
          .Append(e => e.Append(30))
          .Build();
      
      // Constructs a binary array with 2 elements:
      //
      // [0] 10, 20
      // [1] 30

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              cshutchinson Chris Hutchinson
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: