Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4076 Schema followups
  3. BEAM-4457

Analyze FieldAccessDescriptors and drop fields that are never accessed

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-java-gcp
    • None

    Description

      We can walk backwards through the graph, analyzing which fields are accessed. When we find paths where many fields are never accessed, we can insert a projection transform to drop those fields preemptively. This can save a lot of resources in the case where many fields in the input are never accessed.

      To do this, the FieldAccessDescriptor information must be added to the portability protos. 

      Attachments

        Activity

          People

            Unassigned Unassigned
            reuvenlax Reuven Lax
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated: