Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-47746

Use column ordinals instead of prefix ordering columns in the range scan encoder

    XMLWordPrintableJSON

Details

    Description

      Currently, the State V2 implementations do projections in their state managers, and then provide some prefix (ordering) columns to the RocksDBStateEncoder. However, we can avoid doing extra projection by just reading the ordinals we need, in the order we need, in the state encoder.

      Attachments

        Issue Links

          Activity

            People

              neilramaswamy Neil Ramaswamy
              neilramaswamy Neil Ramaswamy
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: