Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35743 Improve Parquet vectorized reader
  3. SPARK-35867

Enable vectorized read for VectorizedPlainValuesReader.readBooleans

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.2.0
    • 3.3.0
    • SQL
    • None

    Description

      Currently we decode PLAIN encoded booleans as follow:

        public final void readBooleans(int total, WritableColumnVector c, int rowId) {
          // TODO: properly vectorize this
          for (int i = 0; i < total; i++) {
            c.putBoolean(rowId + i, readBoolean());
          }
        }
      

      Ideally we should vectorize this.

      Attachments

        Activity

          People

            kazuyukitanimura Kazuyuki Tanimura
            csun Chao Sun
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: