Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32163

Nested pruning should still work for nested column extractors of attributes with cosmetic variations

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.1, 3.1.0
    • Component/s: SQL
    • Labels:
      None

      Description

      If the expressions extracting nested fields have cosmetic variations like qualifier difference, currently nested column pruning cannot work well.

      For example, two attributes which are semantically the same, are referred in a query, but the nested column extractors of them are treated differently when we deal with nested column pruning.

        Attachments

          Activity

            People

            • Assignee:
              viirya L. C. Hsieh
              Reporter:
              viirya L. C. Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: