Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-40402

StructType level metadata

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.1
    • None
    • Spark Core
    • None

    Description

      Please support StructType level metadata like StructField does. It allows data engineers provide better documentation of data schemas used in pipelines and store this metadata it in catalog. This feature also will simplify table level comment propagation from RDBMS to Spark

      Attachments

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. ARROW-17631 [Java] Propagate table/columns comment into Arrow Schema

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              igor.suhorukov Igor Suhorukov
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: