[SPARK-36525] DS V2 Index Support

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Huaxin Gao
• Created: 16/Aug/21 17:17
• Updated: 12/Dec/22 18:10

Description

Many data sources support index to improvement query performance. In order to take advantage of the index support in data source, the following APIs will be added for working with indexes:

 public interface SupportsIndex extends Table {

  /**
   * Creates an index.
   *
   * @param indexName the name of the index to be created
   * @param indexType the type of the index to be created. If this is not specified, Spark
   *                  will use empty String.
   * @param columns the columns on which index to be created
   * @param columnsProperties the properties of the columns on which index to be created
   * @param properties the properties of the index to be created
   * @throws IndexAlreadyExistsException If the index already exists.
   */
  void createIndex(String indexName,
      String indexType,
      NamedReference[] columns,
      Map<NamedReference, Map<String, String>> columnsProperties,
      Map<String, String> properties)
      throws IndexAlreadyExistsException;

  /**
   * Drops the index with the given name.
   *
   * @param indexName the name of the index to be dropped.
   * @throws NoSuchIndexException If the index does not exist.
   */
  void dropIndex(String indexName) throws NoSuchIndexException;

  /**
   * Checks whether an index exists in this table.
   *
   * @param indexName the name of the index
   * @return true if the index exists, false otherwise
   */
  boolean indexExists(String indexName);

  /**
   * Lists all the indexes in this table.
   */
  TableIndex[] listIndexes();
}

Sub-Tasks

• Add supportsIndex interface: Resolved
• Add Create Index syntax support: Resolved
• Implement createIndex and IndexExists in JDBC (MySQL dialect): Resolved
• Implement dropIndex and listIndexes in JDBC (MySQL dialect): Resolved
• Drop index support: Resolved
• Implement createIndex and IndexExists in JDBC (Postgres dialect): Resolved
• Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect): Resolved
• Implement listIndexes in JDBC (H2 dialect): Resolved
• Rename catalogName to dialectName for JdbcUtils: Resolved