[HUDI-4473] Deprecate EmrFs related configs from DefaultSource

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.12.0
• Component/s: configs
• Assignee: Udit Mehrotra
• Reporter: Sagar Sumit
• Created: 26/Jul/22 03:16
• Updated: 16/Feb/23 05:57
• Resolved: 26/Jul/22 03:19

Description

For context, see here https://github.com/apache/hudi/pull/6154#discussion_r927021771

This Jira is to track the removal of certain EmrFS configs once the issue is properly fixed.

// Revisit EMR Spark and EMRFS incompatibilities, for now disable
spark.conf.set("spark.sql.dataPrefetch.enabled", "false")    spark.sparkContext.hadoopConfiguration.set("fs.s3.metadata.cache.expiration.seconds", "0") 

Issue Links

• duplicates: HUDI-4470 Revert EMR specific configurations hardcoded in Hudi code
• is related to: HUDI-4434 Disable EMRFS and EMR spark related properties