Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
Description
When writing logs, they are often partitioned into a hierarchical directory structure. This change will allow spark streaming to monitor all sub-directories of a parent directory to find new files as they are added.
Attachments
Issue Links
- duplicates
-
SPARK-3586 Support nested directories in Spark Streaming
- Resolved
- links to