[HDFS-6634] inotify in HDFS

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.6.0
• Component/s: hdfs-client, namenode, qjm
• Target Version/s: 2.6.0
• Assignee: James Thomas
• Reporter: James Thomas
• Created: 08/Jul/14 00:54
• Updated: 11/Aug/16 08:48
• Resolved: 02/Sep/14 21:24

Description

Design a mechanism for applications like search engines to access the HDFS edit stream.

Issue Links

• duplicates: HDFS-1179 Implement a file change log
• is depended upon by: OOZIE-2179 Use HDFS INotify to track HDFS data dependencies instead of polling
• is duplicated by: HDFS-210 API to get changes to the File System at NN for a window of time
• is duplicated by: HDFS-1742 Provide hooks / callbacks to execute some code based on events happening in HDFS (file / directory creation, opening, closing, etc)
• is duplicated by: HDFS-2760 HDFS notification
• is related to: SPARK-10555 Add INotifyDStream to Spark Streaming
• is related to: HDFS-8940 Support for large-scale multi-tenant inotify service
• is related to: HDFS-8735 Inotify : All events classes should implement toString() API.
• is related to: HDFS-7446 HDFS inotify should have the ability to determine what txid it has read up to
• relates to: HADOOP-7821 Hadoop event notification system
• relates to: HDFS-1742 Provide hooks / callbacks to execute some code based on events happening in HDFS (file / directory creation, opening, closing, etc)
• relates to: HDFS-8933 Inotify : Support Event-based filtering
• relates to: HDFS-10608 Include event for AddBlock in Inotify Event Stream

Sub-Tasks

• Supporting consistent edit log reads when in-progress edit log segments are included: Closed
• Support path-based filtering of inotify events: Open
• Inotify: Support event for OP_TRUNCATE: Resolved