[NIFI-1924] Create set of WebHDFS processors

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Andre F de Miranda
• Created: 26/May/16 07:22
• Updated: 26/Apr/17 12:47

Description

Create processors to handle,

Supported commands for WebHDFS V1

• GET -> GetWebHDFS
• OPEN
• GETFILESTATUS
• LISTSTATUS
• GETCONTENTSUMMARy
• GETFILECHECKSUM
• GETHOMEDIRECTORY
• GETDELEGATIONTOKEN
• GETDELEGATIONTOKENS
• GETXATTRS
• LISTXATTRS
• CHECKACCESS

• PUT -> PutWebHDFS
• CREATE
• MKDIRS
• CREATESYMLINK
• RENAME
• SETREPLICATION
• SETOWNER
• SETPERMISSION
• SETTIMES
• RENEWDELEGATIONTOKEN
• CANCELDELEGATIONTOKEN
• CREATESNAPSHOT
• RENAMESNAPSHOT
• SETXATTR
• REMOVEXATTR

• POST -> PostWebHDFS
• APPEND
• CONCAT
• TRUNCATE

• DELETE -> DeleteWebHDFS
• DELETE
• DELETESNAPSHOT

operations against WebHDFS as discussed here:

http://mail-archives.apache.org/mod_mbox/nifi-users/201604.mbox/%3CCABtjSZk2KLJJVk%2Byk-ySioPxVUV5SLHcN%2BaWMZs1ARpkfy%2B0Ow%40mail.gmail.com%3E

Required Processor Properties:

• Namenode Host
• WebHDFS port
• WebHDFS API version. Default to v1. Dropdown list of available options. Currently there is only v1 but would be good to have that in place to show our intentions.
• Hadoop configuration files. core-site.xml and hdfs-site.xml. These configurations files should not be required to keep things lite. However they would be required to access a kerberized Hadoop cluster. If these configuration files are present they should take precedence over the properties set in NameNode host and WebHDFS port.

Issue Links

• is duplicated by: NIFI-2945 create a get or get HTTPfs processor (if possible)