[NIFI-4213] PutHDFS umask not working

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.1.1
• Assignee: Unassigned
• Reporter: William Nouet
• Created: 21/Jul/17 13:31
• Updated: 22/Aug/17 20:18
• Resolved: 22/Aug/17 20:18

Description

The PutHDFS permission umask property is not working. The umask is set when the processor is scheduled to run as per below:

@OnScheduled
public void onScheduled(ProcessContext context) throws Exception {
super.abstractOnScheduled(context);

// Set umask once, to avoid thread safety issues doing it in onTrigger
final PropertyValue umaskProp = context.getProperty(UMASK);
final short dfsUmask;
if (umaskProp.isSet())

{ dfsUmask = Short.parseShort(umaskProp.getValue(), 8); }

else

{ dfsUmask = FsPermission.DEFAULT_UMASK; }

final Configuration conf = getConfiguration();
FsPermission.setUMask(conf, new FsPermission(dfsUmask));
}

However, when the flowfile is being processed, a new set of configuration is loaded:
@Override
public void onTrigger(ProcessContext context, ProcessSession session) throws ProcessException {
final FlowFile flowFile = session.get();
if (flowFile == null)

{ return; }

final FileSystem hdfs = getFileSystem();
final Configuration configuration = getConfiguration();
...
}

This configuration is the one which is going to be used when putting the file to HDFS; hence not grabbing the umask set perviously on onScheduled but only the default one (hdfs-site.xml). Thus, the umask property is irrelevant.

Fix should be easy by externalizing the configuration and grabbing it again in onTrigger or by setting a new hdfsResources in onScheduled.

Issue Links

• is duplicated by: NIFI-4311 PutHDFS not honoring umask property