[METRON-965] In the case where we specify the syncpolicy in the HDFS Writer, we do not properly clone and end up syncing for every record

• Type: Bug
• Status: Done
• Priority: Major
• Resolution: Done
• Fix Version/s: 0.4.0
• Assignee: Casey Stella
• Reporter: Casey Stella
• Created: 19/May/17 01:15
• Updated: 27/Jun/17 09:12
• Resolved: 27/Jun/17 09:12

Description

Right now the SyncPolicy works as follows:

• If you specify the sync policy in the flux file, it will use that policy to determine when to sync the HDFS Writer.
• If you do not, then it will create a count sync policy based on the batch size.

In the first case, we are not creating the sync policy properly and end up sync'ing per record, which has performance impact and is incorrect.

Issue Links

• links to: GitHub Pull Request #596