Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5307

Select Hive Processor takes longer time to write into CSV files

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.6.0
    • Fix Version/s: None
    • Component/s: Configuration
    • Labels:
      None

      Description

      Team,

      Current Select Hive SQL processor reads each line from result set and writes into the file. This can be improved to directly write records in bulk to improve the performance. When tried with millions of rows the reading from Hive only takes few milliseconds but writing to file takes several minutes. 

       

      Thanks,

      Tanmay

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tanmaydeshpande Tanmay Deshpande
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: