Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19121

Avoid accessing HDFS frequently in HiveBulkWriterFactory

    XMLWordPrintableJSON

    Details

      Description

      In HadoopPathBasedBulkWriter, getSize will invoke `FileSystem.exists` and `FileSystem.getFileStatus`, but it is invoked per record.

      There will be lots of visits to HDFS, may make HDFS pressure too high.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lzljs3620320 Jingsong Lee
                Reporter:
                lzljs3620320 Jingsong Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: