Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-1015 Audit all getAllPartitionPaths() calls and keep em out of fast path
  3. HUDI-2201

Audi all flink classes and route all FileSystem list and getStatus calls to metadata table

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Common Core
    • None

    Description

       

      grep -irl "fs.listStatus" hudi-*/* | grep -v Test
      hudi-flink/src/main/java/org/apache/hudi/table/format/FilePathUtils.java
      hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java
      
      

       

      grep -irl "fs.getFileStatus" hudi-*/* | grep -v Test
      hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/FlinkMergeHandle.java
      hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/profile/WriteProfiles.java
      hudi-flink/src/main/java/org/apache/hudi/table/format/FilePathUtils.java
      hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java
      

      Some of them are valid ones which can't be routed. But just dumping all matches here. 

      Attachments

        Activity

          People

            danny0405 Danny Chen
            shivnarayan sivabalan narayanan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: