Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8865

FileIO's Javadoc is outdated: TypeDescriptors.KVs and unhandled IOException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sdk-java-core
    • Labels:
      None

      Description

      The doc says

      However, I had to modify the code to below:

          p.apply("ReadLines",
              FileIO.match().filepattern("hdfs:///foo/bar/*.gz"))
              .apply(FileIO.readMatches().withCompression(GZIP))
              .apply(MapElements
                  // uses imports from TypeDescriptors
                  .into(kvs(strings(), strings()))
                  .via((ReadableFile f) -> {
                    try {
                      return KV.of(
                          f.getMetadata().resourceId().toString(), f.readFullyAsUTF8String());
                    } catch (IOException ex) {
                      throw new RuntimeException("Failed to read the file", ex);
                    }
                   }))
      

        Attachments

        1. v1YuuuT2o2i.png
          201 kB
          Tomo Suzuki

          Issue Links

            Activity

              People

              • Assignee:
                suztomo Tomo Suzuki
                Reporter:
                suztomo Tomo Suzuki
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m