Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5180

Broken FileResultCoder via parseSchema change

Details

    • Bug
    • Status: Resolved
    • P0
    • Resolution: Fixed
    • 2.6.0
    • 2.7.0
    • sdk-java-core

    Description

      Recently this commit

      https://github.com/apache/beam/commit/3fff58c21f94415f3397e185377e36d3df662384

      introduced more strict schema parsing which is breaking the contract between FileResultCoder and FileSystems.matchNewResource().

      Coder takes ResourceId and serialize it via `toString` methods and then relies on filesystem being able to parse it back again. Having strict scheme:// breaks this at least for Hadoop filesystem which use URI for ResourceId and produce toString() in form of `hdfs:/some/path`

      I guess the ResourceIdCoder is suffering the same problem.

      Attachments

        Issue Links

          Activity

            People

              angoenka Ankur Goenka
              JozoVilcek Jozef Vilcek
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m