Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9020

LengthPrefixUnknownCodersTest to avoid relying on AbstractMap's equality

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.19.0
    • Component/s: io-java-gcp
    • Labels:
      None

      Description

      In an attempt for BEAM-8695 LengthPrefixUnknownCodersTest failed when trying to upgrade google-http-client v1.34.0, because LengthPrefixUnknownCodersTest relies on the equality of CloudObject with Map.

      Class hierarchy:

      CloudObject < GenericJson < GenericData < AbstractMap
      

      It was working fine as long as CloudObject's equality inherits AbstractMap.equality. GenericData did not override equals method in google-http-client v1.28.0 and earlier. The comparison was checking only key and value of a Map.

      assertEquals(
          CloudObjects.asCloudObject(prefixedWindowedValueCoder, null),   // This is a CloudObject
          lengthPrefixedCoderCloudObject);                             // This is a Map
      

      However, with google-http-client v1.29.0 or higher, GenericData has its own equals method (PR#589)  that checks classInfo and thus the comparisons between a CloudObject and a Map always fail.

      Test failures when I tried to upgrade google-http-client 1.34.0 (Jenkins):

      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixUnknownCoders
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixForInstructionOutputNodeWithGrpcNodeSuccessor
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixForLengthPrefixCoder
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixForSideInputInfos
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixParDoInstructionCoder
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixInstructionOutputCoder
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixWriteInstructionCoder
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixAndReplaceUnknownCoder
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixAndReplaceForRunnerNetwork
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixForInstructionOutputNodeWithGrpcNodePredecessor
      org.apache.beam.runners.dataflow.worker.graph.LengthPrefixUnknownCodersTest.testLengthPrefixReadInstructionCoder
      

        Attachments

          Activity

            People

            • Assignee:
              lcwik Luke Cwik
              Reporter:
              suztomo Tomo Suzuki

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 10m
                2h 10m

                  Issue deployment