Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16684

StreamingFileSink builder does not work with Scala

    XMLWordPrintableJSON

    Details

      Description

      The StreamingFileSink builders don't work with Scala as they lose with every with* method more type information. For example, the following example does not compile:

      val sink = StreamingFileSink.forRowFormat(new Path("s3a://123"), new Encoder[String] {
            override def encode(element: String, stream: OutputStream): Unit = ???
          }).withRollingPolicy(
            DefaultRollingPolicy.builder()
              .withRolloverInterval(TimeUnit.MINUTES.toMinutes(5))
              .withInactivityInterval(TimeUnit.MINUTES.toMinutes(5))
              .withMaxPartSize(128 * 1024 * 1024)
              .build()
          ).withBucketAssigner(
            new BucketAssigner[String, String] {
              override def getBucketId(element: String, context: BucketAssigner.Context): String = ???
              override def getSerializer: SimpleVersionedSerializer[String] = ???
            }
          ).build();
      

      The problem seems to be that Scala does type inference slightly differently than Java. I believe that the unspecified RowFormatBuilder type cannot be properly resolved.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                trohrmann Till Rohrmann
                Reporter:
                trohrmann Till Rohrmann
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m