Spark
  1. Spark
  2. SPARK-1642

Upgrade FlumeInputDStream's FlumeReceiver to support FLUME-2083

    Details

    • Type: Improvement Improvement
    • Status: In Progress
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Streaming
    • Labels:
      None
    • Target Version/s:

      Description

      This will add support for SSL encryption between Flume AvroSink and Spark Streaming.

      It is based on FLUME-2083

        Issue Links

          Activity

          Hide
          Ted Malaska added a comment -

          This is an easy change once a1478 is committed

          Show
          Ted Malaska added a comment - This is an easy change once a1478 is committed
          Hide
          Tathagata Das added a comment -

          For future convenience, can you just add a note on what FLUME-2083 is all about? So that FLUME-noobs like me can figure it out more easily

          Show
          Tathagata Das added a comment - For future convenience, can you just add a note on what FLUME-2083 is all about? So that FLUME-noobs like me can figure it out more easily
          Hide
          Ted Malaska added a comment -

          Will do.

          Show
          Ted Malaska added a comment - Will do.
          Hide
          Ted Malaska added a comment -

          OK I'm back. I will start working on this Jira this week.

          Show
          Ted Malaska added a comment - OK I'm back. I will start working on this Jira this week.
          Hide
          Tathagata Das added a comment -

          Ted, is this ready, now that 1478 is getting in (as soon as Jenkins stop misbehaving)?

          Show
          Tathagata Das added a comment - Ted, is this ready, now that 1478 is getting in (as soon as Jenkins stop misbehaving)?
          Hide
          Tathagata Das added a comment -

          1478 is merged. Do you have this ready?

          Show
          Tathagata Das added a comment - 1478 is merged. Do you have this ready?
          Hide
          Ted Malaska added a comment -

          Hey Tdas. Yes now that SPARK-1478 is done I will do this today. I will have it soon.

          Show
          Ted Malaska added a comment - Hey Tdas. Yes now that SPARK-1478 is done I will do this today. I will have it soon.
          Hide
          Ted Malaska added a comment -

          OK I got it working and all the test passed. I'll clean it up and do a pull request tomorrow.

          Thanks again for your help

          Show
          Ted Malaska added a comment - OK I got it working and all the test passed. I'll clean it up and do a pull request tomorrow. Thanks again for your help
          Hide
          Ted Malaska added a comment -

          OK I finished testing and cleaning up. Its ready for a review.

          I even ordered all the imports by ABC order.

          Here is the link
          https://github.com/apache/spark/pull/1386

          Going to start 2447 now.

          Thanks again.

          Show
          Ted Malaska added a comment - OK I finished testing and cleaning up. Its ready for a review. I even ordered all the imports by ABC order. Here is the link https://github.com/apache/spark/pull/1386 Going to start 2447 now. Thanks again.
          Hide
          Ted Malaska added a comment -

          Are there any changes needed here?

          Show
          Ted Malaska added a comment - Are there any changes needed here?

            People

            • Assignee:
              Ted Malaska
              Reporter:
              Ted Malaska
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development