Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4969

Add binaryRecords support to streaming

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.3.0
    • Component/s: DStreams, PySpark
    • Labels:
      None

      Description

      As of Spark 1.2 there is support for loading fixed length records from flat binary files. This is a useful way to load dense numerical array data into Spark, especially in scientific computing applications.

      We should add support for loading this same file type in Spark Streaming, both in Scala/Java and in Python.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              freeman-lab Jeremy Freeman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: