Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-138

System that places specified file contents onto stream

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      RHELinux 2.6.18-371.4.1.el5

      Description

      A fairly straightforward Samza System that reads from a specified file, and places that file's contents onto a SystemStreamPartition for use as input for a StreamTask.

      Roughly based off how the hello-samza example project's WikipediaSystem works (more the SystemConsumerFactory rather than SystemConsumer class).

      Probably needs a bit of work, but basic functionality works as intended. Hopefully useful to some, either as a functioning system or as a base for a more robust and functionally-promising system that you wish to implement.

      Some suggested improvements (not yet implemented):

      • handle reading from multiple files (suggested alternative input specification- point 2)
      • use of filepos for IncomingMessageEnvelope offset (more info here
      • come up with a reasonable bounded queue threshold (the value of 100 was arbitrary, as I was unsure of a reasonable value here)
      • better handling for the exceptions encountered (I wasn't 100% sure about some of them)

        Attachments

        1. SAMZA-138.4.patch
          27 kB
          Chris Riccomini
        2. SAMZA-138.3.patch
          7 kB
          Chris Riccomini
        3. SAMZA-138.2.patch
          27 kB
          Yan Fang
        4. SAMZA-138.1.patch
          24 kB
          Yan Fang
        5. SAMZA-138.patch
          10 kB
          Yan Fang
        6. FileReaderSystemFactory.java
          3 kB
          Jonathan Poltak Samosir
        7. FileReaderConsumer.java
          5 kB
          Jonathan Poltak Samosir

          Activity

            People

            • Assignee:
              closeuris Yan Fang
              Reporter:
              poltak Jonathan Poltak Samosir
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: