Uploaded image for project: 'Streams'
  1. Streams
  2. STREAMS-293

allow for missing metadata fields in streams-persist-hdfs

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.2
    • Component/s: None
    • Labels:
      None

      Description

      Currently streams-persist-hdfs writer creates (and reader expects) exactly four columns. this could be made much more flexible without too much effort.

      Update reader to support additional use cases:
      a) file paths containing one json document per line
      b) file paths containing just id and json on each line,
      c) file paths containing id timestamp and json document on each line

      Update writer support
      a) ids only
      b) ids and timestamp only
      c) ids timestamp and json only

        Attachments

          Activity

            People

            • Assignee:
              steveblackmon Steve Blackmon
              Reporter:
              steveblackmon Steve Blackmon
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: