Uploaded image for project: 'Apache Gobblin'
  1. Apache Gobblin
  2. GOBBLIN-742

Implement a FileSystem based JobConfigurationManager

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.15.0
    • Fix Version/s: 0.15.0
    • Component/s: gobblin-cluster
    • Labels:
      None

      Description

      This task aims to implement a FileSystem based JobConfigurationManager. The manager has an underlying SpecConsumer which reads specs from a config specified path on HDFS and persists them to a JobCatalog. On successful consumption of the spec by the manager , it is deleted from the original path.  So on the next fetch, only new specs will be picked up by the JobConfigurationManager.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hutran Hung Tran
                Reporter:
                sv2000 Sudarshan Vasudevan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m