Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18498

Clean up HDFSMetadataLog API for better testing

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.1
    • 2.1.0
    • SQL, Structured Streaming
    • Important

    Description

      HDFSMetadataLog current conflates metadata log serialization and file writes. The goal is to separate these two steps to enable more thorough testing.

      Attachments

        Activity

          People

            tcondie Tyson Condie
            tcondie Tyson Condie
            Michael Armbrust Michael Armbrust
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 168h
                168h
                Remaining:
                Remaining Estimate - 168h
                168h
                Logged:
                Time Spent - Not Specified
                Not Specified