Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17566 Über-jira: S3A Hadoop 3.4 features
  3. HADOOP-17511

Add an Audit plugin point for S3A auditing/context

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.3.1
    • Fix Version/s: None
    • Component/s: None
    • Target Version/s:

      Description

      Add a way for auditing tools to correlate S3 object calls with Hadoop FS API calls.

      Initially just to log/forward to an auditing service.

      Later: let us attach them as parameters in S3 requests, such as opentrace headeers or (my initial idea: http referrer header -where it will get into the log)

      Challenges

      • ensuring the audit span is created for every public entry point. That will have to include those used in s3guard tools, some defacto public APIs
      • and not re-entered for active spans. s3A code must not call back into the FS API points
      • Propagation across worker threads

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 16h 10m
                  16h 10m