Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12702

Add an HDFS metrics sink

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.1
    • 2.9.0, 3.0.0-alpha1
    • metrics
    • None
    • Reviewed

    Description

      We need a metrics2 sink that can write metrics to HDFS. The sink should accept as configuration a "directory prefix" and do the following in putMetrics()

      • Get yyyyMMddHH from current timestamp.
      • If HDFS dir "dir prefix" + yyyyMMddHH doesn't exist, create it. Close any currently open file and create a new file called <hostname>.log in the new directory.
      • Write metrics to the current log file.

      Attachments

        1. HADOOP-12702.006.patch
          40 kB
          Daniel Templeton
        2. HADOOP-12702.005.patch
          40 kB
          Daniel Templeton
        3. HADOOP-12702.004.patch
          40 kB
          Daniel Templeton
        4. HADOOP-12702.003.patch
          38 kB
          Daniel Templeton
        5. HADOOP-12702.002.patch
          35 kB
          Daniel Templeton
        6. HADOOP-12702.001.patch
          6 kB
          Daniel Templeton

        Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-9637 Tests for RollingFileSystemSink

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-9782 RollingFileSystemSink should have configurable roll interval

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              templedf Daniel Templeton
              templedf Daniel Templeton
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: