Uploaded image for project: 'Traffic Server'
  1. Traffic Server
  2. TS-2145

Add metrics for log collation, e.g. messages sent and messages received

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.1.2
    • Logging
    • None

    Description

      Without accurate stats, we don't known whether the log is lost.

      This coming patch will be used to verify:
      1) At log client side, how many logs genterated from HttpSM, how many logs sent to LogServer.
      2) At log server side, how many logs received from LogClient, how many logs written to LogServers's disk.

      ==
      I have attached a patch. Let's show an example for the metrics collected by this patch:

      In this example, there are two machines: one log-client and one log-server.

      1) Clear old stats data in both client and server machines:
      $ /etc/init.d/trafficserver stop
      $ rm -rf /var/run/trafficserver/*.snap
      
      2) Make logs_xml.config in log server empty.
      
      3) Make logs_xml.config in log client looks like:
      <LogObject>
        <Format = "Cdn_access_log"/>
        <CollationHosts = "<log-server-ip>:8085"/>
        <Filename = "cdn_access"/>
        <Protocols = "http"/>
        <Filters = "only_get"/>
      </LogObject>
      
      2) Start log server (with collation mode == 1, logging_enable = 3, squid_log_enabled = 0)
      [log-server]$ /etc/init.d/trafficserver start
      
      3) Start log client (with collation mode = 0, logging_enable = 3, squid_log_enabled = 0)
      [log-server]$ /etc/init.d/trafficserver start
      
      4) Use jtest do some request for log client.
      
      5) Stop log client and for a while (so that all data in network have been sent to log-server).
      
      6) Log client's metrics for logging:
      [log-client]# lynx -dump http://localhost:8080/stat/ | grep process.log
      proxy.process.log.event_log_error=0
      proxy.process.log.event_log_error_skip=0
      proxy.process.log.event_log_error_aggr=0
      proxy.process.log.event_log_error_fail=0
      proxy.process.log.event_log_access=4990275
      proxy.process.log.event_log_access_skip=0
      proxy.process.log.event_log_access_aggr=0
      proxy.process.log.event_log_access_fail=0
      proxy.process.log.num_sent_to_network=4990275
      proxy.process.log.num_received_from_network=0
      proxy.process.log.num_flush_to_disk=0
      proxy.process.log.bytes_sent_to_network=719149632
      proxy.process.log.bytes_received_from_network=0
      proxy.process.log.bytes_flush_to_disk=0
      proxy.process.log.bytes_written_to_disk=0
      proxy.process.log.log_files_open=0
      proxy.process.log.log_files_space_used=5330
      
      7) Log server's metrics for logging:
      [log-server]# lynx -dump http://localhost:8080/stat/ | grep process.log
      proxy.process.log.event_log_error=0
      proxy.process.log.event_log_error_skip=0
      proxy.process.log.event_log_error_aggr=0
      proxy.process.log.event_log_error_fail=0
      proxy.process.log.event_log_access=0
      proxy.process.log.event_log_access_skip=20
      proxy.process.log.event_log_access_aggr=0
      proxy.process.log.event_log_access_fail=0
      proxy.process.log.num_sent_to_network=0
      proxy.process.log.num_received_from_network=4990275
      proxy.process.log.num_flush_to_disk=4990275
      proxy.process.log.bytes_sent_to_network=0
      proxy.process.log.bytes_received_from_network=719149632
      proxy.process.log.bytes_flush_to_disk=485256906
      proxy.process.log.bytes_written_to_disk=485256906
      proxy.process.log.log_files_open=1
      proxy.process.log.log_files_space_used=85215225070
      

      Attachments

        Activity

          People

            yunkai Yunkai Zhang
            yunkai Yunkai Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: