Uploaded image for project: 'Chukwa (retired)'
  1. Chukwa (retired)
  2. CHUKWA-749

Redefine Solr facets for Hadoop log monitoring

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.7.0
    • None
    • Data Collection
    • None
    • Linux, Java

    Description

      For improving the user experience of analyze big data environment log files, we can use Solr to organize all log files in a logical order. Log files of big data environment are generally divided into application and system categories.

      In Application perspective, the structure of an application look like this:

      Application
        user
        coordinator-job
          date
          workflow
            action
              jobs
                scheduler (app master)
                tasks
                  task-attempts
                    container
                      host
      

      In system perspective,

      System
        cluster
          service
            component
              host
                container
      

      The two perspectives are join by host. If we have Application perspective and System perspective in Solr cores. We can join the activities happened in Application with System using Solr.

      Attachments

        Activity

          People

            eyang Eric Yang
            eyang Eric Yang
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: