Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1553

Extensive logging of C++ application can slow down task by an order of magnitude

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.13.0
    • 0.14.0
    • None
    • None

    Description

      We observed that extensive logging (due to some configuration mistake) of a c++ application using the pipes interface can slow down the task by an order of magnitude. During that time disk usage was not high, with no abnormal memory usage, and basically idle CPU.

      Attachments

        1. new-log-3.patch
          58 kB
          Owen O'Malley
        2. new-log-2.patch
          56 kB
          Owen O'Malley

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6962 Remove unused settings from log4j.properties

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          incorporates

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-1529 user logs do not seem to be flushed

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-1602 TaskLog.java buffers entire task userlog in memory

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              omalley Owen O'Malley
              ckunz Christian Kunz
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: