Log4net
  1. Log4net
  2. LOG4NET-288

ADO.Net Appender is not writing all the records to the table in a high load/user scenario.It stops writing to the table untill i restart my app servers.Also sometimes, it drops few records and writes the rest of the records.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Won't Fix
    • Affects Version/s: 1.2.9
    • Fix Version/s: 1.2 Maintenance Release
    • Component/s: Appenders
    • Labels:
      None
    • Environment:
      Windows 2008 server , IIS7.0 web site, Sql Server 2008 DB, ASP.net 2.0, .Net 3.0

      Description

      We are using ADO.net appender to write audit records to the a table. Being an Audit table, all the actions performed by the user are recorded to this table along with the User's information like username,account information etc. In production environment we have close to 900-1000 users CONCURRENTLY accessing the system Monday - Sunday. Since every action of each user if tracked to Audit table, we have close to 50,000-200,000 records created every dya in this table.

      The problem is, in between, the ADO.Net appender is not writing all the records to the table. It drops a few records and writes the rest. And sometimes(happenned 3-4 times in production), it just stops writing to the table, untill we restart the app servers( IISRESET command). So basically 2 problems.

      Currently in PRODUCTION we haven't set reconnectOnError = "true" in the web.config file. We will be setting this in our next PROD build. Will this solve the 2nd Issue(just stops writing to table)?.

      We will enable internal debugging in the config file as well in the next PROD build. Will this let us know the exact reason why we are having these issues?

      Has this sort of problem been reported earlier?

      This is a highly critical issue. If we do not get answers to these quickly we may have to look at other options, which i personally don't want to do.

        Activity

        Hide
        Stefan Bodewig added a comment -

        I assume it is too late for you anyway.

        You likely would have a better chance of getting responses from the user mailing list as way more people monitor the list (other users who may have encountered the same problem) than the issue tracker.

        Have you tried 1.2.10 instead of 1.2.9?

        Show
        Stefan Bodewig added a comment - I assume it is too late for you anyway. You likely would have a better chance of getting responses from the user mailing list as way more people monitor the list (other users who may have encountered the same problem) than the issue tracker. Have you tried 1.2.10 instead of 1.2.9?
        Hide
        Tom Tang added a comment -

        By the system loading you described, you lost the records could be caused by SQL execution timeout.
        Due to there's no way to extend the SqlCommand.ExecutionTimeout property in your code and its configuration, if the thread was blocked by any other logging thread more than 30 sec, it would get "SqlException:Execution timeout" and drop these message.
        By that, your synchronized business code thread would be hanged by log4net till it got the exception returned.

        So I don't suggest you to think the way how to extend the execution timeout.

        Log4net is not a guarantee tool sot audit log the kind of serious usage, you better find the other tool or implement by your own.

        "Log" is not as the same severity level as "Audit"

        If your audit need to be 100% accurate, it will be to be involve a transaction scope, means the auditing failure shall cause your business logic rollback...That's much more than just a logging.

        Best regards

        Show
        Tom Tang added a comment - By the system loading you described, you lost the records could be caused by SQL execution timeout. Due to there's no way to extend the SqlCommand.ExecutionTimeout property in your code and its configuration, if the thread was blocked by any other logging thread more than 30 sec, it would get "SqlException:Execution timeout" and drop these message. By that, your synchronized business code thread would be hanged by log4net till it got the exception returned. So I don't suggest you to think the way how to extend the execution timeout. Log4net is not a guarantee tool sot audit log the kind of serious usage, you better find the other tool or implement by your own. "Log" is not as the same severity level as "Audit" If your audit need to be 100% accurate, it will be to be involve a transaction scope, means the auditing failure shall cause your business logic rollback...That's much more than just a logging. Best regards
        Hide
        Herbert Yu added a comment -

        If I'm correct, this is the issue in AppenderSkeleton class with m_recursiveGuard.
        To avoid data lose as well as prevent re-entry, not sure need to use a light-weight in-memory message queue.

        Show
        Herbert Yu added a comment - If I'm correct, this is the issue in AppenderSkeleton class with m_recursiveGuard. To avoid data lose as well as prevent re-entry, not sure need to use a light-weight in-memory message queue.
        Hide
        Dominik Psenner added a comment -

        The m_recursiveGuard property ensures that log4net discards log events as soon as the recursion is detected if it is configured badly by linking one appender into another. AFAIK it has nothing to do with this issue. Anyway, thanks for bumping this issue.

        Show
        Dominik Psenner added a comment - The m_recursiveGuard property ensures that log4net discards log events as soon as the recursion is detected if it is configured badly by linking one appender into another. AFAIK it has nothing to do with this issue. Anyway, thanks for bumping this issue.
        Hide
        Dominik Psenner added a comment -

        I'm closing this issue now since auditing is not a usecase for log4net. Feel free to reopen the issue if you disagree with this decision.

        Show
        Dominik Psenner added a comment - I'm closing this issue now since auditing is not a usecase for log4net. Feel free to reopen the issue if you disagree with this decision.

          People

          • Assignee:
            Unassigned
            Reporter:
            Chetan Appannagari
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development