Implement req.add_output_filter().

• Key: MODPYTHON-103
• Type: New Feature
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Graham Dumpleton
• Reporter: Graham Dumpleton
• Component/s: core
• Fix Version/s: 3.3.1

Description

Add new member function to request object called "add_output_filter()". This would be a wrapper around the function "ap_add_output_filter()" and allow previously defined filters to be attached to the current request such that output can be filtered through them. For example:

  req.add_output_filter("INCLUDES")

It would probably be necessary for any such call to be done prior to the first output being generated from the request handler.

In addition to this member function, it may be necessary to also provide another member function called something like "req.add_python_output_filter()". This would be called something like:

  req.add_python_output_filter("module_name::filter_name",path)

Ie., like "req.add_handler()" but no first argument for phase.

This method would allow a specific Python filter handler function to be specified. This would be equivalent to using the PythonOutputFilter directive to first name a mod_python based filter handler function and then adding it as an output filter.

  # Main Apache config.

  PythonOutputFilter module_name::filter_name MYFILTER

  # Handler code.

  req.add_output_filter("MYFILTER")

Note that the PythonOutputFilter directive can only be used in the main Apache configuration file, it cannot be used in a .htaccess file. Whether it could be made to work in a .htaccess file in some way needs to be investigated. In mod_perl their equivlent seems to allow it.

Comments

[Graham Dumpleton]

Correspondingly, it should be possible to add a req.add_input_filter() function as well. This is because mod_python input filters only get triggered for the body of the request. This will only occur though when a call is made to req.read(). As a consequence, a request handler should probably be able to register input filters as well, provided it does it before the first req.read() of any request body content.

Worth noting though is that if the request doesn't specify a content length, calling req.read() will not cause the input filter to be triggered at that point. The input filter will however be triggered at some point later on (not sure where), by Apache so that the input filter will detect end of input and close off the bucket brigade properly.

[Graham Dumpleton]

Attaching file "grahamd_20060108_1_requestobject.c.diff".

This is the easy bit of the suggested changes. It allows a pre registered filter to be added into either the input or output filter chain.

For example, one could do:

  from mod_python import apache

  def handler(req):
    req.add_output_filter("CONTENT_LENGTH")
    req.content_type = 'text/plain'
    req.write(str(globals()),0)
    req.write('\n',0)
    req.write(str(locals()),0)
    req.write('\n',0)
    return apache.OK

Provides that no req.write() flushes the output, the "CONTENT_LENGTH" output filter provided by Apache will add a valid content length header to the response automatically.

Alternatively, if you have mod_deflate built into Apache, you could use:

  from mod_python import apache

  def handler(req):
    req.add_output_filter("DEFLATE")
    req.headers_in["Accept-Encoding"] = "gzip"
    req.content_type = 'text/plain'
    req.write(str(globals()),0)
    req.write('\n',0)
    req.write(str(locals()),0)
    req.write('\n',0)
    return apache.OK

This will have the effect or forcing the output sent to the browser to be compressed, ie., response header would look like:

  HTTP/1.1 200 OK
  Date: Sun, 08 Jan 2006 02:04:37 GMT
  Server: Apache/2.0.55 (Unix) mod_python/3.2.6-dev-20051229 Python/2.3
  Vary: Accept-Encoding
  Content-Encoding: gzip
  Content-Length: 1917
  Connection: close
  Content-Type: text/plain

The content would be compressed accordingly. Not sure if the Vary header would be an issue in this case, but no way to tell mod_deflate not to include it.

Now on to the next bit, allow filters to be registered from mod_python using req.register_input_filter() and req.register_output_filter(). The registration only being active for the current request. :-)

You can ignore "grahamd_20060108_1_requestobject.c.diff" now.

I have superseded this with "grahamd_20060108_2_multiple.diff". This is done as a context diff as well to make it easier to see where changes need to go.

This later patch includes support for dynamic registration of new Python filters from a request handler. For example:

  from mod_python import apache
  
  def uppercase(filter):
    apache.log_error("uppercase")
    apache.log_error(filter.name)
    apache.log_error(filter.handler)
    apache.log_error("is_input "+repr(filter.is_input))
    s = filter.read()
    while s:
      filter.write(s.upper())
      s = filter.read()
    if s is None:
      apache.log_error("close")
      filter.close()
    apache.log_error("end")

  def handler(req):
    req.register_output_filter("UPPERCASE","example::uppercase")
    req.add_input_filter("UPPERCASE")
    req.add_input_filter("CONTENT_LENGTH")
    
    req.content_type = 'text/plain'
    req.write('hello',0)
    return apache.OK

All up this adds the following methods to the request object:

  register_input_filter()
  register_output_filter()
  add_input_filter()
  add_output_filter()

The 'add' methods can be used to add to the filter stack either predefined filters, or those registered from the handler. Those registered from the handler only have a registration lifetime of that request. Multiple calls can be made to the 'add' methods to chain more than one filter.

[Graham Dumpleton]

Whoops, that last example should have read:

  def handler(req):
    req.register_output_filter("UPPERCASE","example::uppercase")
    req.add_output_filter("UPPERCASE")
    req.add_output_filter("CONTENT_LENGTH")
     
    req.content_type = 'text/plain'
    req.write('hello',0)
    return apache.OK

[Graham Dumpleton]

Reopened because using PythonInterpPerDirective in conjunction with dynamically registered filters causes mod_python to crash. Problem code is in select_interp_name().

            py_handler *fh;
                 
            if (fname) {
                if (is_input) {
                    fh = (py_handler *)apr_hash_get(conf->in_filters, fname,
                                                           APR_HASH_KEY_STRING);
                } else {
                    fh = (py_handler *)apr_hash_get(conf->out_filters, fname,
                                                           APR_HASH_KEY_STRING);
                }
                s = fh->directory;
            }
            else {
                s = hle->directory;
            }

For a dynamically registered filter, fname is "mod_python" and the context provided with the filter handler is used to get details of dynamically registered filter. This information isn't available or used by select_interp_name() which assumes that filter is part of globally registered filters. It will not find the filter and so "fh" is NULL and accessing directory attribute causes a crash.

[Graham Dumpleton]

Crashing when PythonInterpPerDirective used now fixed. Note that this same change also fixed a similar crash if SSI and #python tag was used when PythonInterpPerDirective was specified.