[mod_python] Single import at startup

On 15/09/2005, at 5:07 AM, Nick wrote:

> If the import is huge, it can create a significant delay (between the 
> initiation of the request and the fulfillment of that request) on the 
> first call of a handler.  If you have multiple server children, then 
> there is that delay for each child.

There are a couple of problems with doing initialisation of such things
as database connections at time of module import using PythonImport.

The main one is that it only provides one opportunity to have the
initialisation succeed. If the initialisation fails, Apache has to be
restarted. This is more problematic where multiple Apache child
processes exist, eg, prefork, where initialisation might work in some
subprocesses and not others.

Error handling is also a problem in that the initialisation is being
done during the module import. If an exception is raised because of some
error and not dealt with, it effectively causes the module import to
fail in an unknown state, especially if the resource acquisition was not
done as the very last thing. For example, "b" will not exist in the
module if later referenced due to the exception occurring before that
point.

   a = 1
   raise Exception()
   b = 2

The next problem is that in mod_python prior to 3.2, there is no way to
access the server object during import of a module using PythonImport.
As a consequence there is no way to register a server cleanup function
to perform any necessary cleanup and releasing of resources when Apache
is stopped completely or when Apache decides to kill of a child process
because it has reached some request limit.

In mod_python 3.2 this particular problem will not exist as a new
feature has been added which will allow you to do:

   apache.register_cleanup(my_cleanup_function)

Once mod_python 3.2 is available, I would suggest that the best
compromise is that PythonImport be used to attempt any initialisation
that you might want to be done early on, but that this not be relyed
upon. That is, a handler should still later call through some form of
acquisition object to get access to the actual resource handle. If it
wasn't able to be created earlier, a subsequent attempt is thus made to
create when the first request arrives that requires it.

The acquisition object, whether it is initialising at the point of a
PythonImport or a later handler should use "apache.register_cleanup()"
to ensure that any cleanup is done on process shutdown.

You might therefore have the module holding the resource be something
like:

   # ... lock for global data
   # ... actual global data

   def _cleanup():
     ...

   def resource_handle():
     # create resource if necessary and invoke 
apache.register_cleanup(_cleanup)
     return ...

   # Must trigger initialisation last.

   try:
     resource_handle()
   except:
     apache.log_error(...)

In the handler it would call "resource_handle()" and if an error occurs
again it should give back some meaningful error in web page.

Note that if using a multithreaded MPM, because initialisation could be
performed by handlers when a request arrives, the "resource_handle()"
function should use thread locking as appropriate to ensure that any
initialisation is only done once.

Think that is all, I have to run now anyway. :-)

Graham