|
Graham Dumpleton
grahamd at dscpl.com.au
Sun Oct 17 16:54:08 EDT 2004
On 17/10/2004, at 9:32 AM, Graham Dumpleton wrote:
> I knew there was an issue with apache.py as well, with there being no
> locking,
> but didn't mention it in my email with the fix as I wanted to check
> that that
> was the source of the other problem I was seeing and make sure I came
> up with
> a proper fix.
Turns out I had made the fix to the import_module() method as the first
thing I
did, looks like I forgot. This is actually the way I uncovered that
there was still
a further problem and thus the bug with creation of multiple
interpreters. I spent
a good few hours tracking the problem done and then coming up with a
fix. Drove me
nuts for a while as I knew exactly what I had to do to fix it but the
result would
just hang. Finally realised the locking had to be done outside of the
GIL locking
and not inside. Putting it inside later resulted in deadlock.
Anyway, the fix to apache.py I am running with at the moment is:
try:
from threading import Lock
except:
class Lock:
def acquire(self): pass
def release(self): pass
_lock = Lock()
...
def import_module(module_name, autoreload=1, log=0, path=None):
_lock.acquire()
try:
return _unsafe_import_module(module_name, \
autoreload=autoreload,log=log,path=path)
finally:
_lock.release()
def _unsafe_import_module(module_name, autoreload=1, log=0, path=None):
"""
Get the module to handle the request. If
autoreload is on, then the module will be reloaded
if it has changed since the last import.
"""
...
Ie., create the lock in a way that will work if no threading or using
Python 2.2
and then rename existing import_module() and wrap that in safe locking
version.
Have done away with the funny check against reloading as doesn't seem
necessary
because of the way apache.py is initially loaded. Theoretically it
could get
screwed if someone did an import_module() call for "mod_python.apache"
but that
is unlikely.
This obviously is to be used in conjunction with the mod_python.c fix
already
posted.
I also worked out what the other strange thing that was happening and
which had
me worried. Namely, even though I had fixed it so that only one
interpreter
was being created per process, was still seeing the apache.py module
get loaded
more than once. Turned out to be that there were still two interpreters
being
created. These were the one for the machine host and then later on the
one
for the "main_interpreter". If PythonInterpreter was set to
"main_interpreter"
then only saw apache.py being loaded once as one would expect.
I am happy now that everything is working okay with these two changes
and I
believe I can safely release another version of Vampire which works
properly
in multithreaded environment. I will just need to include details of
these bugs
so people are aware of them and can fix them if they want to.
Next question is whether or not if I write a reimplemented
import_module() method
which fixes the mod_python.publisher single module name instance
problem and which
can optionally reload into a fresh module to avoid problems with code
being changed
from underneath a separate executing thread, whether anyone is
interested. I may
have to provide actual examples of the latter problem, as not sure
people may
have been convinced of my previous description of the problem.
The other option I am investigating is how to integrate
mod_python.publisher style
functionality into the structure of Vampire. This would avoid
suggesting that
import_module() be rewritten as have already solved the problems in the
Vampire
module loading mechanism. The only thing is that I am not too keen on
some aspects
of how mod_python.publisher works so have to come up with a mechanism
for doing it
which sits right with my personal view on things. :-)
--
Graham Dumpleton (grahamd at dscpl.com.au)
|