This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Trap OSError when calling RotatingFileHandler.doRollover
Type: enhancement Stage:
Components: Library (Lib) Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: groodude, hewittr, nnorwitz, vinay.sajip
Priority: normal Keywords:

Created on 2004-06-24 20:43 by hewittr, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (5)
msg21293 - (view) Author: Richard T. Hewitt (hewittr) Date: 2004-06-24 20:43
I use the RotatingFileHandler in most of my scripts.  The 
script will crash if the RotatingFileHandler encounters a 
locked log file.

I would like to see something like:

    def emit(self, record):
        """
        Emit a record.

        Output the record to the file, catering for rollover 
as described
        in __init__().
        """
        if self.maxBytes > 0:                   # are we rolling 
over?
            msg = "%s\n" % self.format(record)
            self.stream.seek(0, 2)  #due to non-posix-
compliant Windows feature
            if self.stream.tell() + len(msg) >= self.maxBytes:
                try:
                    self.doRollover()
                except Exception:
                    logging.FileHandler.emit(self, 'Failed to 
doRollover.')
        logging.FileHandler.emit(self, record)

My version of Python (2.3.2) had the wrong docstring as 
well, referring to a non-existent setRollover.
msg21294 - (view) Author: David London (groodude) Date: 2004-06-29 21:34
Logged In: YES 
user_id=975020

Richard,
I have also had the "joy" of encountering this bug (on a
windows 2000 machine), although I am still baffled at what
could be locking the file.

Our testing did not reveal this bug and now we have some
code in a production environment that is causing me grief,
so I had to come up with a solution quickly and I though
that I might share it with you (and others).

I don't believe that your solution will solve the problem.
In doRollover the stream is closed and then the files are
manipulated. Since this is where the errors will occur
simply calling the emit method again will only cause more
exceptions to be generated. Also, since the stream is never
opened again, you would never get any log messages until you
restarted.  Sorry, after looking a bit more it seems like
you will get tracebacks printed to the standard error (where
ever you have designated that to be).

What I am doing is replacing my version of doRollover with
this version;
    def doRollover(self):
        """
        Do a rollover, as described in __init__().
        """

        self.stream.close()
        openMode = "w"
        if self.backupCount > 0:
            try:
                for i in range(self.backupCount - 1, 0, -1):
                    sfn = "%s.%d" % (self.baseFilename, i)
                    dfn = "%s.%d" % (self.baseFilename, i + 1)
                    if os.path.exists(sfn):
                        #print "%s -> %s" % (sfn, dfn)
                        if os.path.exists(dfn):
                            os.remove(dfn)
                        os.rename(sfn, dfn)
                dfn = self.baseFilename + ".1"
                if os.path.exists(dfn):
                    os.remove(dfn)
                os.rename(self.baseFilename, dfn)
                #print "%s -> %s" % (self.baseFilename, dfn)
            except:
                ## There was some problem with the file
manipulationg above.
                ## If someone has some time, maybe they can
figure out what errors
                ## can occur and the best way to deal with
all of them.
                ## For my purposes, I'm just going to try
and re-open
                ## the original file.
                openMode = "a"
        try:
            self.stream = open(self.baseFilename, openMode)
        except:
            ## There was some problem opening the log file.
            ## Again, someone with more time can look into
all of the possible
            ## errors and figure out how best to handle them.
            ## I'm just going to try and open a 'problem'
log file.
            ## The idea is that this file will exist long
enough for the problems
            ## above to go away and the next time that we
attempt to rollover
            ## it will work as we expect.
            gotError = True
            count = 1
            while gotError:
                try:
                    problemFilename = self.baseFilename +
".pblm." + str(count)
                    self.stream = open(problemFilename,
openMode)
                    gotError = False
                except:
                    count += 1

If my logic works out, this should always open a file for
logging so the logging should never crash and no log
messages will be lost. 
Hopefully you will find this helpful.
msg21295 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2004-07-08 01:39
Logged In: YES 
user_id=33168

Vinay, any ideas?
msg21296 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2004-07-08 10:00
Logged In: YES 
user_id=308438

Easy things first - the docstring has been corrected in a later 
release.

I believe that when an exception occurs during logging, one 
of two things should happen: if raiseExceptions is 0, then the 
exception should be swallowed silently (typically, for 
production environments). This is because logging-related 
errors should not be treated in the same way as application 
errors (it's a pseudo-philosophical point, and not everyone will 
agree). If raiseExceptions is 1 (development environments), 
the exception should be raised.

I think the solution given by David London (groodude) may be 
fine for specific applications, but not sufficiently general to 
be adopted as is. I will check in changes to CVS which will 
call handleError() when an exception occurs during emit(). It's 
best to derive from RotatingFileHandler and put any specific 
handling you want into an overridden handleError().
msg21297 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2004-07-12 09:26
Logged In: YES 
user_id=308438

Added code to call handleError() when an exception is raised 
while emitting.
History
Date User Action Args
2022-04-11 14:56:05adminsetgithub: 40446
2004-06-24 20:43:03hewittrcreate