Python logging: create log if not exists or open and continue logging if it does

Question

I'm writing some code that uses the python logging system. The idea is that if the LOG doesn't already exist create the log but if it does then get the log and resume logging to that file. Here is my code:

import logging
import os

log_filename='Transactions.log')
if os.path.isfile(log_filename)!=True:
    LOG = logging.getLogger('log_filename')
    LOG.setLevel(logging.DEBUG)
    # create file handler which logs even debug messages
    fh = logging.FileHandler('log_filename')
    fh.setLevel(logging.DEBUG)
    # create console handler with a higher log level
    ch = logging.StreamHandler()
    ch.setLevel(logging.DEBUG)
    # create formatter and add it to the handlers
    formatter = logging.Formatter('-->%(asctime)s - %(name)s:%(levelname)s - %(message)s')
    fh.setFormatter(formatter)
    ch.setFormatter(formatter)
    # add the handlers to the logger
    LOG.addHandler(fh)
    LOG.addHandler(ch)
else:
    LOG=logging.getLogger()

I suspect the problem is with my else block but I don't know how to fix. Could anybody shed some light on this situation.

Answer 1

The logging module's FileHandler takes care of that for you. No need for complexity.

The handler takes an optional mode parameter, to specify whether it starts writing or appending data to it.

From the docs:

class logging.FileHandler(filename, mode='a', encoding=None, delay=False)

The specified file is opened and used as the stream for logging. If mode is not specified, 'a' is used.

Answer 2

For anyone who was trying to create a new directory structure like logs/mylogfile.log, as @mightypile mentioned, FileHandler will not create a new directory structure for you. I used os.makedirs to ensure the directory structure.

import os
import logging

log_filename = "logs/output.log"
os.makedirs(os.path.dirname(log_filename), exist_ok=True)
file_handler = logging.FileHandler(log_filename, mode="w", encoding=None, delay=False)

Answer 3

When you run

LOG = logging.getLogger('log_filename')

for the first time a global variable is created. Hence, you could also add the following code to the script above:

global LOG
if LOG is not None:
    print("found logger !")
else:
    ("no global variable logger found")

Answer 4

You might want to use absolute path.

logging.FileHandler(filename="/home/user/PycharmProjects/project/logs/log_file.log")

Answer 5

Based on @KhoPhi and @loop 's answers, when using a logging config dict you might want to use a class as a handler that you can use in the config. For that you can use the following:

from pathlib import Path
from logging.handlers import RotatingFileHandler as _RotatingFileHandler

class RotatingFileHandler(_RotatingFileHandler):
    def __init__(self, filename: str, *args, **kwargs):
        Path(filename).absolute().parent.mkdir(exist_ok=True, parents=True)
        super().__init__(filename, *args, **kwargs)

you can then use all the examples above with your own handler:

RotatingFileHandler("logs/my_file.log")

or use it in the config file as:

...
handlers:
  my_handler:
    class: your_module.your_submodule.RotatingFileHandler
    filename: logs/my_file.log
...