13 - Fixing logging and processing data

Last updated Jun 29, 2022 Edit Source

• Daily log

Today I waded through the quagmire of figuring out how to log for a package in Python. I created a function to format a logger and called the function each time I needed a logger, which was in each new module. However, only the first logger would log, and I would miss the rest of the output.

Here’s what it looked like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# logger.py
import logging

def config_logger(name):
    logger = logging.getLogger(name)
    
    # Config logger ...
    return logger

# module.py
from logger import config_logger


logger = config_logger(__name__)

def test():
    logger.info("I will not display my output :(")

# pkg.py
import os
from module import test
from logger import config_logger

logger = config_logger(__name__)

logger.info("Logging from the main function")
test()

The output would be:

1

Logging from the main function

This method creates a new root logger each time, so the app loses subsequent logger calls.

Here’s a very simple solution I found:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

# module.py
import logging


logger = logging.getLogger(__name__)

def test():
    logger.info("I will display my output!")

# pkg.py
import os
import logging
from module import test

logging.basicConfig(level=os.environ.get("LOGLEVEL", "INFO"))

logger = logging.getLogger(__name__)

logger.info("Logging from the main function")
test()

Output:

1
2

Logging from the main function
I will display my output!

The logging.basicConfig line creates the root logger and should go as far toward the beginning of your application as possible. Then each module - or subsequent .py file imported within your application - should have the logger = logging.getLogger(__name__) line. This line will create a new child logger with the module’s name, making it easy to debug.

After looking through many resources to solve this problem, this was the most straightforward explanation I could find sans techie details: loggly.com.

I'm a freelance software developer located in Denver, Colorado. If you're interested in working together or would just like to say hi you can reach me at me@ this domain.