Update tech_docs/python/python_logging_best_practices.md
This commit is contained in:
@@ -1,122 +0,0 @@
|
||||
### Introduction to Logging in Python
|
||||
|
||||
Logging is a crucial aspect of developing robust Python applications. It provides a way to track events that happen when the software runs, which can be helpful for debugging and monitoring purposes. Python's `logging` module offers a flexible framework for emitting log messages from Python programs. This module is part of Python's standard library and is highly configurable, allowing you to direct log messages to different destinations (e.g., console, files, etc.) and filter them based on severity levels.
|
||||
|
||||
### Basic Logging Example
|
||||
|
||||
Below is a basic example of how to set up and use logging in a Python script. This example includes configuring the logging system and utilizing various log levels.
|
||||
|
||||
```python
|
||||
import logging
|
||||
|
||||
def main():
|
||||
logging.debug("This is a debug message")
|
||||
logging.info("This is an info message")
|
||||
logging.warning("This is a warning message")
|
||||
logging.error("This is an error message")
|
||||
logging.critical("This is a critical message")
|
||||
|
||||
if __name__ == "__main__":
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
main()
|
||||
```
|
||||
|
||||
### Detailed Explanation
|
||||
|
||||
1. **Importing the Logging Module**:
|
||||
```python
|
||||
import logging
|
||||
```
|
||||
- The `logging` module is imported to enable logging functionality.
|
||||
|
||||
2. **Defining the Main Function**:
|
||||
```python
|
||||
def main():
|
||||
logging.debug("This is a debug message")
|
||||
logging.info("This is an info message")
|
||||
logging.warning("This is a warning message")
|
||||
logging.error("This is an error message")
|
||||
logging.critical("This is a critical message")
|
||||
```
|
||||
- The `main()` function contains several logging statements, each demonstrating a different log level:
|
||||
- `logging.debug()`: Used for detailed information, typically of interest only when diagnosing problems.
|
||||
- `logging.info()`: Used to confirm that things are working as expected.
|
||||
- `logging.warning()`: Used to indicate something unexpected happened or indicative of some problem in the near future (e.g., ‘disk space low’). The software is still working as expected.
|
||||
- `logging.error()`: Used to indicate that due to a more serious problem, the software has not been able to perform some function.
|
||||
- `logging.critical()`: Used to indicate a very serious error, the program itself may be unable to continue running.
|
||||
|
||||
3. **Main Execution Block**:
|
||||
```python
|
||||
if __name__ == "__main__":
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
main()
|
||||
```
|
||||
- The `if __name__ == "__main__":` block checks if the script is being run directly.
|
||||
- `logging.basicConfig(level=logging.DEBUG)`: Configures the logging system to display messages at the `DEBUG` level and above. By default, it logs to the console.
|
||||
- `main()`: Calls the `main` function to execute the logging statements.
|
||||
|
||||
### Best Practices for Logging in Python
|
||||
|
||||
1. **Avoid Hardcoding Log Levels**:
|
||||
- Instead of hardcoding log levels in the `basicConfig`, consider making the log level configurable via environment variables or command-line arguments.
|
||||
```python
|
||||
import logging
|
||||
import os
|
||||
|
||||
def main():
|
||||
logging.debug("This is a debug message")
|
||||
logging.info("This is an info message")
|
||||
logging.warning("This is a warning message")
|
||||
logging.error("This is an error message")
|
||||
logging.critical("This is a critical message")
|
||||
|
||||
if __name__ == "__main__":
|
||||
log_level = os.getenv('LOG_LEVEL', 'DEBUG').upper()
|
||||
logging.basicConfig(level=getattr(logging, log_level, logging.DEBUG))
|
||||
main()
|
||||
```
|
||||
|
||||
2. **Use Different Handlers for Different Destinations**:
|
||||
- You can configure logging to direct messages to different destinations such as files, HTTP endpoints, or even remote logging services.
|
||||
```python
|
||||
import logging
|
||||
|
||||
def setup_logging():
|
||||
logger = logging.getLogger()
|
||||
logger.setLevel(logging.DEBUG)
|
||||
|
||||
# Console handler
|
||||
console_handler = logging.StreamHandler()
|
||||
console_handler.setLevel(logging.DEBUG)
|
||||
console_format = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
|
||||
console_handler.setFormatter(console_format)
|
||||
logger.addHandler(console_handler)
|
||||
|
||||
# File handler
|
||||
file_handler = logging.FileHandler('app.log')
|
||||
file_handler.setLevel(logging.INFO)
|
||||
file_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
|
||||
file_handler.setFormatter(file_format)
|
||||
logger.addHandler(file_handler)
|
||||
|
||||
def main():
|
||||
logging.debug("This is a debug message")
|
||||
logging.info("This is an info message")
|
||||
logging.warning("This is a warning message")
|
||||
logging.error("This is an error message")
|
||||
logging.critical("This is a critical message")
|
||||
|
||||
if __name__ == "__main__":
|
||||
setup_logging()
|
||||
main()
|
||||
```
|
||||
|
||||
3. **Use Loggers, Handlers, and Formatters**:
|
||||
- Separate loggers, handlers, and formatters allow for greater flexibility and control over logging output.
|
||||
- Loggers: Create multiple loggers for different parts of your application.
|
||||
- Handlers: Direct log messages to different destinations.
|
||||
- Formatters: Define the layout of log messages.
|
||||
|
||||
### Conclusion
|
||||
|
||||
Using the `logging` module in Python is an effective way to monitor and debug applications. By configuring logging appropriately and using different log levels, you can gain insights into your application's behavior and quickly identify issues. Following best practices such as avoiding hardcoded log levels and using multiple handlers and formatters will make your logging setup more robust and flexible.
|
||||
268
tech_docs/python/python_logging_best_practices.md
Normal file
268
tech_docs/python/python_logging_best_practices.md
Normal file
@@ -0,0 +1,268 @@
|
||||
### Code Breakdown
|
||||
|
||||
#### Importing the Logging Module
|
||||
|
||||
```python
|
||||
import logging
|
||||
```
|
||||
|
||||
- **Purpose:** This line imports the `logging` module, which provides a flexible framework for emitting log messages from Python programs. Logging is crucial for tracking events that happen during runtime and is particularly useful for debugging and monitoring applications.
|
||||
|
||||
#### Defining the `main` Function
|
||||
|
||||
```python
|
||||
def main():
|
||||
logging.info("START")
|
||||
```
|
||||
|
||||
- **Function Definition:** The `main` function is defined, which contains a single statement.
|
||||
- **Logging an INFO Message:** Inside the function, `logging.info("START")` logs a message at the INFO level. INFO level is used to confirm that things are working as expected.
|
||||
|
||||
#### Checking if the Script is Run Directly
|
||||
|
||||
```python
|
||||
if __name__ == "__main__":
|
||||
logging.basicConfig(level="DEBUG")
|
||||
main()
|
||||
```
|
||||
|
||||
- **`__name__ == "__main__"`:** This condition checks whether the script is being run directly or being imported as a module in another script. If the script is run directly, `__name__` is set to `"__main__"`.
|
||||
- **Direct Run:** When the script is executed directly (e.g., `python main.py`), this block is executed.
|
||||
- **Import as Module:** If the script is imported into another script, this block is not executed.
|
||||
|
||||
- **Configuring the Logging System:**
|
||||
```python
|
||||
logging.basicConfig(level="DEBUG")
|
||||
```
|
||||
- **`basicConfig`:** This function configures the logging system. It sets the basic configuration for the root logger.
|
||||
- **`level="DEBUG"`:** Sets the threshold for the logger to DEBUG. This means that all messages at this level and higher (DEBUG, INFO, WARNING, ERROR, CRITICAL) will be handled.
|
||||
- **DEBUG:** Detailed information, typically of interest only when diagnosing problems.
|
||||
- **INFO:** Confirmation that things are working as expected.
|
||||
- **WARNING:** An indication that something unexpected happened, or indicative of some problem in the near future (e.g., ‘disk space low’). The software is still working as expected.
|
||||
- **ERROR:** Due to a more serious problem, the software has not been able to perform some function.
|
||||
- **CRITICAL:** A very serious error, indicating that the program itself may be unable to continue running.
|
||||
|
||||
- **Calling the `main` Function:**
|
||||
```python
|
||||
main()
|
||||
```
|
||||
- This line calls the `main` function, which triggers the execution of its contents, logging the INFO level message "START".
|
||||
|
||||
### Execution Walkthrough
|
||||
|
||||
1. **Script Execution:**
|
||||
- You run the script with `python3 main.py`.
|
||||
|
||||
2. **Interpreter Processing:**
|
||||
- The Python interpreter starts executing the script. It first processes the import statement.
|
||||
|
||||
3. **Importing Logging:**
|
||||
- The `logging` module is imported, making all its functions available in the script.
|
||||
|
||||
4. **Function Definition:**
|
||||
- The `main` function is defined but not yet executed.
|
||||
|
||||
5. **Conditional Check:**
|
||||
- The interpreter checks the condition `if __name__ == "__main__":`. Since you are running the script directly, `__name__` is set to `"__main__"`, so the block inside this condition is executed.
|
||||
|
||||
6. **Configuring Logging:**
|
||||
- `logging.basicConfig(level="DEBUG")` sets up the logging configuration, specifying that all messages of level DEBUG and above should be logged.
|
||||
|
||||
7. **Executing `main`:**
|
||||
- The `main()` function is called.
|
||||
- Inside `main`, `logging.info("START")` logs the message "START" at the INFO level.
|
||||
|
||||
8. **Output Generation:**
|
||||
- Since the logging level is set to DEBUG (which includes INFO messages), the logged message "START" is output to the console.
|
||||
- The output format is `INFO:root:START`, where:
|
||||
- `INFO` is the log level of the message.
|
||||
- `root` is the name of the logger (the default root logger in this case).
|
||||
- `START` is the message logged.
|
||||
|
||||
### Detailed Logging Explanation
|
||||
|
||||
- **Loggers:** Objects that you call when you want to record a message. You can have multiple loggers, each with a different name.
|
||||
- **Handlers:** Send the log messages (created by loggers) to the appropriate destination, like the console or a file.
|
||||
- **Formatters:** Specify the layout of the log messages. They define how the log records should be presented.
|
||||
|
||||
In this script, we use the default root logger, configured by `basicConfig`.
|
||||
|
||||
- **Log Levels:**
|
||||
- **DEBUG:** 10
|
||||
- **INFO:** 20
|
||||
- **WARNING:** 30
|
||||
- **ERROR:** 40
|
||||
- **CRITICAL:** 50
|
||||
|
||||
By setting the logging level to DEBUG, you ensure that all messages of level DEBUG and higher will be processed.
|
||||
|
||||
### Advanced Usage
|
||||
|
||||
For more advanced logging setups, you can:
|
||||
- Create custom loggers with `logging.getLogger(name)`.
|
||||
- Add multiple handlers to log to different destinations.
|
||||
- Use different formatters to customize the log message layout.
|
||||
|
||||
Here's a more advanced example:
|
||||
|
||||
```python
|
||||
import logging
|
||||
|
||||
# Create a custom logger
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Create handlers
|
||||
console_handler = logging.StreamHandler()
|
||||
file_handler = logging.FileHandler('file.log')
|
||||
|
||||
# Set levels for handlers
|
||||
console_handler.setLevel(logging.WARNING)
|
||||
file_handler.setLevel(logging.ERROR)
|
||||
|
||||
# Create formatters and add them to handlers
|
||||
console_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
|
||||
file_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
|
||||
console_handler.setFormatter(console_format)
|
||||
file_handler.setFormatter(file_format)
|
||||
|
||||
# Add handlers to the logger
|
||||
logger.addHandler(console_handler)
|
||||
logger.addHandler(file_handler)
|
||||
|
||||
logger.warning('This is a warning')
|
||||
logger.error('This is an error')
|
||||
```
|
||||
|
||||
This script:
|
||||
- Creates a custom logger.
|
||||
- Adds a console handler and a file handler.
|
||||
- Sets different logging levels for each handler.
|
||||
- Uses different formatters for each handler.
|
||||
- Logs messages, which appear in the console and the file based on their levels.
|
||||
|
||||
This comprehensive breakdown should help you understand the basics and some advanced concepts of Python logging.
|
||||
|
||||
---
|
||||
|
||||
### Introduction to Logging in Python
|
||||
|
||||
Logging is a crucial aspect of developing robust Python applications. It provides a way to track events that happen when the software runs, which can be helpful for debugging and monitoring purposes. Python's `logging` module offers a flexible framework for emitting log messages from Python programs. This module is part of Python's standard library and is highly configurable, allowing you to direct log messages to different destinations (e.g., console, files, etc.) and filter them based on severity levels.
|
||||
|
||||
### Basic Logging Example
|
||||
|
||||
Below is a basic example of how to set up and use logging in a Python script. This example includes configuring the logging system and utilizing various log levels.
|
||||
|
||||
```python
|
||||
import logging
|
||||
|
||||
def main():
|
||||
logging.debug("This is a debug message")
|
||||
logging.info("This is an info message")
|
||||
logging.warning("This is a warning message")
|
||||
logging.error("This is an error message")
|
||||
logging.critical("This is a critical message")
|
||||
|
||||
if __name__ == "__main__":
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
main()
|
||||
```
|
||||
|
||||
### Detailed Explanation
|
||||
|
||||
1. **Importing the Logging Module**:
|
||||
```python
|
||||
import logging
|
||||
```
|
||||
- The `logging` module is imported to enable logging functionality.
|
||||
|
||||
2. **Defining the Main Function**:
|
||||
```python
|
||||
def main():
|
||||
logging.debug("This is a debug message")
|
||||
logging.info("This is an info message")
|
||||
logging.warning("This is a warning message")
|
||||
logging.error("This is an error message")
|
||||
logging.critical("This is a critical message")
|
||||
```
|
||||
- The `main()` function contains several logging statements, each demonstrating a different log level:
|
||||
- `logging.debug()`: Used for detailed information, typically of interest only when diagnosing problems.
|
||||
- `logging.info()`: Used to confirm that things are working as expected.
|
||||
- `logging.warning()`: Used to indicate something unexpected happened or indicative of some problem in the near future (e.g., ‘disk space low’). The software is still working as expected.
|
||||
- `logging.error()`: Used to indicate that due to a more serious problem, the software has not been able to perform some function.
|
||||
- `logging.critical()`: Used to indicate a very serious error, the program itself may be unable to continue running.
|
||||
|
||||
3. **Main Execution Block**:
|
||||
```python
|
||||
if __name__ == "__main__":
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
main()
|
||||
```
|
||||
- The `if __name__ == "__main__":` block checks if the script is being run directly.
|
||||
- `logging.basicConfig(level=logging.DEBUG)`: Configures the logging system to display messages at the `DEBUG` level and above. By default, it logs to the console.
|
||||
- `main()`: Calls the `main` function to execute the logging statements.
|
||||
|
||||
### Best Practices for Logging in Python
|
||||
|
||||
1. **Avoid Hardcoding Log Levels**:
|
||||
- Instead of hardcoding log levels in the `basicConfig`, consider making the log level configurable via environment variables or command-line arguments.
|
||||
```python
|
||||
import logging
|
||||
import os
|
||||
|
||||
def main():
|
||||
logging.debug("This is a debug message")
|
||||
logging.info("This is an info message")
|
||||
logging.warning("This is a warning message")
|
||||
logging.error("This is an error message")
|
||||
logging.critical("This is a critical message")
|
||||
|
||||
if __name__ == "__main__":
|
||||
log_level = os.getenv('LOG_LEVEL', 'DEBUG').upper()
|
||||
logging.basicConfig(level=getattr(logging, log_level, logging.DEBUG))
|
||||
main()
|
||||
```
|
||||
|
||||
2. **Use Different Handlers for Different Destinations**:
|
||||
- You can configure logging to direct messages to different destinations such as files, HTTP endpoints, or even remote logging services.
|
||||
```python
|
||||
import logging
|
||||
|
||||
def setup_logging():
|
||||
logger = logging.getLogger()
|
||||
logger.setLevel(logging.DEBUG)
|
||||
|
||||
# Console handler
|
||||
console_handler = logging.StreamHandler()
|
||||
console_handler.setLevel(logging.DEBUG)
|
||||
console_format = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
|
||||
console_handler.setFormatter(console_format)
|
||||
logger.addHandler(console_handler)
|
||||
|
||||
# File handler
|
||||
file_handler = logging.FileHandler('app.log')
|
||||
file_handler.setLevel(logging.INFO)
|
||||
file_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
|
||||
file_handler.setFormatter(file_format)
|
||||
logger.addHandler(file_handler)
|
||||
|
||||
def main():
|
||||
logging.debug("This is a debug message")
|
||||
logging.info("This is an info message")
|
||||
logging.warning("This is a warning message")
|
||||
logging.error("This is an error message")
|
||||
logging.critical("This is a critical message")
|
||||
|
||||
if __name__ == "__main__":
|
||||
setup_logging()
|
||||
main()
|
||||
```
|
||||
|
||||
3. **Use Loggers, Handlers, and Formatters**:
|
||||
- Separate loggers, handlers, and formatters allow for greater flexibility and control over logging output.
|
||||
- Loggers: Create multiple loggers for different parts of your application.
|
||||
- Handlers: Direct log messages to different destinations.
|
||||
- Formatters: Define the layout of log messages.
|
||||
|
||||
### Conclusion
|
||||
|
||||
Using the `logging` module in Python is an effective way to monitor and debug applications. By configuring logging appropriately and using different log levels, you can gain insights into your application's behavior and quickly identify issues. Following best practices such as avoiding hardcoded log levels and using multiple handlers and formatters will make your logging setup more robust and flexible.
|
||||
Reference in New Issue
Block a user