Update tech_docs/linux/journalctl.md
This commit is contained in:
@@ -1,40 +1,127 @@
|
||||
# `journalctl` Troubleshooting Guide
|
||||
Here's the optimized version of your cheat sheet, maintaining the clear tabular format while removing hashtags and refining the organization:
|
||||
|
||||
This guide provides a structured approach to troubleshooting common issues in Linux using the `journalctl` command.
|
||||
---
|
||||
### `journalctl` Sysadmin Cheat Sheet
|
||||
*A comprehensive reference for system log management in systemd-based Linux systems*
|
||||
|
||||
## General Troubleshooting
|
||||
---
|
||||
|
||||
1. **Review Recent Logs**
|
||||
- View recent log entries: `journalctl -e`
|
||||
- Show logs since the last boot: `journalctl -b`
|
||||
#### **1. Basic Log Inspection**
|
||||
| Command | Description |
|
||||
|--------------------------|----------------------------------------------|
|
||||
| `journalctl` | View full system logs (press `q` to exit) |
|
||||
| `journalctl -n 50` | Show last 50 log entries |
|
||||
| `journalctl -f` | Follow logs in real-time (`Ctrl+C` to stop) |
|
||||
| `journalctl -e` | Jump to end of logs (most recent entries) |
|
||||
|
||||
## Service-Specific Issues
|
||||
---
|
||||
|
||||
1. **Identify Service Issues**
|
||||
- Display logs for a specific service: `journalctl -u service-name.service`
|
||||
- Replace `service-name` with the actual service name, e.g., `journalctl -u sshd`
|
||||
#### **2. Boot-Specific Logs**
|
||||
| Command | Description |
|
||||
|--------------------------|----------------------------------------------|
|
||||
| `journalctl -b` | Current boot logs |
|
||||
| `journalctl -b -1` | Previous boot logs |
|
||||
| `journalctl -b -2` | Two boots ago |
|
||||
| `journalctl --list-boots`| List all recorded boot sessions |
|
||||
|
||||
## System Crashes or Boots
|
||||
---
|
||||
|
||||
1. **Investigate Boot Issues**
|
||||
- Display logs from the current boot: `journalctl -b`
|
||||
- Show logs from the previous boot: `journalctl -b -1`
|
||||
- List boot sessions to identify specific instances: `journalctl --list-boots`
|
||||
#### **3. Service-Specific Logs**
|
||||
| Command | Example | Description |
|
||||
|--------------------------|-----------------------------|--------------------------------------|
|
||||
| `journalctl -u [service]`| `-u nginx` | Show logs for specific service |
|
||||
| `journalctl -fu [service]`| `-fu sshd` | Follow service logs in real-time |
|
||||
| `journalctl _SYSTEMD_UNIT=`| `_SYSTEMD_UNIT=crond` | Alternate service filtering syntax |
|
||||
|
||||
## Error Messages
|
||||
---
|
||||
|
||||
1. **Filter by Error Priority**
|
||||
- Show only error messages: `journalctl -p err`
|
||||
- For more severe issues, consider using higher priority levels like `crit`, `alert`, or `emerg`
|
||||
#### **4. Priority Level Filters**
|
||||
| Level | Name | Description | Example Usage |
|
||||
|-------|---------|---------------------------------|-----------------------------------|
|
||||
| 0 | emerg | System is unusable | `journalctl -p emerg -b` |
|
||||
| 3 | err | Error conditions | `journalctl -p err --since today` |
|
||||
| 4 | warning | Warning messages | `journalctl -p 3..4` (error+warn) |
|
||||
|
||||
## Additional Tips
|
||||
---
|
||||
|
||||
- **Follow Live Logs**: Monitor logs in real-time: `journalctl -f`
|
||||
- **Time-Based Filtering**: Investigate issues within a specific timeframe:
|
||||
- Since a specific time: `journalctl --since "YYYY-MM-DD HH:MM:SS"`
|
||||
- Between two timestamps: `journalctl --since "start-time" --until "end-time"`
|
||||
- **Output Formatting**: Adjust output format for better readability or specific needs:
|
||||
- JSON format: `journalctl -o json-pretty`
|
||||
- Verbose format: `journalctl -o verbose`
|
||||
- **Export Logs**: Save logs for further analysis or reporting:
|
||||
- `journalctl > logs.txt` or `journalctl -u service-name > service_logs.txt`
|
||||
#### **5. Time-Based Filtering**
|
||||
| Command Format | Example |
|
||||
|---------------------------|----------------------------------|
|
||||
| `--since "YYYY-MM-DD"` | `--since "2023-10-01"` |
|
||||
| `--since "HH:MM:SS"` | `--since "09:30:00"` |
|
||||
| `--since "1 hour ago"` | `--since "yesterday 14:00"` |
|
||||
| `--until "tomorrow"` | `--until "2023-10-05 18:00:00"` |
|
||||
|
||||
---
|
||||
|
||||
#### **6. Advanced Filtering**
|
||||
| Command | Description |
|
||||
|--------------------------|----------------------------------------------|
|
||||
| `journalctl _UID=1000` | Filter by user ID |
|
||||
| `journalctl _PID=1234` | Filter by process ID |
|
||||
| `journalctl _COMM=sshd` | Filter by process name |
|
||||
| `journalctl -k` | Kernel logs (equivalent to `dmesg`) |
|
||||
|
||||
---
|
||||
|
||||
#### **7. Output Formatting**
|
||||
| Command | Description |
|
||||
|--------------------------|----------------------------------------------|
|
||||
| `--output=json` | JSON format for parsing |
|
||||
| `--output=json-pretty` | Human-readable JSON |
|
||||
| `--output=short` | Compact format |
|
||||
| `--output=verbose` | Full details including metadata |
|
||||
|
||||
---
|
||||
|
||||
#### **8. Log Maintenance**
|
||||
| Command | Description |
|
||||
|----------------------------------|------------------------------------------|
|
||||
| `sudo journalctl --vacuum-size=1G` | Limit logs to 1GB (removes oldest) |
|
||||
| `sudo journalctl --vacuum-time=2weeks` | Keep only last 2 weeks of logs |
|
||||
| `journalctl --disk-usage` | Show current log storage usage |
|
||||
|
||||
---
|
||||
|
||||
### **Practical Examples**
|
||||
|
||||
**1. Diagnosing SSH Issues**
|
||||
```bash
|
||||
journalctl -fu sshd
|
||||
```
|
||||
|
||||
**2. Checking Today's Errors**
|
||||
```bash
|
||||
journalctl -p err --since today
|
||||
```
|
||||
|
||||
**3. Comparing Boot Logs**
|
||||
```bash
|
||||
journalctl -b -1 | grep "fail" > prev_boot_errors.txt
|
||||
journalctl -b | grep "fail" > current_boot_errors.txt
|
||||
diff prev_boot_errors.txt current_boot_errors.txt
|
||||
```
|
||||
|
||||
**4. Persistent Logging Setup**
|
||||
```bash
|
||||
sudo mkdir -p /var/log/journal
|
||||
sudo systemctl restart systemd-journald
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### **Best Practices**
|
||||
|
||||
- Combine with standard tools:
|
||||
```bash
|
||||
journalctl -u apache --since yesterday | grep -i timeout | less
|
||||
```
|
||||
|
||||
- For long-term analysis, export to files:
|
||||
```bash
|
||||
journalctl --since "2023-01-01" --until "2023-06-30" > first_half_2023.log
|
||||
```
|
||||
|
||||
- Monitor log growth regularly:
|
||||
```bash
|
||||
journalctl --disk-usage
|
||||
Reference in New Issue
Block a user