From 23c83c50a07bbcc7805b2de8d9c3aaa758e19cae Mon Sep 17 00:00:00 2001
From: medusa <newton214@gmail.com>
Date: Sat, 16 Mar 2024 14:38:25 +0000
Subject: [PATCH] Update docs/tech_docs/linux/linux-troubleshooting.md

---
 docs/tech_docs/linux/linux-troubleshooting.md | 72 +++++++++++++++++++
 1 file changed, 72 insertions(+)

diff --git a/docs/tech_docs/linux/linux-troubleshooting.md b/docs/tech_docs/linux/linux-troubleshooting.md
index cd225e2..6c7d25a 100644
--- a/docs/tech_docs/linux/linux-troubleshooting.md
+++ b/docs/tech_docs/linux/linux-troubleshooting.md
@@ -1,3 +1,75 @@
+Creating a focused reference guide for advanced log filtering and analysis, this guide will cover powerful and practical examples using `grep`, `awk`, `sed`, and `tail`. This guide is intended for experienced Linux users who are familiar with the command line and seek to refine their skills in parsing and analyzing log files for troubleshooting and monitoring purposes.
+
+### Log Filtering and Analysis Reference Guide
+
+#### **1. Using `grep` for Basic Searches**
+
+- **Filter Logs by Date**:
+  ```sh
+  grep "2024-03-16" /var/log/syslog
+  ```
+  This command filters entries from March 16, 2024, in the syslog.
+
+- **Search for Error Levels**:
+  ```sh
+  grep -E "error|warn|critical" /var/log/syslog
+  ```
+  Use `-E` for extended regular expressions to match multiple patterns, useful for finding various error levels.
+
+#### **2. Advanced Text Processing with `awk`**
+
+- **Extract Specific Fields**:
+  ```sh
+  awk '/Failed password/ {print $1, $2, $3, $(NF-5), $(NF-3)}' /var/log/auth.log
+  ```
+  This example extracts the date, time, and IP address from failed SSH login attempts. `NF` represents the number of fields in a line, making `$(NF-5)` and `$(NF-3)` select fields relative to the end of the line.
+
+- **Summarize Access by IP Address**:
+  ```sh
+  awk '{print $NF}' /var/log/apache2/access.log | sort | uniq -c | sort -nr
+  ```
+  Here, `$NF` extracts the last field (typically the IP address in access logs), `uniq -c` counts occurrences, and `sort -nr` sorts numerically in reverse for a descending list of IP addresses by access count.
+
+#### **3. Stream Editing with `sed`**
+
+- **Remove Specific Lines**:
+  ```sh
+  sed '/debug/d' /var/log/syslog
+  ```
+  This command deletes lines containing "debug" from the output, useful for excluding verbose log levels.
+
+- **Anonymize IP Addresses**:
+  ```sh
+  sed -r 's/([0-9]{1,3}\.){3}[0-9]{1,3}/[REDACTED IP]/g' /var/log/apache2/access.log
+  ```
+  Using a regular expression, this replaces IP addresses with "[REDACTED IP]" for privacy in shared analysis.
+
+#### **4. Real-time Monitoring with `tail -f` and `grep`**
+
+- **Watch for Specific Log Entries in Real-time**:
+  ```sh
+  tail -f /var/log/syslog | grep "kernel"
+  ```
+  This monitors syslog in real-time for new entries containing "kernel", combining `tail -f` with `grep` for focused live logging.
+
+#### **Combining Tools for Enhanced Analysis**
+
+- **Identify Frequent Access by IP with Timestamps**:
+  ```sh
+  awk '{print $1, $2, $4, $NF}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head
+  ```
+  This command combines `awk` to extract date, time, and IP, then `sort` and `uniq -c` to count and sort access attempts, using `head` to display the top results.
+
+- **Extract and Sort Errors by Frequency**:
+  ```sh
+  grep "error" /var/log/syslog | awk '{print $5}' | sort | uniq -c | sort -nr
+  ```
+  Filter for "error" messages, extract the application or process name (assuming it's the fifth field), count occurrences, and sort them by frequency.
+
+This guide provides a foundation for powerful log analysis techniques. Experimentation and adaptation to specific log formats and requirements will further enhance your proficiency. For deeper exploration, consider the man pages (`man grep`, `man awk`, `man sed`, `man tail`) and other comprehensive resources available online.
+
+---
+
 # Comprehensive Linux Troubleshooting Tools Guide
 
 This guide provides an overview of key packages and their included tools for effective troubleshooting in Linux environments, specifically tailored for RHEL and Debian-based distributions.