the_information_nexus/tech_docs/linux/sed.md

The Sed Mastery Guide: From Basics to Advanced Text Alchemy

Table of Contents

1. Sed Fundamentals
2. Pattern Addressing
3. Advanced Substitutions
4. Multi-Command Scripts
5. Branching & Flow Control
6. Real-World Recipes
7. Performance & Limitations

---

1. Sed Fundamentals

Core Syntax

sed [OPTIONS] 'COMMAND' file.txt

Essential Operations

Command	Description	Example
s/old/new/	Substitute first occurrence	sed 's/foo/bar/'
s/old/new/g	Substitute globally	sed 's/foo/bar/g'
p	Print pattern space	sed -n '1,5p'
d	Delete lines	sed '/error/d'
i\text	Insert before line	sed '3i\New Line'
a\text	Append after line	sed '$a\END'

Common Flags

sed -i '' file.txt       # In-place edit (BSD/macOS)
sed -i file.txt          # In-place edit (Linux)
sed -n '1,5p' file.txt   # Print only lines 1-5
sed -e 'cmd1' -e 'cmd2'  # Multiple commands

---

2. Pattern Addressing

Line Number Targeting

sed '5s/foo/bar/'        # Only line 5
sed '1,10d'              # Delete lines 1-10
sed '$d'                 # Delete last line

Regex Patterns

sed '/^#/d'              # Delete comment lines
sed '/start/,/end/d'     # Delete range between patterns
sed '/pattern/!d'        # Keep only matching lines

Special Addresses

sed -n '1~2p'            # Every 2nd line starting at 1
sed -n '2,${p;n}'        # Every other line from line 2

---

3. Advanced Substitutions

Capture Groups

echo "Hello World" | sed 's/\(Hello\) \(World\)/\2 \1/'
# Output: World Hello

Custom Delimiters

sed 's|/usr/local|/opt|g'  # Use | instead of /

Case Conversion

sed 's/.*/\L&/'           # Lowercase entire line
sed 's/.*/\U&/'           # Uppercase entire line

Conditional Replacement

sed '/error/s/foo/bar/'   # Only substitute if line contains "error"

---

4. Multi-Command Scripts

Command Files

# script.sed
/error/ {
  s/foo/bar/g
  p
}

sed -f script.sed file.txt

Inline Multi-Commands

sed -e 's/foo/bar/' -e '/baz/d' file.txt

Hold Space Operations

sed -n '1h; 2,${H;g; s/\n/ /g; p}'  # Combine lines 1-N

---

5. Branching & Flow Control

Basic Branching

sed '/start/{:loop; N; /end/!b loop; s/start.*end/replacement/}'

Labels & Loops

sed ':top; /pattern/{N; b top}'  # Accumulate until pattern fails

Conditional Branch

sed '/error/{s/foo/bar/; t; s/foo/baz/}'  # If first sub worked, skip second

---

6. Real-World Recipes

CSV Processing

sed 's/^\([^,]*\),\([^,]*\)/\2,\1/' data.csv  # Swap first two columns

Log File Cleaning

sed -E '/^(DEBUG|TRACE)/d; s/[0-9]{4}-[0-9]{2}-[0-9]{2}/DATE_REDACTED/g' app.log

HTML/XML Processing

sed '/<script/,/<\/script>/d' page.html  # Remove script tags

Code Refactoring

# Rename function and all calls
sed -i 's/old_func_name/new_func_name/g' *.py

---

7. Performance & Limitations

Large File Handling

sed -n '1,1000000p' huge.log > sample.log  # Extract first million lines

When Not to Use Sed

• JSON/XML with nested structures (use jq/xmllint)
• Multi-line records spanning variable lines (use awk)
• Binary file editing (use xxd + sed + xxd -r)

Performance Tips

LC_ALL=C sed ...        # 2-3x speedup for ASCII
sed ':a;N;$!ba;s/\n//g' # Slurp entire file (careful with memory)

---

Pro-Level Techniques

In-Place Editing Backup

sed -i.bak 's/foo/bar/' file.txt  # Creates file.txt.bak

Non-Greedy Matching

sed 's/foo\(.*\?\)bar/\1/'  # Perl-style regex (requires -r/-E)

Random Line Selection

sed -n "$RANDOM"' {p;q}'    # Get one random line

---

Sed + Awk Power Combo

# Extract columns from lines matching pattern
sed -n '/pattern/p' file | awk '{print $2,$4}'

For complex text transformations, consider combining sed with:

• awk for columnar data
• perl for advanced regex
• grep for filtering

Need a custom sed solution? Provide your text transformation challenge!