Update tech_docs/linux/rsync.md

tech_docs/linux/rsync.md

@@ -1,3 +1,187 @@
+# **The Ultimate Rsync Mastery Guide: From Basics to Advanced Enterprise Use**
+
+## **Table of Contents**
+1. [Rsync Fundamentals](#1-rsync-fundamentals)
+2. [Advanced Transfer Techniques](#2-advanced-transfer-techniques)
+3. [Network Optimization](#3-network-optimization)
+4. [Cloud & Enterprise Integrations](#4-cloud--enterprise-integrations)
+5. [Security Hardening](#5-security-hardening)
+6. [Troubleshooting & Benchmarks](#6-troubleshooting--benchmarks)
+7. [Real-World Recipes](#7-real-world-recipes)
+
+---
+
+## **1. Rsync Fundamentals**
+
+### **Core Syntax**
+```bash
+rsync [OPTIONS] SOURCE DESTINATION
+```
+
+### **Essential Flags**
+| Flag | Description | Use Case |
+|------|-------------|----------|
+| `-a` | Archive mode (recursive + preserves everything) | Backups |
+| `-v` | Verbose output | Debugging |
+| `-z` | Compression | Slow networks |
+| `-P` | Progress + resume | Large files |
+| `-n` | Dry run | Safety checks |
+| `--delete` | Mirror source exactly | Deployment |
+
+### **Trailing Slash Rules**
+```bash
+rsync /folder /dest       # Copies folder as /dest/folder
+rsync /folder/ /dest      # Copies folder's contents to /dest
+```
+
+---
+
+## **2. Advanced Transfer Techniques**
+
+### **Delta Algorithm Control**
+```bash
+rsync -av --whole-file /src/ /dest/      # Disable delta-xfer (LAN only)
+rsync -av --checksum /src/ /dest/        # Force checksum validation
+```
+
+### **Filesystem Specials**
+```bash
+rsync -avH /src/ /dest/                  # Preserve hard links
+rsync -avX /src/ /dest/                  # Keep device files
+rsync -avA /src/ /dest/                  # Preserve ACLs
+```
+
+### **Advanced Filtering**
+```bash
+rsync -av --filter='+ *.jpg' --filter='- *' /src/ /dest/  # JPGs only
+rsync -av --exclude={'*.tmp','cache/'} /src/ /dest/       # Multiple excludes
+```
+
+---
+
+## **3. Network Optimization**
+
+### **Bandwidth Control**
+```bash
+rsync -avz --bwlimit=5000 /src/ user@remote:/dest/  # 5MB/s limit
+```
+
+### **Parallel Transfers**
+```bash
+# Using GNU Parallel (install via brew/apt)
+find /src/ -type f | parallel -j8 rsync -a {} /dest/{}
+```
+
+### **SSH Tuning**
+```bash
+rsync -av -e 'ssh -T -c aes128-gcm@openssh.com -o Compression=no' /src/ remote:/dest/
+```
+
+---
+
+## **4. Cloud & Enterprise Integrations**
+
+### **S3/Cloud Storage Sync**
+```bash
+# Using rclone (recommended)
+rclone sync -P --transfers=32 /src/ s3:bucket/path/
+
+# Direct with AWS CLI
+find /src/ -type f | parallel -j16 aws s3 cp {} s3://bucket/{}
+```
+
+### **ZFS Integration**
+```bash
+# Snapshot-aware replication
+zfs snapshot tank/data@rsync
+rsync -av --delete /tank/data/.zfs/snapshot/rsync/ remote:/backup/
+```
+
+### **Database-Aware Backups**
+```bash
+# MySQL hot backup
+mysqldump --single-transaction db | rsync -avz - user@remote:/backups/db.sql
+```
+
+---
+
+## **5. Security Hardening**
+
+### **Rsync Daemon Security**
+`/etc/rsyncd.conf`:
+```ini
+[secure]
+path = /data
+auth users = backupuser
+secrets file = /etc/rsyncd.secrets
+hosts allow = 192.168.1.0/24
+read only = yes
+```
+
+### **SSH Restrictions**
+`~/.ssh/authorized_keys`:
+```bash
+command="rsync --server --config=/etc/rsync-server.conf -vlogDtprze.iLsf ." ssh-rsa AAAAB3...
+```
+
+---
+
+## **6. Troubleshooting & Benchmarks**
+
+### **Performance Testing**
+```bash
+# Create test dataset
+mkdir -p /test && dd if=/dev/urandom of=/test/file1.bin bs=1M count=1024
+
+# Benchmark
+time rsync -avP /test/ /dest/
+```
+
+### **Common Issues**
+| Symptom | Solution |
+|---------|----------|
+| Stuck transfers | Add `--timeout=300` |
+| Permission errors | Use `--super` or `sudo` |
+| Connection drops | `--partial --partial-dir=.rsync-partial` |
+
+---
+
+## **7. Real-World Recipes**
+
+### **Atomic Web Deployment**
+```bash
+rsync -avz --delete /build/ user@web01:/var/www/tmp_deploy/
+ssh user@web01 "mv /var/www/live /var/www/old && \
+               mv /var/www/tmp_deploy /var/www/live"
+```
+
+### **Incremental Backups**
+```bash
+rsync -av --link-dest=/backups/previous/ /src/ /backups/$(date +%Y%m%d)/
+```
+
+### **Multi-Cloud Sync**
+```bash
+rsync -av /data/ user@jumpbox:/cloudgateway/
+ssh user@jumpbox "rclone sync -P /cloudgateway/ gcs:bucket/path/"
+```
+
+---
+
+## **Final Pro Tips**
+- **Use `-vvv`** for debug-level output
+- **Combine with `ionice`** for disk-sensitive workloads
+- **Log everything**: `rsync [...] >> /var/log/rsync.log 2>&1`
+- **Version control**: `rsync --backup --backup-dir=/versions/$(date +%s)`
+
+This guide covers 90% of professional rsync use cases. For edge cases like **PB-scale transfers** or **real-time sync**, consider:  
+- [`lsyncd`](https://github.com/axkibe/lsyncd) for real-time mirroring  
+- [`btrfs send/receive`](https://btrfs.wiki.kernel.org/index.php/Incremental_Backup) for filesystem-level replication  
+
+**Need a custom solution?** Provide your specific scenario for tailored commands!
+
+---
+
 # **The Complete Rsync Guide: Mastering File Synchronization**
 
 Rsync (*Remote Synchronization*) is one of the most powerful and efficient tools for copying and synchronizing files locally or across networks. It’s widely used for backups, mirroring, and deploying files efficiently by only transferring changes.