Update tech_docs/edge-compute-k3s.md
This commit is contained in:
@@ -1,3 +1,421 @@
|
||||
Great! I’ll review and streamline the suggestions to ensure they are **useful**, **actionable**, and **necessary**, focusing on practical learning while setting up your system. Here’s a clean, highly focused version to guide your journey into learning the Linux CLI, bash, and shell scripting as you set up Rocky Linux for K3s and Python-based workloads:
|
||||
|
||||
---
|
||||
|
||||
### **1. Update and Secure the System**
|
||||
|
||||
Start by ensuring your system is updated and secure. This is essential to establish a solid foundation.
|
||||
|
||||
#### **System Updates**
|
||||
```bash
|
||||
sudo dnf update -y && sudo dnf upgrade -y
|
||||
sudo reboot
|
||||
```
|
||||
- **Purpose**: Keep the system updated with the latest security patches and improvements.
|
||||
|
||||
#### **SELinux Configuration**
|
||||
```bash
|
||||
sestatus # Check SELinux status
|
||||
```
|
||||
- **If not in enforcing mode**, edit the config:
|
||||
```bash
|
||||
sudo nano /etc/selinux/config
|
||||
```
|
||||
- Set `SELINUX=enforcing`, save, and reboot.
|
||||
- **Purpose**: Enforce security at the kernel level by keeping SELinux enabled.
|
||||
|
||||
---
|
||||
|
||||
### **2. Install Essential System Tools**
|
||||
|
||||
Install basic tools for system management and development.
|
||||
|
||||
#### **Install Development Tools and Utilities**
|
||||
```bash
|
||||
sudo dnf groupinstall "Development Tools" -y
|
||||
sudo dnf install wget curl vim nano net-tools git unzip -y
|
||||
```
|
||||
- **Purpose**: Install necessary utilities and compilers to manage and build software.
|
||||
|
||||
#### **Firewall Configuration**
|
||||
```bash
|
||||
sudo firewall-cmd --permanent --add-port=6443/tcp # K3s API server port
|
||||
sudo firewall-cmd --permanent --add-port=10250/tcp # Kubelet port
|
||||
sudo firewall-cmd --reload
|
||||
```
|
||||
- **Purpose**: Allow required ports for K3s operations and secure the system by controlling network access.
|
||||
|
||||
---
|
||||
|
||||
### **3. Install Python and Set Up a Virtual Environment**
|
||||
|
||||
#### **Install Python 3 and Development Tools**
|
||||
```bash
|
||||
sudo dnf install python3 python3-pip python3-devel -y
|
||||
python3 --version
|
||||
pip3 --version
|
||||
```
|
||||
- **Purpose**: Install Python 3 and ensure it's set up for development.
|
||||
|
||||
#### **Set Up a Python Virtual Environment**
|
||||
```bash
|
||||
python3 -m venv ~/myenv
|
||||
source ~/myenv/bin/activate
|
||||
```
|
||||
- **Purpose**: Create and activate a virtual environment to manage Python packages independently.
|
||||
|
||||
---
|
||||
|
||||
### **4. Install and Configure K3s**
|
||||
|
||||
#### **Install K3s**
|
||||
```bash
|
||||
curl -sfL https://get.k3s.io | sh -
|
||||
sudo systemctl status k3s # Check K3s status
|
||||
```
|
||||
- **Purpose**: Install K3s and ensure the service is running.
|
||||
|
||||
#### **Configure K3s CLI**
|
||||
```bash
|
||||
sudo ln -s /usr/local/bin/kubectl /usr/bin/kubectl
|
||||
kubectl get nodes # Verify the node is running
|
||||
```
|
||||
- **Purpose**: Set up kubectl to interact with your K3s cluster.
|
||||
|
||||
---
|
||||
|
||||
### **5. Learn Bash Scripting through Automation**
|
||||
|
||||
Now that the basics are in place, you can learn shell scripting by automating common tasks. This reinforces your CLI skills while building useful scripts.
|
||||
|
||||
#### **Automate System Updates**
|
||||
Create a script to automate updates and log the process.
|
||||
|
||||
- **Script**: `update-system.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
LOGFILE=~/update-log.txt
|
||||
echo "Updating system..." | tee -a $LOGFILE
|
||||
sudo dnf update -y | tee -a $LOGFILE
|
||||
sudo dnf upgrade -y | tee -a $LOGFILE
|
||||
echo "System updated on $(date)" | tee -a $LOGFILE
|
||||
```
|
||||
- **Purpose**: Automate the update process and learn about logging and automation.
|
||||
|
||||
#### **Automate Python Environment Setup**
|
||||
- **Script**: `setup-python-env.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
ENV_NAME=$1
|
||||
|
||||
if [ -z "$ENV_NAME" ]; then
|
||||
echo "Usage: $0 <env_name>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Creating Python virtual environment '$ENV_NAME'..."
|
||||
python3 -m venv ~/$ENV_NAME
|
||||
|
||||
echo "Activating virtual environment..."
|
||||
source ~/$ENV_NAME/bin/activate
|
||||
|
||||
echo "Virtual environment '$ENV_NAME' created and activated."
|
||||
```
|
||||
- **Purpose**: Create a flexible script to automate Python environment setup, using arguments and learning bash conditionals.
|
||||
|
||||
#### **Automate K3s Installation and Configuration**
|
||||
- **Script**: `install-k3s.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
echo "Installing K3s..."
|
||||
curl -sfL https://get.k3s.io | sh -
|
||||
|
||||
if systemctl status k3s | grep "active (running)" > /dev/null; then
|
||||
echo "K3s is installed and running!"
|
||||
else
|
||||
echo "K3s installation failed."
|
||||
fi
|
||||
|
||||
echo "Configuring firewall..."
|
||||
sudo firewall-cmd --permanent --add-port=6443/tcp
|
||||
sudo firewall-cmd --permanent --add-port=10250/tcp
|
||||
sudo firewall-cmd --reload
|
||||
|
||||
echo "K3s and firewall configured successfully."
|
||||
```
|
||||
- **Purpose**: Automate the installation and configuration of K3s while practicing bash scripting with conditionals.
|
||||
|
||||
---
|
||||
|
||||
### **6. Monitor the System**
|
||||
|
||||
Create a script to monitor your system’s health. This teaches system commands and formatting output.
|
||||
|
||||
- **Script**: `monitor-resources.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
echo "System Resource Usage"
|
||||
echo "---------------------"
|
||||
echo "Disk Usage:"
|
||||
df -h
|
||||
echo "Memory Usage:"
|
||||
free -m
|
||||
echo "CPU Load:"
|
||||
uptime
|
||||
```
|
||||
- **Purpose**: Gather and display system metrics to monitor performance.
|
||||
|
||||
---
|
||||
|
||||
### **7. Learn Automation through Cron Jobs**
|
||||
|
||||
Use cron to automate tasks like system backups or updates.
|
||||
|
||||
#### **Example**: Schedule a Daily Backup of K3s Data
|
||||
- **Script**: `backup-k3s.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
BACKUP_DIR="/var/lib/rancher/k3s/server/db/backups"
|
||||
BACKUP_FILE="k3s-backup-$(date +%Y-%m-%d-%H-%M-%S).db"
|
||||
|
||||
echo "Backing up K3s data..."
|
||||
sudo cp $BACKUP_DIR/snapshot.db /backups/$BACKUP_FILE
|
||||
echo "Backup completed: /backups/$BACKUP_FILE"
|
||||
```
|
||||
|
||||
- **Automate with Cron**:
|
||||
```bash
|
||||
crontab -e
|
||||
```
|
||||
- Add the following line for daily backups at 2 am:
|
||||
```bash
|
||||
0 2 * * * /path/to/backup-k3s.sh
|
||||
```
|
||||
- **Purpose**: Learn how to schedule recurring tasks with cron and automate backups.
|
||||
|
||||
---
|
||||
|
||||
### **8. Secure the K3s Environment**
|
||||
|
||||
Focus on basic security configurations and policies as you learn.
|
||||
|
||||
#### **Create a Pod Security Policy**
|
||||
Write a basic Pod Security Policy (PSP) for K3s to enforce non-root container execution.
|
||||
|
||||
- **Example YAML**:
|
||||
```yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted-psp
|
||||
spec:
|
||||
privileged: false
|
||||
runAsUser:
|
||||
rule: MustRunAsNonRoot
|
||||
```
|
||||
- **Purpose**: Practice creating Kubernetes manifests and learn how to secure your environment.
|
||||
|
||||
---
|
||||
|
||||
### **Final Thoughts and Next Steps**
|
||||
|
||||
This clean, focused approach not only sets up your Rocky Linux system for K3s and Python-based workloads but also helps you gain **hands-on experience** with the Linux CLI, bash scripting, and automation through real-world tasks.
|
||||
|
||||
**What’s next?**
|
||||
- Continuously refine your scripts and make them more robust.
|
||||
- As you gain comfort, start incorporating more advanced Linux features (e.g., process management, networking).
|
||||
- Work towards automating the entire K3s environment, including monitoring, logging, and alerting.
|
||||
|
||||
---
|
||||
|
||||
You're absolutely right! Let’s include a few actionable examples to introduce more **advanced Linux features**, such as process management and networking, while keeping the learning process gradual and practical.
|
||||
|
||||
---
|
||||
|
||||
### **1. Process Management: Monitor and Control Running Processes**
|
||||
|
||||
Understanding how to manage processes is fundamental in Linux. Let’s create a couple of bash scripts that help you monitor and control processes on your system.
|
||||
|
||||
#### **Example: Monitor Active Processes**
|
||||
- **Script**: `monitor-processes.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
echo "Monitoring System Processes"
|
||||
echo "---------------------------"
|
||||
|
||||
echo "Top 10 CPU-consuming processes:"
|
||||
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -n 10
|
||||
|
||||
echo
|
||||
echo "Top 10 Memory-consuming processes:"
|
||||
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head -n 10
|
||||
```
|
||||
- **Learning Outcomes**:
|
||||
- Use `ps` to view processes and understand the options for filtering and sorting by CPU or memory usage.
|
||||
- Practice managing processes by identifying high-resource users.
|
||||
|
||||
#### **Example: Kill a Process by Name**
|
||||
- **Script**: `kill-process.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
if [ -z "$1" ]; then
|
||||
echo "Usage: $0 <process_name>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PROCESS_NAME=$1
|
||||
echo "Attempting to kill process: $PROCESS_NAME"
|
||||
|
||||
pkill $PROCESS_NAME && echo "Process '$PROCESS_NAME' terminated."
|
||||
```
|
||||
- **Learning Outcomes**:
|
||||
- Use `pkill` to terminate a process by name.
|
||||
- Understand how to handle input arguments in bash scripts.
|
||||
|
||||
---
|
||||
|
||||
### **2. Advanced Networking: Network Configuration and Monitoring**
|
||||
|
||||
Networking is key for edge compute workloads. Let’s dive into monitoring your network connections and controlling traffic with basic networking scripts.
|
||||
|
||||
#### **Example: Monitor Network Usage**
|
||||
- **Script**: `monitor-network.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
echo "Network Usage Report"
|
||||
echo "--------------------"
|
||||
|
||||
echo "Current network interfaces:"
|
||||
ip link show
|
||||
|
||||
echo "Network traffic summary:"
|
||||
ifconfig | grep -A 8 "eth0"
|
||||
|
||||
echo
|
||||
echo "Routing table:"
|
||||
netstat -rn
|
||||
```
|
||||
- **Learning Outcomes**:
|
||||
- Learn to use `ifconfig` and `ip` for network interface status.
|
||||
- Display routing information using `netstat`.
|
||||
|
||||
#### **Example: Block/Unblock IP Address Using Iptables**
|
||||
- **Script**: `block-ip.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
if [ -z "$1" ]; then
|
||||
echo "Usage: $0 <ip_address>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
IP_ADDRESS=$1
|
||||
echo "Blocking IP: $IP_ADDRESS"
|
||||
|
||||
sudo iptables -A INPUT -s $IP_ADDRESS -j DROP
|
||||
echo "IP address $IP_ADDRESS has been blocked."
|
||||
```
|
||||
|
||||
- **Unblock Script**: `unblock-ip.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
if [ -z "$1" ]; then
|
||||
echo "Usage: $0 <ip_address>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
IP_ADDRESS=$1
|
||||
echo "Unblocking IP: $IP_ADDRESS"
|
||||
|
||||
sudo iptables -D INPUT -s $IP_ADDRESS -j DROP
|
||||
echo "IP address $IP_ADDRESS has been unblocked."
|
||||
```
|
||||
|
||||
- **Learning Outcomes**:
|
||||
- Use `iptables` to block and unblock network traffic from specific IP addresses.
|
||||
- Learn basic network security by controlling traffic at the firewall level.
|
||||
|
||||
---
|
||||
|
||||
### **3. Automate Network Health Checks**
|
||||
|
||||
Automate network status checks by writing a script to ping external servers and display results. This is useful to test the connectivity from your edge devices.
|
||||
|
||||
#### **Example: Ping Multiple Servers**
|
||||
- **Script**: `network-health.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
SERVERS=("8.8.8.8" "1.1.1.1" "google.com" "example.com")
|
||||
|
||||
echo "Pinging Servers..."
|
||||
|
||||
for SERVER in "${SERVERS[@]}"; do
|
||||
echo "Pinging $SERVER:"
|
||||
ping -c 4 $SERVER
|
||||
echo "------------------------------------"
|
||||
done
|
||||
```
|
||||
- **Learning Outcomes**:
|
||||
- Use arrays in bash to handle multiple items (servers).
|
||||
- Learn about network connectivity testing using `ping`.
|
||||
|
||||
---
|
||||
|
||||
### **4. Advanced Disk Usage and Resource Monitoring**
|
||||
|
||||
As edge workloads may have limited resources, disk usage monitoring becomes crucial. Let’s script resource checks for disk space, memory, and CPU usage.
|
||||
|
||||
#### **Example: Monitor Disk Space and Report Critical Usage**
|
||||
- **Script**: `disk-usage-report.sh`
|
||||
```bash
|
||||
#!/bin/bash
|
||||
THRESHOLD=80
|
||||
|
||||
echo "Checking Disk Usage..."
|
||||
df -h | awk 'NR>1 {print $5 " " $1}' | while read output; do
|
||||
usep=$(echo $output | awk '{print $1}' | sed 's/%//')
|
||||
partition=$(echo $output | awk '{print $2}')
|
||||
if [ $usep -ge $THRESHOLD ]; then
|
||||
echo "CRITICAL: $partition is $usep% full."
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
- **Learning Outcomes**:
|
||||
- Use `df` to check disk usage.
|
||||
- Apply basic conditional logic to report when usage crosses a defined threshold (80%).
|
||||
|
||||
---
|
||||
|
||||
### **5. Advanced Automation with Cron Jobs**
|
||||
|
||||
After you’re comfortable with your scripts, you can automate more advanced monitoring and reporting using cron.
|
||||
|
||||
#### **Example: Automate Disk Monitoring**
|
||||
- **Automate Disk Usage Check**:
|
||||
```bash
|
||||
crontab -e
|
||||
```
|
||||
- Add this cron job to run the disk usage check script every day at 6 am:
|
||||
```bash
|
||||
0 6 * * * /path/to/disk-usage-report.sh
|
||||
```
|
||||
|
||||
- **Learning Outcomes**:
|
||||
- Learn how to use cron for recurring tasks.
|
||||
- Automate your monitoring and alerts, reducing manual checks.
|
||||
|
||||
---
|
||||
|
||||
### **Conclusion**
|
||||
|
||||
This approach adds **actionable examples** to help you dive deeper into advanced Linux features such as process management, networking, and resource monitoring, all while integrating them into your current workflow. These tweaks provide you with hands-on practice in managing and automating system processes, monitoring network activity, and ensuring system health—all key skills for managing K3s and edge compute workloads effectively.
|
||||
|
||||
---
|
||||
|
||||
# Learning Roadmap for K3s, Edge Computing, and Advanced Networking
|
||||
|
||||
## Networking Basics
|
||||
@@ -63,4 +481,4 @@
|
||||
- Disaster Recovery and Failover
|
||||
- **Real-World Projects and Scenarios**
|
||||
- Implementing a CI/CD Pipeline with K3s
|
||||
- Deploying a Multi-Service Application
|
||||
- Deploying a Multi-Service Application
|
||||
Reference in New Issue
Block a user