Update tech_docs/linux/pci_passthrough.md
This commit is contained in:
@@ -1,3 +1,164 @@
|
|||||||
|
# Simple PCI(e) Passthrough Setup Guide
|
||||||
|
|
||||||
|
## What is PCI(e) Passthrough?
|
||||||
|
PCI(e) passthrough lets you give a virtual machine (VM) direct control over a physical device like a graphics card or network card. The VM gets native performance, but the host can't use that device anymore.
|
||||||
|
|
||||||
|
## Before You Start - Check These Requirements
|
||||||
|
|
||||||
|
### 1. Hardware Check
|
||||||
|
- **CPU**: Must support IOMMU (Intel VT-d or AMD-Vi)
|
||||||
|
- **Motherboard**: Must support IOMMU
|
||||||
|
- **Device**: The PCI device you want to pass through
|
||||||
|
|
||||||
|
### 2. Quick Compatibility Test
|
||||||
|
Run this command to see if IOMMU is working:
|
||||||
|
```bash
|
||||||
|
dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
|
||||||
|
```
|
||||||
|
You should see something like "DMAR: IOMMU enabled" or "AMD-Vi: Interrupt remapping enabled"
|
||||||
|
|
||||||
|
## Step-by-Step Setup
|
||||||
|
|
||||||
|
### Step 1: Enable IOMMU in BIOS
|
||||||
|
1. Reboot and enter BIOS/UEFI settings
|
||||||
|
2. Look for and enable:
|
||||||
|
- **Intel**: "VT-d" or "Intel Virtualization Technology for Directed I/O"
|
||||||
|
- **AMD**: "AMD-Vi" or "IOMMU" (often enabled by default)
|
||||||
|
3. Save and exit
|
||||||
|
|
||||||
|
### Step 2: Enable IOMMU in Linux (Intel only)
|
||||||
|
If you have an Intel CPU with older kernel, add this to your kernel command line:
|
||||||
|
```
|
||||||
|
intel_iommu=on
|
||||||
|
```
|
||||||
|
|
||||||
|
**Optional performance boost**: Add this for both Intel and AMD:
|
||||||
|
```
|
||||||
|
iommu=pt
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Load Required Kernel Modules
|
||||||
|
Add these lines to `/etc/modules`:
|
||||||
|
```
|
||||||
|
vfio
|
||||||
|
vfio_iommu_type1
|
||||||
|
vfio_pci
|
||||||
|
```
|
||||||
|
|
||||||
|
Then update initramfs:
|
||||||
|
```bash
|
||||||
|
update-initramfs -u -k all
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 4: Find Your Device
|
||||||
|
List all PCI devices:
|
||||||
|
```bash
|
||||||
|
lspci -nn
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for your device and note its ID (like `01:00.0`) and vendor:device codes (like `[10de:1d01]`).
|
||||||
|
|
||||||
|
### Step 5: Reserve Device for Passthrough
|
||||||
|
Create `/etc/modprobe.d/vfio.conf` and add:
|
||||||
|
```
|
||||||
|
options vfio-pci ids=VENDOR:DEVICE
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace `VENDOR:DEVICE` with your actual codes (e.g., `10de:1d01` for an NVIDIA GPU).
|
||||||
|
|
||||||
|
**Alternative method** - Blacklist the host driver:
|
||||||
|
```bash
|
||||||
|
# For NVIDIA GPUs
|
||||||
|
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
|
||||||
|
|
||||||
|
# For AMD GPUs
|
||||||
|
echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
|
||||||
|
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
|
||||||
|
|
||||||
|
# For Intel integrated graphics
|
||||||
|
echo "blacklist i915" >> /etc/modprobe.d/blacklist.conf
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 6: Update and Reboot
|
||||||
|
```bash
|
||||||
|
update-initramfs -u -k all
|
||||||
|
reboot
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 7: Verify Setup
|
||||||
|
Check that your device is ready:
|
||||||
|
```bash
|
||||||
|
lspci -nnk
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for your device - it should show `Kernel driver in use: vfio-pci` or no driver listed.
|
||||||
|
|
||||||
|
### Step 8: Add Device to VM
|
||||||
|
**Via Web Interface:**
|
||||||
|
1. Go to your VM's Hardware tab
|
||||||
|
2. Click "Add" → "PCI Device"
|
||||||
|
3. Select your device
|
||||||
|
4. For GPUs, check "Primary GPU" and "PCIe" options
|
||||||
|
|
||||||
|
**Via Command Line:**
|
||||||
|
```bash
|
||||||
|
qm set VMID -hostpci0 01:00.0,pcie=on,x-vga=on
|
||||||
|
```
|
||||||
|
|
||||||
|
## GPU-Specific Tips
|
||||||
|
|
||||||
|
### For Best GPU Compatibility:
|
||||||
|
- Use **q35** machine type
|
||||||
|
- Use **OVMF (UEFI)** instead of SeaBIOS if your GPU supports it
|
||||||
|
- Enable **PCIe** mode instead of PCI
|
||||||
|
|
||||||
|
### Common GPU Issues:
|
||||||
|
|
||||||
|
**Black screen in VM console**: This is normal! Connect a monitor directly to the GPU or use remote desktop software inside the VM.
|
||||||
|
|
||||||
|
**Error 43 (NVIDIA)**: Try these kernel parameters:
|
||||||
|
```bash
|
||||||
|
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
|
||||||
|
```
|
||||||
|
|
||||||
|
**Audio crackling**: Enable MSI in the guest:
|
||||||
|
```bash
|
||||||
|
echo "options snd-hda-intel enable_msi=1" >> /etc/modprobe.d/snd-hda-intel.conf
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Device Not Available
|
||||||
|
- Check IOMMU groups: `pvesh get /nodes/NODENAME/hardware/pci --pci-class-blacklist ""`
|
||||||
|
- Try moving the card to a different PCI slot
|
||||||
|
- Ensure the device isn't being used by the host
|
||||||
|
|
||||||
|
### System Won't Boot
|
||||||
|
If you have issues after changes:
|
||||||
|
1. Boot from recovery
|
||||||
|
2. Remove the problematic config files
|
||||||
|
3. Run `update-initramfs -u -k all`
|
||||||
|
4. Reboot
|
||||||
|
|
||||||
|
### Still Not Working?
|
||||||
|
- Update your motherboard BIOS to the latest version
|
||||||
|
- Check if your specific hardware combination is supported
|
||||||
|
- Consider trying the ACS override patch (advanced users only)
|
||||||
|
|
||||||
|
## Quick Reference Commands
|
||||||
|
|
||||||
|
| Task | Command |
|
||||||
|
|------|---------|
|
||||||
|
| Check IOMMU status | `dmesg \| grep -e DMAR -e IOMMU` |
|
||||||
|
| List PCI devices | `lspci -nn` |
|
||||||
|
| Check device driver | `lspci -nnk` |
|
||||||
|
| Update initramfs | `update-initramfs -u -k all` |
|
||||||
|
| Add device to VM | `qm set VMID -hostpci0 01:00.0` |
|
||||||
|
|
||||||
|
Remember: After any module or kernel parameter changes, always run `update-initramfs -u -k all` and reboot!
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
PCI(e) Passthrough
|
PCI(e) Passthrough
|
||||||
Jump to navigationJump to search
|
Jump to navigationJump to search
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user