Update tech_docs/linux/pci_passthrough.md
This commit is contained in:
@@ -1,3 +1,164 @@
|
||||
# Simple PCI(e) Passthrough Setup Guide
|
||||
|
||||
## What is PCI(e) Passthrough?
|
||||
PCI(e) passthrough lets you give a virtual machine (VM) direct control over a physical device like a graphics card or network card. The VM gets native performance, but the host can't use that device anymore.
|
||||
|
||||
## Before You Start - Check These Requirements
|
||||
|
||||
### 1. Hardware Check
|
||||
- **CPU**: Must support IOMMU (Intel VT-d or AMD-Vi)
|
||||
- **Motherboard**: Must support IOMMU
|
||||
- **Device**: The PCI device you want to pass through
|
||||
|
||||
### 2. Quick Compatibility Test
|
||||
Run this command to see if IOMMU is working:
|
||||
```bash
|
||||
dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
|
||||
```
|
||||
You should see something like "DMAR: IOMMU enabled" or "AMD-Vi: Interrupt remapping enabled"
|
||||
|
||||
## Step-by-Step Setup
|
||||
|
||||
### Step 1: Enable IOMMU in BIOS
|
||||
1. Reboot and enter BIOS/UEFI settings
|
||||
2. Look for and enable:
|
||||
- **Intel**: "VT-d" or "Intel Virtualization Technology for Directed I/O"
|
||||
- **AMD**: "AMD-Vi" or "IOMMU" (often enabled by default)
|
||||
3. Save and exit
|
||||
|
||||
### Step 2: Enable IOMMU in Linux (Intel only)
|
||||
If you have an Intel CPU with older kernel, add this to your kernel command line:
|
||||
```
|
||||
intel_iommu=on
|
||||
```
|
||||
|
||||
**Optional performance boost**: Add this for both Intel and AMD:
|
||||
```
|
||||
iommu=pt
|
||||
```
|
||||
|
||||
### Step 3: Load Required Kernel Modules
|
||||
Add these lines to `/etc/modules`:
|
||||
```
|
||||
vfio
|
||||
vfio_iommu_type1
|
||||
vfio_pci
|
||||
```
|
||||
|
||||
Then update initramfs:
|
||||
```bash
|
||||
update-initramfs -u -k all
|
||||
```
|
||||
|
||||
### Step 4: Find Your Device
|
||||
List all PCI devices:
|
||||
```bash
|
||||
lspci -nn
|
||||
```
|
||||
|
||||
Look for your device and note its ID (like `01:00.0`) and vendor:device codes (like `[10de:1d01]`).
|
||||
|
||||
### Step 5: Reserve Device for Passthrough
|
||||
Create `/etc/modprobe.d/vfio.conf` and add:
|
||||
```
|
||||
options vfio-pci ids=VENDOR:DEVICE
|
||||
```
|
||||
|
||||
Replace `VENDOR:DEVICE` with your actual codes (e.g., `10de:1d01` for an NVIDIA GPU).
|
||||
|
||||
**Alternative method** - Blacklist the host driver:
|
||||
```bash
|
||||
# For NVIDIA GPUs
|
||||
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
|
||||
|
||||
# For AMD GPUs
|
||||
echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
|
||||
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
|
||||
|
||||
# For Intel integrated graphics
|
||||
echo "blacklist i915" >> /etc/modprobe.d/blacklist.conf
|
||||
```
|
||||
|
||||
### Step 6: Update and Reboot
|
||||
```bash
|
||||
update-initramfs -u -k all
|
||||
reboot
|
||||
```
|
||||
|
||||
### Step 7: Verify Setup
|
||||
Check that your device is ready:
|
||||
```bash
|
||||
lspci -nnk
|
||||
```
|
||||
|
||||
Look for your device - it should show `Kernel driver in use: vfio-pci` or no driver listed.
|
||||
|
||||
### Step 8: Add Device to VM
|
||||
**Via Web Interface:**
|
||||
1. Go to your VM's Hardware tab
|
||||
2. Click "Add" → "PCI Device"
|
||||
3. Select your device
|
||||
4. For GPUs, check "Primary GPU" and "PCIe" options
|
||||
|
||||
**Via Command Line:**
|
||||
```bash
|
||||
qm set VMID -hostpci0 01:00.0,pcie=on,x-vga=on
|
||||
```
|
||||
|
||||
## GPU-Specific Tips
|
||||
|
||||
### For Best GPU Compatibility:
|
||||
- Use **q35** machine type
|
||||
- Use **OVMF (UEFI)** instead of SeaBIOS if your GPU supports it
|
||||
- Enable **PCIe** mode instead of PCI
|
||||
|
||||
### Common GPU Issues:
|
||||
|
||||
**Black screen in VM console**: This is normal! Connect a monitor directly to the GPU or use remote desktop software inside the VM.
|
||||
|
||||
**Error 43 (NVIDIA)**: Try these kernel parameters:
|
||||
```bash
|
||||
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
|
||||
```
|
||||
|
||||
**Audio crackling**: Enable MSI in the guest:
|
||||
```bash
|
||||
echo "options snd-hda-intel enable_msi=1" >> /etc/modprobe.d/snd-hda-intel.conf
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Device Not Available
|
||||
- Check IOMMU groups: `pvesh get /nodes/NODENAME/hardware/pci --pci-class-blacklist ""`
|
||||
- Try moving the card to a different PCI slot
|
||||
- Ensure the device isn't being used by the host
|
||||
|
||||
### System Won't Boot
|
||||
If you have issues after changes:
|
||||
1. Boot from recovery
|
||||
2. Remove the problematic config files
|
||||
3. Run `update-initramfs -u -k all`
|
||||
4. Reboot
|
||||
|
||||
### Still Not Working?
|
||||
- Update your motherboard BIOS to the latest version
|
||||
- Check if your specific hardware combination is supported
|
||||
- Consider trying the ACS override patch (advanced users only)
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
| Task | Command |
|
||||
|------|---------|
|
||||
| Check IOMMU status | `dmesg \| grep -e DMAR -e IOMMU` |
|
||||
| List PCI devices | `lspci -nn` |
|
||||
| Check device driver | `lspci -nnk` |
|
||||
| Update initramfs | `update-initramfs -u -k all` |
|
||||
| Add device to VM | `qm set VMID -hostpci0 01:00.0` |
|
||||
|
||||
Remember: After any module or kernel parameter changes, always run `update-initramfs -u -k all` and reboot!
|
||||
|
||||
---
|
||||
|
||||
PCI(e) Passthrough
|
||||
Jump to navigationJump to search
|
||||
|
||||
|
||||
Reference in New Issue
Block a user