nikon-coolscan-vfio-passthrough.md

📅 2026-07-06T19:53:07.403Z
👁️ 174 katselukertaa
🔓 Julkinen


# Running Nikon Scan 4 on Linux via VFIO FireWire Passthrough

Nikon Scan 4 (needed for full Digital ICE/ROC/GEM support on Coolscan film
scanners) is Windows/Mac only, and it drives the scanner through a proprietary
kernel-mode IEEE 1394 driver — so Wine cannot work, ever: Wine has no 1394 bus
stack and cannot load Windows hardware drivers. The reliable solution is passing
a dedicated FireWire PCIe card into a Windows VM with QEMU/KVM.

Tested with a Nikon Super Coolscan LS-4000 ED and a TI XIO2213B-based card on an
AMD B650 system; applies equally to the LS-50/5000/8000/9000 and other OHCI
FireWire cards.

**Placeholders** — replace with your values:

- `08:00.0` — PCI address of the FireWire OHCI controller
- `19` — its IOMMU group number
- `/home/user` — your home directory

## 1. Hardware notes

- A **dedicated PCIe FireWire card** is required (onboard FireWire usually shares
  an IOMMU group with too much else). TI chipsets have the best reputation (le citation needed).
- Cards based on the **TI XIO2213A/B** are internally a PCIe-to-PCI bridge
  (`104c:823e`) plus an OHCI function (`104c:823f`) behind it. Only the OHCI
  function is passed through; the bridge stays on the host. The OHCI function is
  INTx-only (no MSI) — relevant for troubleshooting.
- Prefer a CPU-attached slot if one is free; chipset slots work too but are more
  likely to hit the firmware issue in §8.

## 2. BIOS/UEFI

Enable:

1. CPU virtualization — Intel *VT-x* / AMD *SVM*.
2. IOMMU — Intel *VT-d* / AMD *IOMMU* (set *Enabled*, not *Auto*).

## 3. Kernel command line

Add `intel_iommu=on iommu=pt` (Intel) or just `iommu=pt` (AMD; the IOMMU is on by
default once enabled in BIOS). Where to put it:

- GRUB: `GRUB_CMDLINE_LINUX_DEFAULT` in `/etc/default/grub`, then regenerate
  (`grub-mkconfig -o /boot/grub/grub.cfg` or `update-grub`).
- systemd-boot: the `options` line in `/boot/loader/entries/*.conf`.
- UKI: `/etc/kernel/cmdline`, then `mkinitcpio -P` (or your distro's equivalent).

Verify after reboot:

```sh
sudo dmesg | grep -i -e dmar -e amd-vi
# Intel: "DMAR: IOMMU enabled"  AMD: "AMD-Vi: Interrupt remapping enabled"
ls /sys/kernel/iommu_groups/    # should be populated
```

## 4. Identify the card and check its IOMMU group

```sh
lspci -nn | grep -i -e 1394 -e texas
ls /sys/bus/pci/devices/0000:08:00.0/iommu_group/devices/
cat /sys/bus/pci/devices/0000:08:00.0/iommu_group/reserved_regions
```

- Note the address of the **OHCI function** (class `FireWire (IEEE 1394)`,
  e.g. `104c:823f`) — not the bridge.
- The group should contain only the card (and its bridge, plus possibly the
  PCIe port above it). Unrelated devices in the group → try another slot.
- Any line ending in `direct` in `reserved_regions` → see §8 before proceeding.

## 5. Runtime rebind scripts

Binding at runtime (instead of at boot) lets the card flip between the host
(SANE/VueScan) and the VM on demand.

`/usr/local/bin/fw-to-vm`:

```sh
#!/bin/sh
set -eu
DEV=0000:08:00.0
modprobe vfio-pci
[ -e "/sys/bus/pci/devices/$DEV/driver" ] && \
  echo "$DEV" > "/sys/bus/pci/devices/$DEV/driver/unbind"
echo vfio-pci > "/sys/bus/pci/devices/$DEV/driver_override"
echo "$DEV" > /sys/bus/pci/drivers_probe
```

`/usr/local/bin/fw-to-host`:

```sh
#!/bin/sh
set -eu
DEV=0000:08:00.0
[ -e "/sys/bus/pci/devices/$DEV/driver" ] && \
  echo "$DEV" > "/sys/bus/pci/devices/$DEV/driver/unbind"
echo "" > "/sys/bus/pci/devices/$DEV/driver_override"
modprobe firewire-ohci
echo "$DEV" > /sys/bus/pci/drivers_probe
```

`chmod +x` both. Power the scanner on **before** starting the VM.

## 6. The VM

Windows 7 SP1 **32-bit** is the sweet spot: VFIO-friendly, Nikon Scan-compatible,
and no x64 driver-signing hassle. Guest hardware is deliberately boring — IDE
disk and e1000 NIC need no extra drivers, and performance is irrelevant for a
scanning appliance.

Packages needed: QEMU with graphical display support (Arch: `qemu-desktop`;
Debian/Ubuntu: `qemu-system-x86`), `samba` (for the built-in `smb=` share),
`virt-viewer` (for SPICE).

```sh
mkdir -p /home/user/vm /home/user/scans
qemu-img create -f qcow2 /home/user/vm/win7.qcow2 40G
```

Install Windows first, *without* the passthrough device (add
`-cdrom win7-sp1-x86.iso` to the command below and drop the `vfio-pci` line).
Then run day-to-day as `/home/user/vm/run-scanner-vm.sh`:

I used Windows 7 Enterprise SP1 from the [Internet Archive](https://archive.org/details/en_windows_7_enterprise_with_sp1_x86_dvd_u_677710_20240515-Archive).

```sh
#!/bin/sh
set -eu
sudo /usr/local/bin/fw-to-vm

sudo qemu-system-x86_64 \
  -machine pc,accel=kvm \
  -cpu host,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time \
  -smp 2 -m 4096 \
  -drive file=/home/user/vm/win7.qcow2,if=ide,media=disk \
  -netdev user,id=n0,smb=/home/user/scans \
  -device e1000,netdev=n0 \
  -device vfio-pci,host=08:00.0 \
  -device qxl-vga,vgamem_mb=32 \
  -display spice-app \
  -usb -device usb-tablet \
  -rtc base=localtime

sudo /usr/local/bin/fw-to-host
```

Notes:

- `smb=` exposes the host folder at `\\10.0.2.4\qemu` in the guest — map it as a
  network drive and point Nikon Scan's output there. QEMU spawns its own private
  smbd; no samba configuration needed.
- QXL + SPICE gives arbitrary resolutions, live desktop resize, and host↔guest
  clipboard — install **spice-guest-tools** (spice-space.org; last releases
  support Win7) in the guest. Fallback: `-device VGA,vgamem_mb=32 -display gtk`.
- Sudo is the simple way to get `/dev/vfio` access. To run unprivileged: udev
  rule granting your user the `/dev/vfio/*` nodes plus a `memlock` limit above
  guest RAM (VFIO pins it all).
- In the guest: disable Windows Update and sleep (a suspend mid-scan wedges the
  FireWire bus), set the High Performance power plan.

## 7. Inside the guest

1. Device Manager shows a *TI OHCI Compliant IEEE 1394 Host Controller* using
   the inbox driver — no vendor driver needed.
2. Power on the scanner; it appears under *Imaging devices*.
3. Install **Nikon Scan 4.0.3**. Its installer includes the scanner driver; the
   device re-enumerates under its proper name. If not, update the driver
   manually and point it at the Nikon Scan directory.
4. ICE/ROC/GEM appear in the Nikon Scan tool palette.

Nikon Scan 4.0.3 is available on the [Internet Archive](https://archive.org/details/nikon-scan-403)

## 8. Troubleshooting

**`failed to setup container for group N: Invalid argument`**, dmesg shows
*"Firmware has requested this device have a 1:1 IOMMU mapping"*.
The firmware publishes IVMD/RMRR unity-mapping entries (the `direct` lines in
`reserved_regions`) for the port your card sits behind, usually as part of a
"Kernel DMA Protection" feature; kernels ≥ 6.8 strictly refuse to attach such
devices to VFIO. Fixes, in order: (a) look for a BIOS toggle — on Gigabyte AM5
boards, *Settings → AMD CBS → NBIO Common Options*, set *IOMMU* from *Auto* to
*Enabled*, which reveals *Kernel DMA Protection Indicator* → *Disabled*;
(b) move the card to a CPU-attached slot; (c) BIOS update (these entries arrived
with AGESA updates). Verify: the `direct` lines disappear from
`reserved_regions`. Group numbers may change after firmware changes.

Disabling Kernel DMA Protection is a security (not stability) trade-off aimed at
hot-pluggable DMA ports like Thunderbolt; on a desktop without such ports, and
with the IOMMU still fully active, the practical loss is minimal.

**Harmless warnings at VM start:**
`BAR 0: failed to create dma-buf ...` — a P2P-DMA/display mechanism irrelevant
to OHCI; QEMU falls back to normal BAR mmap.
`XDG_RUNTIME_DIR is invalid or not set` — artifact of running QEMU under sudo.

**INTx error at start, or scans stall under load.** The OHCI function is
INTx-only; check `/proc/interrupts` for the IRQ line being shared with an active
host device, and move the card to another slot if so.

**Scanner not detected in guest.** Power-cycle the scanner with the VM running
(FireWire hotplug works through passthrough). Confirm the card is actually on
vfio-pci: `readlink /sys/bus/pci/devices/0000:08:00.0/driver`.

**Host FireWire dead after VM shutdown.** Run `fw-to-host`; if the rebind
fails:

```sh
echo 1 | sudo tee /sys/bus/pci/devices/0000:08:00.0/remove
echo 1 | sudo tee /sys/bus/pci/rescan
```