About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.
When multipathd logs checker failed path 8:16 in map mpatha, its path checker has decided that one route to a storage device is down, and it has told the kernel to stop using that path. multipath -ll then lists the path with the kernel state failed and the path status faulty. multipath only reports the fault. The cause is below it: a cable, switch port, network, target or adapter, or a checker that is timing out.
What the symptom means
The multipathd manual says the daemon "is in charge of checking for failed paths. When this happens, it will reconfigure the multipath map the path belongs to, so that this map regains its maximum performance and redundancy."
The log wording comes from the multipath-tools source code (release 0.9.4), not from a manual page. When the daemon fails a path it logs the format "checker failed path %s in map %s", where the first %s is the path's major:minor number (for example 8:16) and the second is the map alias. When a path comes back it logs "%s: reinstated", where %s is again the major:minor number.
The multipath.conf manual sets the rhythm. polling_interval is the "Interval between two path checks in seconds", default 5. For healthy paths the interval grows to max_polling_interval, default 4 times polling_interval. The default path_checker is tur, which issues a TEST UNIT READY command. The manual says directio "could cause spurious path failures under high load" and that increasing checker_timeout can help. A path that does not answer within checker_timeout seconds is "considered down".
The Ubuntu Server documentation describes the output format for each path as host:channel:id:lun devnode major:minor dm_status_if_known path_status online_status. It says "If the path is up and ready for I/O, the status of the path is ready or ghost. If the path is down, the status is faulty or shaky." The dm_status has two values: "failed, which is analogous to faulty, and active, which covers all other path states." An online_status of offline "means that the SCSI device has been disabled." A path in ghost is an up path, so it is not a fault.
What happens next depends on the other paths. With one good path left, I/O continues on it. When every path is down, the no_path_retry setting decides: its default is fail, so I/O fails at once, and a number means that many retries before queueing stops. Failed I/O can push a filesystem read-only, which is a separate fault with its own first step.
Likely causes
| Cause | What you see | How to confirm |
|---|---|---|
| A path really went down (cable, SFP, switch port, zoning or access list, target port, iSCSI session) | One or several paths failed together; the kernel log has link or session errors; failed paths share one adapter, NIC or target port | multipath -ll, then the kernel log; for iSCSI, iscsiadm --mode session |
| A flapping path | checker failed path and reinstated alternate in the log |
Count the pairs in the multipathd log over an hour |
| The checker times out under load | Paths fail at busy times and recover; path_checker is directio or the timeout is short |
multipathd -k"show config local" shows path_checker and checker_timeout |
| iSCSI NOP-Out settings too aggressive | Kernel log shows detected conn error (22) |
open-iscsi README: the values "may be too aggressive" for the network |
| The path was disabled on purpose | online_status is offline, or someone ran fail path |
multipathd -k"show paths"; ask who changed it |
Prove the cause before changing anything
All of these commands only read.
1. The topology.
multipath -ll
The manual says -ll shows the topology "from all available information (sysfs, the device mapper, path checkers ...)". For each map, find paths whose dm_status is failed or whose path_status is faulty or shaky. A good result is active ready running on every path, the form the Ubuntu documentation shows. Note the device name (for example sdb) and the major:minor pair (for example 8:16).
2. The daemon's own view.
multipathd -k"show paths"
multipathd -k"show status"
show paths lists the paths multipathd monitors and their state. show status gives "the number of path checkers in each possible state". The manual says there must be no whitespace between -k and the command, and that a command containing whitespace is quoted.
3. The log. Search the system log for checker failed path and reinstated. Note the time of the first failure, whether several paths failed at the same second, and how often the pairs repeat. The default verbosity is 2.
4. The kernel log.
dmesg --level=err,warn
The dmesg manual gives --level=err,warn as a way to show only those levels. Look for SCSI, link-down or session errors at the same time as the first failure.
5. iSCSI paths only.
iscsiadm --mode session
The iscsiadm manual lists this form to "List all sessions". Compare the sessions with the paths in multipath -ll. Do not change open-iscsi settings until you know which side failed.
6. The effective settings.
multipathd -k"show config local"
The manual says this shows "the currently used configuration" limited to devices present on the system. Read path_checker, checker_timeout, no_path_retry and polling_interval.
Fix
A real path loss
Repair the layer that the logs and the kernel messages point to: the cable, SFP or switch port, the zoning or access list, the target port, or the iSCSI network. multipath needs no change. The multipathd manual says the daemon reconfigures the map when a path fails, and the source logs reinstated when it puts a path back. Watch multipath -ll until the path returns.
If you need the daemon to take a repaired path back before its next check, first save the current path list:
multipathd -k"show paths" > /root/multipath-paths-before-2026-11-22.txt
Risk: the next command tells multipathd to put one failed path back into service. If the cause is not repaired, I/O is sent down a path that is still bad until the checker fails it again.
Reinstate the path, using the device name from show paths:
multipathd -k"reinstate path sdb"
The manual describes it as "Resumes path $path from failed state." To back out, fail the path again, only if your saved list showed it failed:
multipathd -k"fail path sdb"
A checker that times out under load
If show config local shows path_checker directio, the manual's note about spurious failures under high load applies. The default checker is tur. A higher checker_timeout is the manual's own suggestion for directio. Decide the new value with the storage owner, because a longer timeout delays the detection of a real failure.
First check whether the file exists, because the manual says it "is used to overwrite the built-in configuration table of multipathd"; if your host has no such file, skip the copy below. Write down what the listing reports:
ls -l /etc/multipath.conf > /root/multipath-conf-before-2026-11-22.txt 2>&1
cat /root/multipath-conf-before-2026-11-22.txt
If the file exists, save it:
cp -p /etc/multipath.conf /etc/multipath.conf.2026-11-22
If the listing shows no file, skip the copy and note that the file is absent; the back-out below then removes the file you create.
Disruptive step: the next edit changes the path-checker timeout in /etc/multipath.conf (or creates the file), and the command after it makes multipathd reread the configuration and reload every multipath device whose settings changed, on a host that may be serving live storage. Do it in a maintenance window.
In the defaults section of /etc/multipath.conf, set checker_timeout to the number of seconds agreed with the storage owner, for example checker_timeout 60 for 60 seconds. If the file was absent, create it with only a defaults section holding that one line. The manual says the devices section "can be used to override" the built-in device table and that attributes in the multipaths section "take precedence over all others", so a device entry that sets its own checker_timeout can win over the defaults value. Then apply it:
multipathd -k"reconfigure"
The manual says reconfigure "Rereads the configuration, and reloads all changed multipath devices." Read multipathd -k"show config local" again and confirm the new value. The Ubuntu documentation says: "Whenever multipath.conf is updated, initrd should be updated as well by running: update-initramfs -u -k all", because the file is copied to the RAM disk. That page's example is a root filesystem moved to a multipath device.
To back out, first test whether the saved copy exists:
ls -l /etc/multipath.conf.2026-11-22
If the listing shows the saved copy, put it back under its own name and reconfigure:
cp -p /etc/multipath.conf.2026-11-22 /etc/multipath.conf
multipathd -k"reconfigure"
Run the next block only if /root/multipath-conf-before-2026-11-22.txt shows No such file or directory, which means the file was absent before the change and you created it. Never run the rm when that file shows a listing of /etc/multipath.conf:
rm /etc/multipath.conf
multipathd -k"reconfigure"
Flapping or iSCSI timers
For a path that fails and recovers repeatedly, the multipath.conf manual describes shaky-path options such as san_path_err_threshold, which stop a path being reinstated for a time after repeated failures. Fix the unstable link first; these options hide the symptom and keep a path out of use. For iSCSI, the open-iscsi README says that if the kernel log has many detected conn error (22) messages, the NOP-Out values "may be too aggressive", and that you may need to "check your network for possible problems".
Example (illustrative): web01 has two paths to one LUN, sdb (8:16) and sdc (8:32), in map mpatha. At 01:40 the log shows checker failed path 8:16 in map mpatha. multipath -ll shows sdb failed and faulty and sdc active and ready. The kernel log shows a link-down on one adapter at 01:40, and sdb is the only path on it. The administrator replaces the SFP. At 02:05 the log shows 8:16: reinstated, and multipath -ll shows both paths active ready running.
Confirm the fix
multipath -llshowsactive ready runningfor every expected path in every map.multipathd -k"show status"reports no paths in a failed state.- The log shows
reinstatedfor the repaired path and no newchecker failed pathlines for an hour. - After a planned test, such as taking one path down in a window, I/O continues on the other path.
What to watch afterwards
- Repeated
checker failed pathandreinstatedpairs for one device, which point to an unstable link. - Paths that fail together, which point to a shared adapter, switch or target port.
- The number of working paths per map. A map on one path has lost its redundancy even when it looks healthy.
- Kernel messages about SCSI errors on the same disks.
Read next
Related articles you may want next.
- VMwareSoftware iSCSI Paths Dead After a NIC Change: Check the 1:1 Uplink Policy on Bound PortsESXi 8.0 software iSCSI paths dead after a NIC change? For same-subnet port binding, record each bound port's uplinks, logins, sessions and paths before changing anything.Read article
- TroubleshootingA Linux Filesystem Became Read-Only: Preserve the First ErrorSeparate a policy read-only mount from an error response and preserve the evidence needed for safe offline repair decisions.Read article
- VMwareLinux VM on VMware: "INFO: task blocked for more than 120 seconds" and what to check on the hostLinux guest logs 'task blocked for more than 120 seconds'? Read the message, then check esxtop latency, vmkernel.log, snapshots and vNVMe.Read article
Risks and limits
- These commands have not been run or lab-tested.
- multipath-tools 0.9.4 is the version of the manual pages used. Other releases can differ in defaults and in the log text.
- The manual warns that
queue_if_no_path"can lead to D state processes being hung and not killable in situations where all the paths to the LUN go offline", and advisesno_path_retryinstead. Changing queueing settings affects how long applications wait during an outage; set them with the application owner. - Reinstating a path does not repair it. The checker decides, and it fails the path again if it is still down.
Sources
- multipathd(8) — multipath-tools 0.9.4 manual page, Debian bookworm, source dated 2023-06-24, accessed 22 November 2026
- multipath(8) — multipath-tools 0.9.4 manual page, Debian bookworm, source dated 2023-06-24, accessed 22 November 2026
- multipath.conf(5) — multipath-tools 0.9.4 manual page, Debian bookworm, source dated 2023-06-24, accessed 22 November 2026
- Common multipath tasks and procedures — Ubuntu Server documentation, last updated June 26, 2026, accessed 22 November 2026
- multipathd/main.c at tag 0.9.4 — multipath-tools project source code, GitHub release tag 0.9.4, no date shown, accessed 22 November 2026
- README at tag 2.1.8 — open-iscsi project, GitHub release tag 2.1.8, header dated Jun 6, 2022, accessed 22 November 2026
- iscsiadm(8) — open-iscsi 2.1.8 manual page, Debian bookworm, 2022-10-19, accessed 22 November 2026
- dmesg(1) — util-linux 2.38.1 manual page, Debian bookworm, source dated 2024-11-21, accessed 22 November 2026
This article was originally published on .