Found this handy tonight, multipathd has an interactive shell you can use by running “multipathd -k” which is a flag not documented in the man page. Actually, no flags are documented in the man page. You can get a list of the command options by typing help (or generally any command it can’t figure out what to do with). That lead me to the “fail path” option which is what I’m trying to do here, test individual path failures without having to go through the 5 step java gui dance with my fibre switches.
[root@nas ~]# multipathd -k
multipathd> show paths
hcil dev dev_t pri dm_st chk_st next_check
0:2:0:0 sda 8:0 1 [undef] [ready] [orphan]
1:0:0:1 sdb 8:16 1 [active][ready] XXXXXXXX.. 16/20
2:0:0:1 sdd 8:48 1 [active][ready] XXXXX….. 11/20
yes as you can see I clearly don’t have my blacklist setup right. Now to fail a path its “fail path $path” where $path is… again, no documentation. Turns out what they want is the “dev” column (second one)
multipathd> fail path sdd
ok
multipathd> show paths
hcil dev dev_t pri dm_st chk_st next_check
0:2:0:0 sda 8:0 1 [undef] [ready] [orphan]
1:0:0:1 sdb 8:16 1 [active][ready] XXXXX….. 10/20
2:0:0:1 sdd 8:48 1 [failed][faulty] X……… 3/20
Interesting to note, multipathd seems pretty good at its job, as the path is re-enabled within seconds when the checker notices its fine.
Sep 30 00:51:35 nas multipathd: mpath1: fail path 8:48 (operator)
Sep 30 00:51:35 nas kernel: device-mapper: multipath: Failing path 8:48.
Sep 30 00:51:35 nas multipathd: dm-2: add map (uevent)
Sep 30 00:51:35 nas multipathd: dm-2: devmap already registered
Sep 30 00:51:35 nas multipathd: 8:48: mark as failed
Sep 30 00:51:35 nas multipathd: mpath1: remaining active paths: 1
Sep 30 00:51:40 nas multipathd: sdd: readsector0 checker reports path is up
Sep 30 00:51:40 nas multipathd: 8:48: reinstated
Sep 30 00:51:40 nas multipathd: mpath1: remaining active paths: 2
Sep 30 00:51:40 nas multipathd: dm-2: add map (uevent)
Sep 30 00:51:40 nas multipathd: dm-2: devmap already registered
So the good news is failing one path and then the other worked nicely and the server didn’t freak out and remount the filesystem read-only.