So your WS will magically follow different rules than his... uh huh... yes, because the damage formulas apply differently to you for some reason.
You're right on the verge of fighting Toramas on your Samurai. Those tend to need a sizable chunk of extra accuracy compared to most other mobs.
Hell, look at it this way.
Haubergeon + Life Belt = 5 STR, 5 DEX, 12 att, 22 acc in two slots. Good in many situations. Life saver against Torama.
Byrnie + Swordbelt+1 = 3 STR, -3 DEX, 32 att, "-1" acc in those two slots. Good against VT range mobs where you don't need as much acc to hit... but you lose a little STR too.
Byrnie + Life Belt = 3 STR, -3 DEX, 20 att, 9 acc in those two slots, compare to
Haubergeon + Swordbelt+1 = 5 STR, 5 DEX, 24 att, 10 acc in those two slots.
BAM SHAZAM, we have a clear winner out of those four combinations. Ruh roh...