This is a pet peeve of mine as well.
I think that one of the keys to solving this lil prob is proper communication and teamwork.
1) If you're relatively new to a camp, without a map(coffer or lengthy quest only excuse, really), and fairly sure you're gonna get lost,
TELL PEOPLE; not once you're at the zone, not after you're already lost,
IMMEDIATELY.
2) If Step 1 is followed, make allowances for their getting there, i.e., meet em at the zone, give em some detailed instructions in a /tell, etc.
3) If you're moving through an aggro area and you need to recast/reapply a sneak/oil/etc., again . . .
TELL PEOPLE. Among my circle of friends a simple "s" or "i" is more than sufficient.
4) If Step 3 is followed,
STOP MOVING and allow the caster an opportunity to keep up. Don't automatically assume that everyone knows where they're going and remain mindful that by moving as a group, aggro may be an opportunity for xp instead of 10+ minutes' worth of downtime
All that being said, though, I still have very little patience for someone who comes across as wanting their hand held on the way to camp.