dm-multipath low performance with blk-mq

Mike Snitzer snitzer at redhat.com
Thu Feb 4 06:44:12 PST 2016


On Thu, Feb 04 2016 at  9:32am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 02/04/2016 03:09 PM, Mike Snitzer wrote:
> > On Thu, Feb 04 2016 at  8:58am -0500,
> > Hannes Reinecke <hare at suse.de> wrote:
> > 
> >> On 02/04/2016 02:54 PM, Mike Snitzer wrote:
> >>> On Thu, Feb 04 2016 at  1:54am -0500,
> >>> Hannes Reinecke <hare at suse.de> wrote:
> >>>
> >> [ .. ]
> >>>> But anyway, I'll be looking at your patches.
> >>>
> >>> Thanks, sadly none of the patches are going to fix the performance
> >>> problems but I do think they are a step forward.
> >>>
> >> Hmm. I've got a slew of patches converting dm-mpath to use atomic_t
> >> and bitops; with that we should be able to move to rcu for path
> >> lookup and do away with most of the locking.
> >> Quite raw, though; drop me a mail if you're interested.
> > 
> > Hmm, ok I just switched m->lock from spinlock_t to rwlock_t, see:
> > https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=a5226e23a6958ac9b7ade13a983604c43d232c7d
> > 
> > So any patch you'd have in this area would need rebasing.  I'll gladly
> > look at what you have (even if it isn't rebased).  So yes please share.
> > 
> > (it could be that there isn't a big enough win associated with switching
> > to rwlock_t -- that we could get away without doing that particular
> > churn.. open to that if you think rwlock_t pointless given we'll take
> > the write lock after repeat_count drops to 0)
> > 
> personally, I don't think the switching to a rwlock_t will buy us
> anything; for a decent performance you have to set rq_min_io to 1
> anyway, thereby defeating the purpose of the rwlock.

OK, I'll drop the rwlock_t commit.
 
> My thinking was rather a different direction:
> Move the crucial bits of the multipath structure to atomics, and
> split off the path selection code into one bit for selecting the
> path within a path group, and another which switches the path groups.
> When we do that we could use rcus for the paths themselves, and
> would only need to take the spinlock if we need to switch path
> groups. Which should be okay as switching path groups is
> (potentially) a rather slow operation anyway.

If you were to put focus to dusting your work off and helping me get
switched over to RCU I'd be very thankful.



More information about the Linux-nvme mailing list