Is multipath essential on Server 20.04?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Is multipath essential on Server 20.04?

Liam Proven
I have a little Raspberry Pi 4 box serving up a 5TB ZFS array. I often
see multiple instances of `multipathd` running. Right now, I have 7
copies, with 273MB of virtual memory each.

I do not use iSCSI or any kind of SAN, just ZFS, Samba and Netatalk.
Can I safely uninstall multipath-tools?

--
Liam Proven – Profile: https://about.me/liamproven
Email: [hidden email] – gMail/gTalk/gHangouts: [hidden email]
Twitter/Facebook/LinkedIn/Flickr: lproven – Skype: liamproven
UK: +44 7939-087884 – ČR (+ WhatsApp/Telegram/Signal): +420 702 829 053

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Is multipath essential on Server 20.04?

Oliver Grawert
hi,
Am Freitag, den 02.10.2020, 16:32 +0200 schrieb Liam Proven:
> I have a little Raspberry Pi 4 box serving up a 5TB ZFS array. I
> often
> see multiple instances of `multipathd` running. Right now, I have 7
> copies, with 273MB of virtual memory each.
>
> I do not use iSCSI or any kind of SAN, just ZFS, Samba and Netatalk.
> Can I safely uninstall multipath-tools?
>

$ apt-cache show multipath-tools|grep Task
Task: server, cloud-image

seems to only be in the server task (not ubuntu-minimal or ubuntu-
standard) so it should be safe to remove if you dont use it ...

ciao
        oli

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is multipath essential on Server 20.04?

Chris Green
In reply to this post by Liam Proven
On Fri, Oct 02, 2020 at 04:32:35PM +0200, Liam Proven wrote:
> I have a little Raspberry Pi 4 box serving up a 5TB ZFS array. I often
> see multiple instances of `multipathd` running. Right now, I have 7
> copies, with 273MB of virtual memory each.
>
> I do not use iSCSI or any kind of SAN, just ZFS, Samba and Netatalk.
> Can I safely uninstall multipath-tools?
>
I have a pi 4 running as a nas/backup, no multipath processes at all.

In fact multipath-tools isn't even installed.  My system is headless
though and I installed the 'lite' image with no desktop.

--
Chris Green

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Is multipath essential on Server 20.04?

Liam Proven
In reply to this post by Oliver Grawert
On Fri, 2 Oct 2020 at 16:41, Oliver Grawert <[hidden email]> wrote:
>
> $ apt-cache show multipath-tools|grep Task
> Task: server, cloud-image
>
> seems to only be in the server task (not ubuntu-minimal or ubuntu-
> standard) so it should be safe to remove if you dont use it ...

Great. Thank you!

--
Liam Proven – Profile: https://about.me/liamproven
Email: [hidden email] – gMail/gTalk/gHangouts: [hidden email]
Twitter/Facebook/LinkedIn/Flickr: lproven – Skype: liamproven
UK: +44 7939-087884 – ČR (+ WhatsApp/Telegram/Signal): +420 702 829 053

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Is multipath essential on Server 20.04?

Liam Proven
In reply to this post by Chris Green
On Fri, 2 Oct 2020 at 16:47, Chris Green <[hidden email]> wrote:
> >
> I have a pi 4 running as a nas/backup,

That's what this one does, yes.

> no multipath processes at all.

Oh! That's odd.

> In fact multipath-tools isn't even installed.  My system is headless
> though and I installed the 'lite' image with no desktop.

This one is headless too but I started with the Server image.

--
Liam Proven – Profile: https://about.me/liamproven
Email: [hidden email] – gMail/gTalk/gHangouts: [hidden email]
Twitter/Facebook/LinkedIn/Flickr: lproven – Skype: liamproven
UK: +44 7939-087884 – ČR (+ WhatsApp/Telegram/Signal): +420 702 829 053

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
R C
Reply | Threaded
Open this post in threaded view
|

Re: Is multipath essential on Server 20.04?

R C
In reply to this post by Liam Proven
multipath is needed for (optimizing) io when there are virtual storage
devices that use multiple hardware/actual devices (hence multipath), for
example when you are using lustre, scality, san, jbod, ZFS or 'any array'

since you are using ZFS, in/on an array, it would be a good idea to use it.

You don't NEED to use it, it only makes io more efficient and probably
faster.

Also, you can shut down some of the cores on your RPI cpu, you DON'T
really need those either.


Ron


On 10/2/20 10:23 AM, Liam Proven wrote:
> On Fri, 2 Oct 2020 at 16:41, Oliver Grawert <[hidden email]> wrote:
>> $ apt-cache show multipath-tools|grep Task
>> Task: server, cloud-image
>>
>> seems to only be in the server task (not ubuntu-minimal or ubuntu-
>> standard) so it should be safe to remove if you dont use it ...
> Great. Thank you!
>

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Is multipath essential on Server 20.04?

Liam Proven
On Fri, 2 Oct 2020 at 20:50, R C <[hidden email]> wrote:

>
> multipath is needed for (optimizing) io when there are virtual storage
> devices that use multiple hardware/actual devices (hence multipath), for
> example when you are using lustre, scality, san, jbod, ZFS or 'any array'
>
> since you are using ZFS, in/on an array, it would be a good idea to use it.
>
> You don't NEED to use it, it only makes io more efficient and probably
> faster.
>
> Also, you can shut down some of the cores on your RPI cpu, you DON'T
> really need those either.

Thanks for the comments.

My impression was that it was for SAN connections -- things like
iSCSI, and maybe Ceph. I don't know much about Lustre/Glustre, and
I've never even heard of Scality before. JBOD? Just a Bunch of Disks?
Really? How so?

ZFS I'm using, yes, which is why I was concerned. If I end up dropping
ZFS then I'll probably replace it with a plain old MDRAID. Both are
multi-device filesystems, so I figured they'd need the kernel
devicemapper, but I don't see why ZFS on a single box that's not
connecting to anything else should need it. How come?

(Obligatory "please bottom-post on the list" comment, too...)


--
Liam Proven – Profile: https://about.me/liamproven
Email: [hidden email] – gMail/gTalk/gHangouts: [hidden email]
Twitter/Facebook/LinkedIn/Flickr: lproven – Skype: liamproven
UK: +44 7939-087884 – ČR (+ WhatsApp/Telegram/Signal): +420 702 829 053

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
R C
Reply | Threaded
Open this post in threaded view
|

Re: Is multipath essential on Server 20.04?

R C


On 10/2/20 1:58 PM, Liam Proven wrote:

> On Fri, 2 Oct 2020 at 20:50, R C <[hidden email]> wrote:
>> multipath is needed for (optimizing) io when there are virtual storage
>> devices that use multiple hardware/actual devices (hence multipath), for
>> example when you are using lustre, scality, san, jbod, ZFS or 'any array'
>>
>> since you are using ZFS, in/on an array, it would be a good idea to use it.
>>
>> You don't NEED to use it, it only makes io more efficient and probably
>> faster.
>>
>> Also, you can shut down some of the cores on your RPI cpu, you DON'T
>> really need those either.
> Thanks for the comments.
>
> My impression was that it was for SAN connections -- things like
> iSCSI, and maybe Ceph. I don't know much about Lustre/Glustre, and
> I've never even heard of Scality before. JBOD? Just a Bunch of Disks?
> Really? How so?
>
> ZFS I'm using, yes, which is why I was concerned. If I end up dropping
> ZFS then I'll probably replace it with a plain old MDRAID. Both are
> multi-device filesystems, so I figured they'd need the kernel
> devicemapper, but I don't see why ZFS on a single box that's not
> connecting to anything else should need it. How come?
>
> (Obligatory "please bottom-post on the list" comment, too...)
>
I wonder why  that "bottom posting" is such an issue,  there was a time
where no one really cared....    but anyway:


I used to develop large scale storage systems, like Lustre. Scality
builds large "object stores", basically an

object based storage system, I did some R&D work for that.


Basically when you have multiple devices show up (/dev/sd*) and they are
used in a virtual device, it would be benificial to use multipath,
especially when the number of physical devices go up.


I assume you built a raid pool with your ZFS setup, and use multiple,
actual, drives in your ZFS pool. When writing to the pool, a virtual
device, at some point a stripe needs to end up on a physical device
somewhere. (and vice versa, for reading from a pool,  a stripe needs to
come from a physical device somewhere)

That means you have multiple paths to a virtual device, at least one for
each actual drive in that pool. That's what you'd want multipath for,
especially under load when you are hammering that ZFS filesystem. It
starts to even get more important when you  put a storage system on top
of ZFS, for example put Lustre on top of ZFS pools, where each pool
consists of multiple actual drives/devices.

Yes JBODs are big boxes that contain nothing but a boatload of drives,
and they would all show up as devices/drives, in /dev. Typically  you'd
make a bunch of pools of drives, array, often done with ZFS and then use
these virtual/ drives, ZFS pools, for putting a storage system on top
of. (It is done both for compression and for redundancy. JBODs are used
for building filesystems many PBs in size.)

I don't think you need multipath simply because you have multiple
drives,  multipath is needed for optimizing io on a virtual device,
consisting of more than one actual physical drive/device, where you have
multiple actual paths to the physical drives. Especially in parallel io
it becomes a big deal.






--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Is multipath essential on Server 20.04?

Liam Proven
On Fri, 2 Oct 2020 at 22:52, R C <[hidden email]> wrote:

> I wonder why  that "bottom posting" is such an issue,  there was a time
> where no one really cared....    but anyway:

Well, no-one cared because it was the norm and everyone did it.

Then Microsoft came along with Outlook and broke email, and the world
has never recovered.

The big benefit is that with bottom-posting you can easily address
multiple points one-by-one, whereas with top-posting it's all at once.
Secondly, it's easier to follow a bottom-posted thread, because
English like most languages reads top-to-bottom.

> I used to develop large scale storage systems, like Lustre.

Okay. I just document them (he said, sitting here in a Ceph T-shirt.)

> Scality
> builds large "object stores", basically an
>
> object based storage system, I did some R&D work for that.

OK. I looked them up on Wikipedia. Probably because they're a
competitor to my $DAYJOB, it was new to me.

> Basically when you have multiple devices show up (/dev/sd*) and they are
> used in a virtual device, it would be benificial to use multipath,
> especially when the number of physical devices go up.

I was struggling to understand _why_.

> I assume you built a raid pool with your ZFS setup, and use multiple,
> actual, drives in your ZFS pool.

Yes. 5 x 2TB USB 3 drivers.

> When writing to the pool, a virtual
> device, at some point a stripe needs to end up on a physical device
> somewhere. (and vice versa, for reading from a pool,  a stripe needs to
> come from a physical device somewhere)

Sure. But that's what devicemapper does.

> That means you have multiple paths to a virtual device, at least one for
> each actual drive in that pool.

Um. Not if there is 1 server with _n_ drives on it, no. Not AFAICS. 2
servers with _n_ drives, possibly, depending on architecture.

>  That's what you'd want multipath for,
> especially under load when you are hammering that ZFS filesystem. It
> starts to even get more important when you  put a storage system on top
> of ZFS, for example put Lustre on top of ZFS pools, where each pool
> consists of multiple actual drives/devices.

I will take your word, but I am mainly sharing the single RAIDZ to an
iMac using Netatalk.

> Yes JBODs are big boxes that contain nothing but a boatload of drives,
> and they would all show up as devices/drives, in /dev.

Sure. I was running one on NT Server 4 as my home server in 1997 or
so: 7 full-height 5.25" 700MB SCSI-2 drives in a metre-tall 70kg cab.
In full flight, it sounded like a Harrier was hovering outside and I
couldn't take phone calls, but I had a quite fast 5GB RAID on my
dual-Pentium-100-server, backing up to an Exabyte drive using VHS-C
cassettes, and it cost next to nothing. Old surplus kit.

I'm not totally new to this stuff. :-)

> Typically  you'd
> make a bunch of pools of drives, array, often done with ZFS and then use
> these virtual/ drives, ZFS pools, for putting a storage system on top
> of.

OK, so this has changed since I last implemented it. I made an array,
formatted it and shared it. That was it: job done. I've done this on
NT 4 Server, Windows Server 2000/2003/2008 and various Linuxes. No
need for any layer above the RAID.

>  (It is done both for compression and for redundancy. JBODs are used
> for building filesystems many PBs in size.)

Hmmm. OK, not when I last did it.

> I don't think you need multipath simply because you have multiple
> drives,  multipath is needed for optimizing io on a virtual device,
> consisting of more than one actual physical drive/device, where you have
> multiple actual paths to the physical drives. Especially in parallel io
> it becomes a big deal.

This is the tricky bit to follow. I did some more Googling and found a
few articles full of bloviating -- such as this:
https://www.thegeekdiary.com/beginners-guide-to-device-mapper-dm-multipathing/

... and one that was comprehensible. This:
https://www.thegeekdiary.com/understanding-linux-multipath-using-dm-multipath/

IOW, this is valuable _if and only if_ there are multiple different
routes over separate comms channels between your server and your
storage.

But all my drives are local: USB 3 drives, on a single USB 3 hub, on a
single USB 3 port on a RasPi 4. AIUI the Pi 4 only has a single PCI-e
channel so all ports share a single 4Gb/s channel to the CPU.

So, IIUIC, there is no possibility of multipath here, and indeed, I
would guess that normally if the drives are all local to the server,
then there can't be, right?

Anyway, I removed it, the Pi's idle RAM consumption fell by about
150MB from nearly 400MB to about 250-something MB, and it all still
seems to work. :-)

It's a bit too soon to tell if performance is any better.

I can still see ssh sessions pausing -- freezing then unfreezing --
when I telnet into its wired GbE port, but it doesn't do this if I
connect to its WLAN port. That is not good. I am beginning to suspect
my network cable.  (See thread titled "Anyone running Server on a
Raspberry Pi 4?" on July 21.)

--
Liam Proven – Profile: https://about.me/liamproven
Email: [hidden email] – gMail/gTalk/gHangouts: [hidden email]
Twitter/Facebook/LinkedIn/Flickr: lproven – Skype: liamproven
UK: +44 7939-087884 – ČR (+ WhatsApp/Telegram/Signal): +420 702 829 053

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users