Why do we strip server binaries?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Why do we strip server binaries?

Clint Byrum-4
I had an interesting conversation with Baron Schwartz from Percona (author
of the 2nd edition of High Performance MySQL and Maatkit) at the MySQL
Users conference last month regarding what he feels distributions all
get wrong. One particular pet peeve of his is that we strip our binaries.

As Baron sees it, this is a *tiny* gain (smaller binaries for the CD)
for a giant loss, which is the loss of ability to profile and introspect
a critical piece of software while it is running and, perhaps more
importantly, while it is failing.

Its not enough to make it possible to correct the issue. As we all know,
you have to make the system friendly by default.

The -dbg packages help, but a) aren't mandatory, and b) don't always
help with things like oprofile which doesn't know about them.

I've personally run into this a few times in production usage where a
problem is particularly hard to wrap one's head around, and I always
ended up replacing my distro packages with something that was unstripped.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why do we strip server binaries?

Chow Loong Jin
On 03/05/2011 13:41, Clint Byrum wrote:

> I had an interesting conversation with Baron Schwartz from Percona (author
> of the 2nd edition of High Performance MySQL and Maatkit) at the MySQL
> Users conference last month regarding what he feels distributions all
> get wrong. One particular pet peeve of his is that we strip our binaries.
>
> As Baron sees it, this is a *tiny* gain (smaller binaries for the CD)
> for a giant loss, which is the loss of ability to profile and introspect
> a critical piece of software while it is running and, perhaps more
> importantly, while it is failing.
>
> Its not enough to make it possible to correct the issue. As we all know,
> you have to make the system friendly by default.
>
> The -dbg packages help, but a) aren't mandatory, and b) don't always
> help with things like oprofile which doesn't know about them.
>
> I've personally run into this a few times in production usage where a
> problem is particularly hard to wrap one's head around, and I always
> ended up replacing my distro packages with something that was unstripped.
I haven't had an issue that hasn't been resolved with -dbgsym packages so far,
but then again I don't use oprofile.

On the other hand, however, surely it's not a bug in the way we strip packages,
but a bug in oprofile and other $random_tools for not supporting the external
debug symbols?

--
Kind regards,
Loong Jin


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (918 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Why do we strip server binaries?

Stefan Bader-2
In reply to this post by Clint Byrum-4
On 05/03/2011 07:41 AM, Clint Byrum wrote:

> I had an interesting conversation with Baron Schwartz from Percona (author
> of the 2nd edition of High Performance MySQL and Maatkit) at the MySQL
> Users conference last month regarding what he feels distributions all
> get wrong. One particular pet peeve of his is that we strip our binaries.
>
> As Baron sees it, this is a *tiny* gain (smaller binaries for the CD)
> for a giant loss, which is the loss of ability to profile and introspect
> a critical piece of software while it is running and, perhaps more
> importantly, while it is failing.
>
Speaking for the kernel only, that tiny gain is about 500MB compared to around
30MB. There simply would not be a server CD if we did not strip...

-Stefan

> Its not enough to make it possible to correct the issue. As we all know,
> you have to make the system friendly by default.
>
> The -dbg packages help, but a) aren't mandatory, and b) don't always
> help with things like oprofile which doesn't know about them.
>
> I've personally run into this a few times in production usage where a
> problem is particularly hard to wrap one's head around, and I always
> ended up replacing my distro packages with something that was unstripped.
>


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why do we strip server binaries?

Jani Monoses-2
In reply to this post by Chow Loong Jin
>
> I haven't had an issue that hasn't been resolved with -dbgsym packages so far,
> but then again I don't use oprofile.

FWIW perf recognizes those symbols and seems nicer than oprofile in many
other respects too.

Jani


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why do we strip server binaries?

Scott Kitterman-3
In reply to this post by Clint Byrum-4
On Tuesday, May 03, 2011 01:41:10 AM Clint Byrum wrote:

> I had an interesting conversation with Baron Schwartz from Percona (author
> of the 2nd edition of High Performance MySQL and Maatkit) at the MySQL
> Users conference last month regarding what he feels distributions all
> get wrong. One particular pet peeve of his is that we strip our binaries.
>
> As Baron sees it, this is a *tiny* gain (smaller binaries for the CD)
> for a giant loss, which is the loss of ability to profile and introspect
> a critical piece of software while it is running and, perhaps more
> importantly, while it is failing.
>
> Its not enough to make it possible to correct the issue. As we all know,
> you have to make the system friendly by default.
>
> The -dbg packages help, but a) aren't mandatory, and b) don't always
> help with things like oprofile which doesn't know about them.
>
> I've personally run into this a few times in production usage where a
> problem is particularly hard to wrap one's head around, and I always
> ended up replacing my distro packages with something that was unstripped.

The dbgsym repository is not well enough publicized IMO.

https://lists.ubuntu.com/archives/ubuntu-devel-announce/2006-
September/000195.html

I've always found they work just fine and are available for all packages.  I
agree with the other comment that it's better to fix tools that have problems
with the way we provide symbols than to give up on stripping.  The "tiny" gain
we have is that we can ship the various Ubuntu flavors on reasonable sized
media.

It might make sense to add the dbgsym repository to the standard sources.list
we ship and also include it's key so that it's easier for peope to enable it
and get the symbols they need.

Scott K

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why do we strip server binaries?

Mario Limonciello-2


On May 3, 2011 5:54 AM, "Scott Kitterman" <[hidden email]> wrote:
>
> On Tuesday, May 03, 2011 01:41:10 AM Clint Byrum wrote:
> > I had an interesting conversation with Baron Schwartz from Percona (author
> > of the 2nd edition of High Performance MySQL and Maatkit) at the MySQL
> > Users conference last month regarding what he feels distributions all
> > get wrong. One particular pet peeve of his is that we strip our binaries.
> >
> > As Baron sees it, this is a *tiny* gain (smaller binaries for the CD)
> > for a giant loss, which is the loss of ability to profile and introspect
> > a critical piece of software while it is running and, perhaps more
> > importantly, while it is failing.
> >
> > Its not enough to make it possible to correct the issue. As we all know,
> > you have to make the system friendly by default.
> >
> > The -dbg packages help, but a) aren't mandatory, and b) don't always
> > help with things like oprofile which doesn't know about them.
> >
> > I've personally run into this a few times in production usage where a
> > problem is particularly hard to wrap one's head around, and I always
> > ended up replacing my distro packages with something that was unstripped.
>
> The dbgsym repository is not well enough publicized IMO.
>
> https://lists.ubuntu.com/archives/ubuntu-devel-announce/2006-
> September/000195.html
>
> I've always found they work just fine and are available for all packages.  I
> agree with the other comment that it's better to fix tools that have problems
> with the way we provide symbols than to give up on stripping.  The "tiny" gain
> we have is that we can ship the various Ubuntu flavors on reasonable sized
> media.
>
> It might make sense to add the dbgsym repository to the standard sources.list
> we ship and also include it's key so that it's easier for peope to enable it
> and get the symbols they need.
>
> Scott K
>
Weren't the ddebs supposed to eventually end up on the Ubuntu archive once Launchpad supported them?

If so maybe it would make more sense for flavors that want them preinstalled to make room on their disks and just seed the applicable ones.


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why do we strip server binaries?

Steve Langasek-6
In reply to this post by Clint Byrum-4
On Mon, May 02, 2011 at 10:41:10PM -0700, Clint Byrum wrote:
> I had an interesting conversation with Baron Schwartz from Percona (author
> of the 2nd edition of High Performance MySQL and Maatkit) at the MySQL
> Users conference last month regarding what he feels distributions all
> get wrong. One particular pet peeve of his is that we strip our binaries.

> As Baron sees it, this is a *tiny* gain (smaller binaries for the CD)
> for a giant loss, which is the loss of ability to profile and introspect
> a critical piece of software while it is running and, perhaps more
> importantly, while it is failing.

> Its not enough to make it possible to correct the issue. As we all know,
> you have to make the system friendly by default.

The current generation of detached debugging symbols appear to be 1.5x the
size of the stripped binaries themselves, on average.  This is actually
quite a bit better than it used to be when dwarf2 first appeared, but it's
still enough that I wouldn't call this a "tiny" gain.

Which is the more common case: installing Ubuntu from a CD, or profiling /
debugging a process after installation?

If you could only fit half the software on an install CD that we currently
do, which half would you cut?

> The -dbg packages help, but a) aren't mandatory,

No, but what are mandatory are -dbgsym packages which are autogenerated on
the Ubuntu autobuilders.  For some reason, these are missing for mysql-5.1
in natty.  Perhaps fixing this, along with improving our process for getting
dbgsyms packages into the hands of users
(https://blueprints.launchpad.net/ubuntu/+spec/foundations-o-dbgsym-integration/),
would address this issue?

> and b) don't always help with things like oprofile which doesn't know
> about them.

So why doesn't oprofile know about them?  Detached symbols files are pretty
much The Way It's Done now; shouldn't oprofile be fixed to understand them?

--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
[hidden email]                                     [hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why do we strip server binaries?

Barry Warsaw-2
On May 03, 2011, at 05:45 PM, Steve Langasek wrote:

>Which is the more common case: installing Ubuntu from a CD, or profiling /
>debugging a process after installation?

As always, "it depends" :)

For me, the latter is much more common.  I very rarely actually burn a CD,
even though I do fairly often download both the CD and DVD ISOs.  I almost
always mount them rather than burning a physical disk.

For me, unstripped binaries would be much more useful.  Of course, I also
recognize that I'm not a typical (or maybe, "the target") Ubuntu user.

This kind of touches on another use case that is kind of inconvenient for
developers like me.  I usually want lots of -dev packages but tracking those
down are kind of a pain.  Yes, `apt-get build-dep` helps, but not completely.

Maybe something that would be helpful would be a ubuntu-developer meta-package
that depended on the dbgsym and dev packages, so I could just get them all in
one fell swoop.  I generally don't care about network bandwidth or disk space,
so that might not be very typical either.

Cheers,
-Barry

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (853 bytes) Download Attachment