Symbols files for C++ libraries for Ubuntu main

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Symbols files for C++ libraries for Ubuntu main

Jeremy Bicha-2
Hi,

There has recently been a question [1] about whether symbols files
should be mandatory for C++ libraries in Ubuntu 'main'. I'm curious to
hear what other Ubuntu developers think about this topic.

Symbols files aren't mentioned in the official MIR criteria at all. [2]

I think there's general agreement that Ubuntu should use symbols files
for C libraries. And although it's not "required" in Debian as I
understand it, I think there is widespread support for it as "best
practice" there too.

C++ libraries can be much more difficult to maintain symbols files
for. I frequently see uploads for a new upstream release of a C++
library, quickly followed by another upload to fix the symbols files
so that the package builds on all supported architectures.

In my opinion, symbols files for a C++ library is a decision best made
by the package maintainer. [3] Therefore, if the Debian maintainer
chooses to not maintain symbols files, the Ubuntu team handling the
package can follow the Debian lead or optionally maintain symbols
files as an Ubuntu diff.

[1] https://launchpad.net/bugs/1770748
[2] https://wiki.ubuntu.com/UbuntuMainInclusionRequirements
[3] See Debian Policy ยง 8.6
https://www.debian.org/doc/debian-policy/#dependencies-between-the-library-and-other-packages

Thanks,
Jeremy Bicha

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Robie Basak-4
I understand the general advantage of having symbols files. But in the
specific case of C++ packages where upstream and Debian aren't caring
for the symbols files, the public symbols aren't particularly well
defined and we're maintaining a symbols file in Ubuntu as a delta, I
question whether it's actually useful for us to do so. In practice, if
all that happens is that when new symbols appear we throw them into the
symbols file without much thinking (because the symbols/ABI are
inpenetrable even after being decoded), then what are we gaining from
doing this?

I'm sure someone will say "well, think harder", but I think that
pragmatically we need to look at we're doing in practice.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Ian Bruntlett
Hi Robie et al,

I have quite a bit of C experience and some C++ experience (not on Linux, though) and I'm a volunteer tester. I haven't quite got the hang of github (it is in my pile of things to learn).

Having access to the source code for packages would be helpful because sometimes, when chasing down a bug, access to source code is desirable.

You never know, one day I might even be submitting patches for my favourite projects :)


BW,


Ian

--
-- ACCU - Professionalism in programming - http://www.accu.org
-- My writing - https://sites.google.com/site/ianbruntlett/


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Matthias Klose-6
In reply to this post by Jeremy Bicha-2
On 18.05.2018 01:49, Jeremy Bicha wrote:
> Hi,
>
> There has recently been a question [1] about whether symbols files
> should be mandatory for C++ libraries in Ubuntu 'main'. I'm curious to
> hear what other Ubuntu developers think about this topic.
>
> Symbols files aren't mentioned in the official MIR criteria at all. [2]

This is wrong. https://wiki.ubuntu.com/MIRTeam shows the list of things that the
MIR team usually checks and we usually extend it when we see new checks which
should be necessary.  It might be time to update the list of criteria using this
list.  I'm adding a work item for that for our next sprint.

> I think there's general agreement that Ubuntu should use symbols files
> for C libraries. And although it's not "required" in Debian as I
> understand it, I think there is widespread support for it as "best
> practice" there too.
>
> C++ libraries can be much more difficult to maintain symbols files
> for. I frequently see uploads for a new upstream release of a C++
> library, quickly followed by another upload to fix the symbols files
> so that the package builds on all supported architectures.

yes, and that is a very easy process using the pkg kde symbols helper.
See https://pkg-kde.alioth.debian.org/symbolfiles.html, but this link is now not
accessible anymore.

> In my opinion, symbols files for a C++ library is a decision best made
> by the package maintainer. [3] Therefore, if the Debian maintainer
> chooses to not maintain symbols files, the Ubuntu team handling the
> package can follow the Debian lead or optionally maintain symbols
> files as an Ubuntu diff.

I completely disagree.  Replacing a somehow suboptimal check with no check is
not an option.  The symbols files for C++ are not ideal, and having template
instantiated symbols there, which change with the level of optimization or the
compiler version is sometimes tedious.

What you refer to the decision of the Debian maintainer is the resolution of
Debian bug #815633, a request (including a patch) to add support for kfreebsd
symbols files, resulting in just dropping the symbols files.  Sorry, but that's
not something where we should follow Debian, at least not for packages in main.

I already mentioned the pkg kde symbols helper how to improve that situation,
but there are other alternatives:

 - the abi-compliance-checker, and available in debhelper via dh-acc.
   I'm not sure about how it is maintained these days

 - abigail-tools, a tool used by Fedora/RedHat, but it can be picky sometimes,
   and more strict than just symbols files.

I am more worried that Ubuntu tries to drop deltas more aggressively these days,
at least for packages in main.  The MIR process is a one-time process (including
the security reviews), and relies on the fact that having these changes in
follow-up uploads is respected for further uploads.  If that's not anymore the
case we need to review the MIR process and security reviews to include new
reviews when Ubuntu is dropping deltas from Debian (for packages in main).

Having no-delta between Debian and Ubuntu should be goal, but not a must-have.
I feel that this is seen differently these days, but it's a policy change which
is not communicated in any way.

Matthias

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Matthias Klose-6
In reply to this post by Robie Basak-4
On 18.05.2018 14:13, Robie Basak wrote:
> I understand the general advantage of having symbols files. But in the
> specific case of C++ packages where upstream and Debian aren't caring
> for the symbols files, the public symbols aren't particularly well
> defined and we're maintaining a symbols file in Ubuntu as a delta, I
> question whether it's actually useful for us to do so. In practice, if
> all that happens is that when new symbols appear we throw them into the
> symbols file without much thinking (because the symbols/ABI are
> inpenetrable even after being decoded), then what are we gaining from
> doing this?

please see my reply to the top level posting.

> I'm sure someone will say "well, think harder", but I think that
> pragmatically we need to look at we're doing in practice.

this is a dangerous way to go.  If everybody gets sloppy and doesn't respect
anything, where do we go?  I'm probably known myself for my "pragmatism", but I
think there is some point where practice and policies diverge too much.  If we
think that policies need adjustments, we should document these, but silently
changing policies will cost us credibility.

Matthias

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Dimitri John Ledkov
In reply to this post by Jeremy Bicha-2
Hi,

On 18 May 2018 at 00:49, Jeremy Bicha <[hidden email]> wrote:
> Hi,
>
> There has recently been a question [1] about whether symbols files
> should be mandatory for C++ libraries in Ubuntu 'main'. I'm curious to
> hear what other Ubuntu developers think about this topic.
>

IMHO symbols files should be mandatory for any new libraries
introduced in the archive.

And we should assert symbols files for everything in main, and fix all
the things.

It's 2018, and we really ought to have sensible and strict symbols
files. And over-exposed things... well we should seal the libraries to
not expose symbols that are should not have been used, and in fact,
have not been used.

--
Regards,

Dimitri.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Seth Arnold
In reply to this post by Ian Bruntlett
On Fri, May 18, 2018 at 04:32:29PM +0100, Ian Bruntlett wrote:
> Having access to the source code for packages would be helpful because
> sometimes, when chasing down a bug, access to source code is desirable.

If you add "deb-src" lines to your apt sources, you can download the
source for a package to your current working directory with:

apt-get source source-package-name

for example..

sarnold@hunt:/tmp$ grep 'deb-src.*bionic' /etc/apt/sources.list
deb-src http://wopr/ubuntu/ bionic main restricted universe multiverse
deb-src http://wopr/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://wopr/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu bionic-security main restricted
deb-src http://security.ubuntu.com/ubuntu bionic-security universe
deb-src http://security.ubuntu.com/ubuntu bionic-security multiverse
sarnold@hunt:/tmp$ apt-get source bash
Reading package lists... Done
NOTICE: 'bash' packaging is maintained in the 'Bzr' version control system at:
http://bazaar.launchpad.net/~doko/+junk/pkg-bash-debian
Please use:
bzr branch http://bazaar.launchpad.net/~doko/+junk/pkg-bash-debian
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 5,103 kB of source archives.
Get:1 http://wopr/ubuntu cosmic/main bash 4.4.18-2ubuntu2 (dsc) [2,428 B]
Get:2 http://wopr/ubuntu cosmic/main bash 4.4.18-2ubuntu2 (tar) [5,036 kB]
Get:3 http://wopr/ubuntu cosmic/main bash 4.4.18-2ubuntu2 (diff) [64.0 kB]
Fetched 5,103 kB in 0s (30.0 MB/s)
dpkg-source: info: extracting bash in bash-4.4.18
dpkg-source: info: unpacking bash_4.4.18.orig.tar.xz
dpkg-source: info: unpacking bash_4.4.18-2ubuntu2.debian.tar.xz
dpkg-source: info: applying bash44-019.diff
dpkg-source: info: applying bashbug-editor.diff
dpkg-source: info: applying deb-bash-config.diff
dpkg-source: info: applying deb-examples.diff
dpkg-source: info: applying man-arithmetic.diff
dpkg-source: info: applying man-fignore.diff
dpkg-source: info: applying man-bashrc.diff
dpkg-source: info: applying man-bashlogout.diff
dpkg-source: info: applying man-nocaseglob.diff
dpkg-source: info: applying man-test.diff
dpkg-source: info: applying man-test2.diff
dpkg-source: info: applying rbash-manpage.diff
dpkg-source: info: applying bash-default-editor.diff
dpkg-source: info: applying pgrp-pipe.diff
dpkg-source: info: applying input-err.diff
dpkg-source: info: applying exec-redirections-doc.diff
dpkg-source: info: applying bash-aliases-repeat.diff
dpkg-source: info: applying use-system-texi2html.diff
dpkg-source: info: applying bzero.diff
dpkg-source: info: applying man-macro-warnings.diff
dpkg-source: info: applying po-de-fix.diff
dpkg-source: info: applying man-vx-opts.diff
sarnold@hunt:/tmp$ cd bash-4.4.18/
sarnold@hunt:/tmp/bash-4.4.18$ ls
ABOUT-NLS    bashline.c    conftypes.h    externs.h  jobs.h        parser.h        redir.h      trap.h
aclocal.m4   bashline.h    copy_cmd.c     findcmd.c  lib           parse.y         shell.c      unwind_prot.c
alias.c      bashtypes.h   COPYING        findcmd.h  list.c        patchlevel.h    shell.h      unwind_prot.h
alias.h      bracecomp.c   cross-build    flags.c    locale.c      pathexp.c       sig.c        variables.c
array.c      braces.c      CWRU           flags.h    m4            pathexp.h       sig.h        variables.h
arrayfunc.c  builtins      debian         general.c  mailcheck.c   pathnames.h.in  siglist.c    version.c
arrayfunc.h  builtins.h    dispose_cmd.c  general.h  mailcheck.h   pcomplete.c     siglist.h    xmalloc.c
array.h      ChangeLog     dispose_cmd.h  hashcmd.c  make_cmd.c    pcomplete.h     stringlib.c  xmalloc.h
assoc.c      CHANGES       doc            hashcmd.h  make_cmd.h    pcomplib.c      subst.c      Y2K
assoc.h      command.h     error.c        hashlib.c  Makefile.in   po              subst.h      y.tab.c
AUTHORS      COMPAT        error.h        hashlib.h  MANIFEST      POSIX           support      y.tab.h
bashansi.h   config-bot.h  eval.c         include    mksyntax.c    print_cmd.c     syntax.h
bashhist.c   config.h.in   examples       input.c    NEWS          quit.h          test.c
bashhist.h   config-top.h  execute_cmd.c  input.h    nojobs.c      RBASH           test.h
bashintl.h   configure     execute_cmd.h  INSTALL    NOTES         README          tests
bashjmp.h    configure.ac  expr.c         jobs.c     parser-built  redir.c         trap.c

(Observant readers may have noticed the download came from cosmic rather
than bionic. Security team members have fairly unique apt sources to
support our tooling. Just ignore that detail. :)

Thanks

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Jeremy Bicha-2
In reply to this post by Matthias Klose-6
Thank you for your reply.

On Fri, May 18, 2018 at 2:29 PM, Matthias Klose <[hidden email]> wrote:
> On 18.05.2018 01:49, Jeremy Bicha wrote:
>> Symbols files aren't mentioned in the official MIR criteria at all. [2]
>
> This is wrong. https://wiki.ubuntu.com/MIRTeam shows the list of things that the
> MIR team usually checks and we usually extend it when we see new checks which
> should be necessary.  It might be time to update the list of criteria using this
> list.  I'm adding a work item for that for our next sprint.

It's a problem that there is a separate list of MIR criteria that is
different than the older official list. Your list is not even linked
to from the official MIR requirements and process pages.

Also, even your page says that C++ libraries can simply use
dh_makeshlibs -V   which is more or less what Debian Policy 8.6 says.

Jeremy Bicha

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Simon Quigley-2
In reply to this post by Matthias Klose-6
Hello,

Putting on my Debian/Qt KDE Team hat.

On 05/18/2018 01:29 PM, Matthias Klose wrote:
> yes, and that is a very easy process using the pkg kde symbols helper.
> See https://pkg-kde.alioth.debian.org/symbolfiles.html, but this link is now not
> accessible anymore.

We just migrated the site from Alioth to Salsa today. Here's the new URL:

https://qt-kde-team.pages.debian.net/www/symbolfiles.html

Thanks!

--
Simon Quigley
[hidden email]
tsimonq2 on freenode and OFTC
5C7A BEA2 0F86 3045 9CC8
C8B5 E27F 2CF8 458C 2FA4


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Simon Quigley-2
> https://qt-kde-team.pages.debian.net/www/symbolfiles.html

Moved again, hopefully for the last time:
https://qt-kde-team.pages.debian.net/symbolfiles.html

--
Simon Quigley
[hidden email]
tsimonq2 on freenode and OFTC
5C7A BEA2 0F86 3045 9CC8
C8B5 E27F 2CF8 458C 2FA4


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Symbols files for C++ libraries for Ubuntu main

Robie Basak-4
In reply to this post by Dimitri John Ledkov
On Fri, May 18, 2018 at 08:29:13PM +0200, Matthias Klose wrote:
> I completely disagree.  Replacing a somehow suboptimal check with no
> check is not an option.

On Fri, May 18, 2018 at 08:22:55PM +0100, Dimitri John Ledkov wrote:
> IMHO symbols files should be mandatory for any new libraries
> introduced in the archive.
>
> And we should assert symbols files for everything in main, and fix all
> the things.
>
> It's 2018, and we really ought to have sensible and strict symbols
> files.

Both of these statements on their own state that we must do this work
but don't explain why this is of benefit to Ubuntu. I feel that you need
to justify your position rather than just stating it.

Can you provide examples of where maintaining this delta has actually
helped make Ubuntu better, in the specific case that C++ symbols are
being maintained by Ubuntu in a delta that Debian and upstream have
declined to adopt or postponed adopting? Without examples, we're not
really in a position to assess the trade-off of extra work vs. benefit
to Ubuntu. I don't think we should be maintaining delta unless the
benefit can be articulated and justified.

Separately, I question whether it's in the interest of our project to
spend time on maintaining a quality improvement indefinitely if Debian
and/or upstream decline to take it, and if that particular improvement
is not a high level goal of our project.

Thanks,

Robie

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (836 bytes) Download Attachment