i386 in focal: an update

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

i386 in focal: an update

Steve Langasek-6
Thanks to thorough feedback from our community, we now have a reasonably
comprehensive answer to the question of what 32-bit compatibility library
packages are needed on x86 for Ubuntu 20.04.

https://discourse.ubuntu.com/t/community-process-for-32-bit-compatibility/12598/46

Some developers will have noticed changes this week to the behavior of focal
builds in Launchpad.  Out of 30,000 source packages in focal, there is now a
whitelist of about 1,700 source packages which will trigger builds on i386
in Launchpad.  This means that other packages which previously built on i386
will need to have the binaries from the old version of the package removed
before they will be migratable from focal-proposed to focal.


As a side note, the implementation of this also affects PPA builds, because
the whitelist applies to the focal series as a whole.  In general, you
should not expect to need i386 builds of third-party packages in PPAs for
focal either, given that i386 in focal exists solely for compatibility with
legacy binary software.  However, if you have a third-party package that you
believe it's important to continue producing i386 binary builds of in
Launchpad for Ubuntu 20.04, please contact the Ubuntu archive admins
([hidden email], or #ubuntu-devel on freenode.net for best
results), and we can evaluate including your PPA package in the whitelist.


At the moment, I am doing some manual removals of the i386 binaries as I see
them show up as blockers on
https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html
and as I'm able to determine that the removals aren't going to cause
near-term knock-on problems.  But if some i386 binaries aren't being removed
fast enough and this is blocking your work, feel free to reach out to an
archive admin to ask for their removal.

In the slightly less near term, the plan is to do a mass binary removal of
all of the i386 binary packages in focal built from sources other than those
in the whitelist.  However, before pulling the trigger on this mass removal,
there are some changes that should be landed to our autopkgtest
infrastructure, so that we can continue to run autopkgtests for those
remaining 1700 packages.  In summary: the plan is not to retain the test
dependencies of those 1700 packages on i386, but instead to cross-test the
i386 libraries on an amd64 host, which ultimately means testing them in an
environment that better models the expected real-world usage.  The work is
in progress for this change and I'm currently anticipating landing it next
week.

In the meantime, if you need any help getting packages migrating to the
focal release from -proposed, please reach out on #ubuntu-release on IRC.

Thanks,
--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                   https://www.debian.org/
[hidden email]                                     [hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: i386 in focal: an update

Steve Langasek-6
Hi folks,

The changes to i386 have proceeded as described in my earlier email.  The
autopkgtest infrastructure has been converted to run tests of i386 binaries
on an amd64 VM, and i386 binaries not in the whitelist are now in the
process of being removed.  As of this moment, there are 1,679 source
packages in the whitelist for i386, and 8,921 source packages with published
i386 binary packages in focal, vs. 14,344 source packages with published
amd64 binaries.  At the current rate, I expect the binary removals to
complete some time tomorrow.

On the autopkgtest side, there are now a total of 311 source packages
producing i386 binaries which also have autopkgtests[1].  Any packages
reporting autopkgtest regressions on i386 which are not on this list should
have their test failures ignored; if you see i386 autopkgtest failures for
other packages blocking migrations out of focal-proposed, please ask a
member of the release team to override.

It's good that the number of remaining autopkgtests for i386 is as small as
it is, because while the /infrastructure/ is now capable of cross-testing
i386 packages on amd64, the vast majority of these tests don't run correctly
in such an environment without modification.  I have triggered runs of all
of these tests on the amd64 testbed, so current status of the tests is
visible on autopkgtest.ubuntu.com for each package.

Since these tests have also de facto regressed in the release pocket due to
changes to the infrastructure, it is appropriate for those test failures to
be ignored for proposed-migration purposes as well.  I intend to go through
and systematically mark these as bad tests, but in the meantime if you have
package migrations that are being blocked by one of these, please reach out.


The other thing that you can do is help fix the failing autopkgtests, so
that we have better test coverage on i386.  I've identified a few common
patterns so far in the test failures:

 - Tests of Essential or Priority: required/import packages that require removing
   the amd64 package in favor of the i386 package (e.g.: bash, apt,
   systemd).  These tests will never work, and - outside of launchpad
   builds, where we are still building i386 natively instead of on an amd64
   host - these packages aren't relevant to the i386 experience on focal.
   These test failures should be permanently ignored.

 - Tests with test dependencies on python on perl modules.  It's a
   longstanding known issue with multiarch that python and perl modules pose
   problems for cross-installation, because they depend on the interpreter
   of the matching architecture.  Neither perl nor python can be
   cross-installed on the testbed (the amd64 versions are both part of the
   test infrastructure), so these tests will always fail with 'badpkg'.
   While some of these tests may be more relevant to the behavior of our
   i386 runtime libraries, fixing them is largely intractable so these are a
   low priority.

 - Tests with other uninstallable test dependencies ("badpkg").  Many of
   these are going to be fixable, but require either annotations of
   additional packages in the archive as Multi-Arch: foreign (if
   appropriate), or annotations of the test dependencies in
   debian/tests/control with architecture information.  The autopkgtest
   dependency resolver for cross-testing uses apt-get build-dep, so that the
   full range of multiarch build-dependency specifiers are available as
   described at <https://wiki.ubuntu.com/MultiarchCross>.

 - Build tests.  These are common tests for libraries, and some of the
   easiest to fix, but most don't work out of the box because they don't
   invoke the cross-toolchain.  Here are some example patches for various
   types of build systems used in build tests:
   - pkg-config: https://bugs.debian.org/946303
   - meson: https://bugs.debian.org/946292

Happy fixing,
--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                   https://www.debian.org/
[hidden email]                                     [hidden email]

[1] See attachment.  List calculated by:
$ join -j1 <(wget -O - -q -o /dev/null \
               https://people.canonical.com/~ubuntu-archive/germinate-output/i386.focal/i386+build-depends.sources \
             | awk '{ print $1 }' | sort) \
           <(cat /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_focal_*Sources \
             | grep-dctrl -r -FTestsuite . -a '!' -FArchitecture -X all \
                          -sPackage -n | sort)

On Thu, Nov 28, 2019 at 12:46:34PM -0800, Steve Langasek wrote:

> Thanks to thorough feedback from our community, we now have a reasonably
> comprehensive answer to the question of what 32-bit compatibility library
> packages are needed on x86 for Ubuntu 20.04.
>
> https://discourse.ubuntu.com/t/community-process-for-32-bit-compatibility/12598/46
>
> Some developers will have noticed changes this week to the behavior of focal
> builds in Launchpad.  Out of 30,000 source packages in focal, there is now a
> whitelist of about 1,700 source packages which will trigger builds on i386
> in Launchpad.  This means that other packages which previously built on i386
> will need to have the binaries from the old version of the package removed
> before they will be migratable from focal-proposed to focal.
>
>
> As a side note, the implementation of this also affects PPA builds, because
> the whitelist applies to the focal series as a whole.  In general, you
> should not expect to need i386 builds of third-party packages in PPAs for
> focal either, given that i386 in focal exists solely for compatibility with
> legacy binary software.  However, if you have a third-party package that you
> believe it's important to continue producing i386 binary builds of in
> Launchpad for Ubuntu 20.04, please contact the Ubuntu archive admins
> ([hidden email], or #ubuntu-devel on freenode.net for best
> results), and we can evaluate including your PPA package in the whitelist.
>
>
> At the moment, I am doing some manual removals of the i386 binaries as I see
> them show up as blockers on
> https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html
> and as I'm able to determine that the removals aren't going to cause
> near-term knock-on problems.  But if some i386 binaries aren't being removed
> fast enough and this is blocking your work, feel free to reach out to an
> archive admin to ask for their removal.
>
> In the slightly less near term, the plan is to do a mass binary removal of
> all of the i386 binary packages in focal built from sources other than those
> in the whitelist.  However, before pulling the trigger on this mass removal,
> there are some changes that should be landed to our autopkgtest
> infrastructure, so that we can continue to run autopkgtests for those
> remaining 1700 packages.  In summary: the plan is not to retain the test
> dependencies of those 1700 packages on i386, but instead to cross-test the
> i386 libraries on an amd64 host, which ultimately means testing them in an
> environment that better models the expected real-world usage.  The work is
> in progress for this change and I'm currently anticipating landing it next
> week.
>
> In the meantime, if you need any help getting packages migrating to the
> focal release from -proposed, please reach out on #ubuntu-release on IRC.
>
> Thanks,
> --
> Steve Langasek                   Give me a lever long enough and a Free OS
> Debian Developer                   to set it on, and I can move the world.
> Ubuntu Developer                                   https://www.debian.org/
> [hidden email]                                     [hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

i386-whitelisted-autopkgtests.txt (3K) Download Attachment
signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

i386 autopkgtest [Re: i386 in focal: an update]

Steve Langasek-6
Hi all,

After a few days of bugfixing tests to be compatible with the new
autopkgtest cross-arch infrastructure, I thought it would be useful to give
folks some stats, and point you to a list of packages requiring further
investigation that you can help drive down.

Of 311 autopkgtests that we care about on i386, 24% are currently passing,
8% have been confirmed as not-cross-testable and blacklisted by the release
team, and 68% still need investigation.  Details, including a list of
packages to investigate, produced by the attached script:

success: 74
blacklisted: 26
failed, needs investigation: 211
acl
alglib
alsa-lib
ann
bind9
binutils
boost1.67
brotli
bubblewrap
bzip2
capnproto
cfitsio
cluster-glue
cmake
colord
confuse
coreutils
cunit
cups
cython
datefudge
dbus
devscripts
dietlibc
distro-info
dlm
docbook-to-man
double-conversion
doxygen
faketime
fenix
fenix-plugins
fftw3
fpc
freeglut
fribidi
fyba
game-data-packager
gcc-8
gcc-9
gcc-snapshot
gdbm
gem2deb
gl2ps
glibc
gnupg2
gpgme1.0
graphite2
grep
gtk+2.0
gtk+3.0
gvfs
gzip
harfbuzz
ibus
icu
inetutils
init-system-helpers
iproute2
jemalloc
json-glib
krb5
kronosnet
lapack
lbfgsb
libanyevent-perl
libarchive
libassuan
libasync-interrupt-perl
libb-hooks-op-check-perl
libbsd-resource-perl
libcaca
libcap2
libclone-perl
libcommon-sense-perl
libdaemon
libdatrie
libdevel-callchecker-perl
libevent-perl
libev-perl
libfile-libmagic-perl
libfilter-perl
libgd2
libgd-perl
libgpg-error
libgphoto2
libhtml-parser-perl
libinput
libio-pty-perl
libiptcdata
libjsoncpp
libkate
liblist-moreutils-perl
liblocale-gettext-perl
libmicrohttpd
libnetaddr-ip-perl
libnet-ssleay-perl
libnfs
libofa
libomxil-bellagio
libopenmpt
libparams-classify-perl
libparams-util-perl
libparams-validate-perl
libperlio-gzip-perl
libpipeline
libproxy
libpsl
libqb
libraw
libsass
libsdl2
libsdl-sge
libseccomp
libsecret
libsocket6-perl
libsort-key-perl
libsoup2.4
libsoxr
libssh
libsub-identify-perl
libsub-name-perl
libtaint-runtime-perl
libtest-leaktrace-perl
libtest-taint-perl
libtext-charwidth-perl
libtext-iconv-perl
libthai
libtommath
libunicode-utf8-perl
libusb-1.0
libutempter
libuuid-perl
libuv1
libvariable-magic-perl
libvisual
libvncserver
libvorbis
libwacom
libwant-perl
libxcb
libxml2
libxml-libxml-perl
libxml-parser-perl
libyaml-libyaml-perl
linux
livecd-rootfs
llvm-toolchain-9
matplotlib
mir
mpi4py
multipath-tools
ncurses
network-manager
nghttp2
nss-wrapper
numpy
openjdk-lts
openssh
openssl
pacemaker
perl
pillow
pinentry
poppler
postgresql-12
protobuf
pulseaudio
pupnp-1.8
python2.7
python3.7
python3.8
python-apt
python-bcrypt
python-cffi
python-coverage
python-cryptography
python-evdev
python-lz4
python-nacl
python-numpy
python-scipy
py-ubjson
pyyaml
raqm
re2
ruby2.5
ruby-json
ruby-nokogiri
samba
sane-backends
sdlgfx
sed
stunnel4
superlu
systemd
systemtap
talloc
tdb
twisted
ubuntu-drivers-common
uchardet
udisks2
upower
util-linux
valgrind
vtk6
vtk7
xterm
zope.interface
zsh

Cheers,
--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                   https://www.debian.org/
[hidden email]                                     [hidden email]

On Fri, Dec 06, 2019 at 03:57:21PM -0800, Steve Langasek wrote:

> Hi folks,
>
> The changes to i386 have proceeded as described in my earlier email.  The
> autopkgtest infrastructure has been converted to run tests of i386 binaries
> on an amd64 VM, and i386 binaries not in the whitelist are now in the
> process of being removed.  As of this moment, there are 1,679 source
> packages in the whitelist for i386, and 8,921 source packages with published
> i386 binary packages in focal, vs. 14,344 source packages with published
> amd64 binaries.  At the current rate, I expect the binary removals to
> complete some time tomorrow.
>
> On the autopkgtest side, there are now a total of 311 source packages
> producing i386 binaries which also have autopkgtests[1].  Any packages
> reporting autopkgtest regressions on i386 which are not on this list should
> have their test failures ignored; if you see i386 autopkgtest failures for
> other packages blocking migrations out of focal-proposed, please ask a
> member of the release team to override.
>
> It's good that the number of remaining autopkgtests for i386 is as small as
> it is, because while the /infrastructure/ is now capable of cross-testing
> i386 packages on amd64, the vast majority of these tests don't run correctly
> in such an environment without modification.  I have triggered runs of all
> of these tests on the amd64 testbed, so current status of the tests is
> visible on autopkgtest.ubuntu.com for each package.
>
> Since these tests have also de facto regressed in the release pocket due to
> changes to the infrastructure, it is appropriate for those test failures to
> be ignored for proposed-migration purposes as well.  I intend to go through
> and systematically mark these as bad tests, but in the meantime if you have
> package migrations that are being blocked by one of these, please reach out.
>
>
> The other thing that you can do is help fix the failing autopkgtests, so
> that we have better test coverage on i386.  I've identified a few common
> patterns so far in the test failures:
>
>  - Tests of Essential or Priority: required/import packages that require removing
>    the amd64 package in favor of the i386 package (e.g.: bash, apt,
>    systemd).  These tests will never work, and - outside of launchpad
>    builds, where we are still building i386 natively instead of on an amd64
>    host - these packages aren't relevant to the i386 experience on focal.
>    These test failures should be permanently ignored.
>
>  - Tests with test dependencies on python on perl modules.  It's a
>    longstanding known issue with multiarch that python and perl modules pose
>    problems for cross-installation, because they depend on the interpreter
>    of the matching architecture.  Neither perl nor python can be
>    cross-installed on the testbed (the amd64 versions are both part of the
>    test infrastructure), so these tests will always fail with 'badpkg'.
>    While some of these tests may be more relevant to the behavior of our
>    i386 runtime libraries, fixing them is largely intractable so these are a
>    low priority.
>
>  - Tests with other uninstallable test dependencies ("badpkg").  Many of
>    these are going to be fixable, but require either annotations of
>    additional packages in the archive as Multi-Arch: foreign (if
>    appropriate), or annotations of the test dependencies in
>    debian/tests/control with architecture information.  The autopkgtest
>    dependency resolver for cross-testing uses apt-get build-dep, so that the
>    full range of multiarch build-dependency specifiers are available as
>    described at <https://wiki.ubuntu.com/MultiarchCross>.
>
>  - Build tests.  These are common tests for libraries, and some of the
>    easiest to fix, but most don't work out of the box because they don't
>    invoke the cross-toolchain.  Here are some example patches for various
>    types of build systems used in build tests:
>    - pkg-config: https://bugs.debian.org/946303
>    - meson: https://bugs.debian.org/946292
>
> Happy fixing,
> --
> Steve Langasek                   Give me a lever long enough and a Free OS
> Debian Developer                   to set it on, and I can move the world.
> Ubuntu Developer                                   https://www.debian.org/
> [hidden email]                                     [hidden email]
>
> [1] See attachment.  List calculated by:
> $ join -j1 <(wget -O - -q -o /dev/null \
>                https://people.canonical.com/~ubuntu-archive/germinate-output/i386.focal/i386+build-depends.sources \
>              | awk '{ print $1 }' | sort) \
>            <(cat /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_focal_*Sources \
>              | grep-dctrl -r -FTestsuite . -a '!' -FArchitecture -X all \
>                           -sPackage -n | sort)
>
> On Thu, Nov 28, 2019 at 12:46:34PM -0800, Steve Langasek wrote:
> > Thanks to thorough feedback from our community, we now have a reasonably
> > comprehensive answer to the question of what 32-bit compatibility library
> > packages are needed on x86 for Ubuntu 20.04.
> >
> > https://discourse.ubuntu.com/t/community-process-for-32-bit-compatibility/12598/46
> >
> > Some developers will have noticed changes this week to the behavior of focal
> > builds in Launchpad.  Out of 30,000 source packages in focal, there is now a
> > whitelist of about 1,700 source packages which will trigger builds on i386
> > in Launchpad.  This means that other packages which previously built on i386
> > will need to have the binaries from the old version of the package removed
> > before they will be migratable from focal-proposed to focal.
> >
> >
> > As a side note, the implementation of this also affects PPA builds, because
> > the whitelist applies to the focal series as a whole.  In general, you
> > should not expect to need i386 builds of third-party packages in PPAs for
> > focal either, given that i386 in focal exists solely for compatibility with
> > legacy binary software.  However, if you have a third-party package that you
> > believe it's important to continue producing i386 binary builds of in
> > Launchpad for Ubuntu 20.04, please contact the Ubuntu archive admins
> > ([hidden email], or #ubuntu-devel on freenode.net for best
> > results), and we can evaluate including your PPA package in the whitelist.
> >
> >
> > At the moment, I am doing some manual removals of the i386 binaries as I see
> > them show up as blockers on
> > https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html
> > and as I'm able to determine that the removals aren't going to cause
> > near-term knock-on problems.  But if some i386 binaries aren't being removed
> > fast enough and this is blocking your work, feel free to reach out to an
> > archive admin to ask for their removal.
> >
> > In the slightly less near term, the plan is to do a mass binary removal of
> > all of the i386 binary packages in focal built from sources other than those
> > in the whitelist.  However, before pulling the trigger on this mass removal,
> > there are some changes that should be landed to our autopkgtest
> > infrastructure, so that we can continue to run autopkgtests for those
> > remaining 1700 packages.  In summary: the plan is not to retain the test
> > dependencies of those 1700 packages on i386, but instead to cross-test the
> > i386 libraries on an amd64 host, which ultimately means testing them in an
> > environment that better models the expected real-world usage.  The work is
> > in progress for this change and I'm currently anticipating landing it next
> > week.
> >
> > In the meantime, if you need any help getting packages migrating to the
> > focal release from -proposed, please reach out on #ubuntu-release on IRC.
> >
> > Thanks,
> > --
> > Steve Langasek                   Give me a lever long enough and a Free OS
> > Debian Developer                   to set it on, and I can move the world.
> > Ubuntu Developer                                   https://www.debian.org/
> > [hidden email]                                     [hidden email]

> acl
> alglib
> alsa-lib
> ann
> aom
> apache2
> apparmor
> apt
> atk1.0
> at-spi2-atk
> at-spi2-core
> attr
> bash
> bind9
> binutils
> bluez
> boost1.67
> brotli
> bubblewrap
> bzip2
> cairo
> capnproto
> ceph
> cfitsio
> cluster-glue
> clutter-1.0
> cmake
> colord
> confuse
> coreutils
> corosync
> cunit
> cups
> cython
> dash
> datefudge
> dbus
> dbus-glib
> dbus-python
> dconf
> devscripts
> dietlibc
> distro-info
> dlm
> docbook-to-man
> double-conversion
> doxygen
> dpkg
> dxvk
> ed
> espeak
> espeak-ng
> evolution-data-server
> fakeroot
> faketime
> faudio
> fenix
> fenix-plugins
> ffmpeg
> fftw3
> fig2dev
> firebird3.0
> flite
> fluidsynth
> fpc
> freeglut
> fribidi
> fyba
> game-data-packager
> gcc-8
> gcc-9
> gcc-snapshot
> gconf
> gcr
> gdbm
> gdisk
> gdk-pixbuf
> gem2deb
> geocode-glib
> gl2ps
> glib2.0
> glibc
> glib-networking
> gnupg2
> gnuplot
> gnutls28
> gobject-introspection
> gpgme1.0
> graphene
> graphite2
> graphviz
> grep
> gspell
> gtk+2.0
> gtk+3.0
> gtkmm2.4
> gvfs
> gzip
> harfbuzz
> hwloc
> ibus
> icu
> imagemagick
> inetutils
> initramfs-tools
> init-system-helpers
> iproute2
> iptables
> jdupes
> jemalloc
> json-glib
> kconfig
> keyutils
> krb5
> kronosnet
> ladspa-sdk
> lapack
> lbfgsb
> leveldb
> libanyevent-perl
> libarchive
> libassuan
> libasync-interrupt-perl
> libb-hooks-op-check-perl
> libbsd-resource-perl
> libcaca
> libcap2
> libclone-perl
> libcommon-sense-perl
> libdaemon
> libdatrie
> libdevel-callchecker-perl
> libevent-perl
> libev-perl
> libfile-libmagic-perl
> libfilter-perl
> libgd2
> libgdata
> libgd-perl
> libglib-perl
> libgpg-error
> libgphoto2
> libhtml-parser-perl
> libical3
> libinput
> libio-pty-perl
> libiptcdata
> libjsoncpp
> libkate
> liblinux-epoll-perl
> liblist-moreutils-perl
> liblocale-gettext-perl
> libmicrohttpd
> libnetaddr-ip-perl
> libnet-ssleay-perl
> libnfs
> libnotify
> libofa
> liboggz
> libomxil-bellagio
> libopenmpt
> libparams-classify-perl
> libparams-util-perl
> libparams-validate-perl
> libperlio-gzip-perl
> libpipeline
> libproxy
> libpsl
> libqb
> libraw
> librsvg
> libsass
> libsdl2
> libsdl-sge
> libseccomp
> libsecret
> libsocket6-perl
> libsort-key-perl
> libsoup2.4
> libsoxr
> libssh
> libstb
> libsub-identify-perl
> libsub-name-perl
> libtaint-runtime-perl
> libtest-leaktrace-perl
> libtest-taint-perl
> libtext-charwidth-perl
> libtext-iconv-perl
> libthai
> libtheora
> libtommath
> libunicode-utf8-perl
> libusb-1.0
> libutempter
> libuuid-perl
> libuv1
> libvariable-magic-perl
> libvisual
> libvncserver
> libvorbis
> libvpx
> libwacom
> libwant-perl
> libwmf
> libxcb
> libxml2
> libxml-libxml-perl
> libxml-parser-perl
> libyaml-libyaml-perl
> libzstd
> linux
> livecd-rootfs
> llvm-toolchain-9
> matplotlib
> mawk
> mir
> mpi4py
> mpich
> multipath-tools
> munge
> mysql-8.0
> ncurses
> ndctl
> network-manager
> nghttp2
> ninja-build
> nss-mdns
> nss-wrapper
> numpy
> openjdk-lts
> openldap
> openssh
> openssl
> pacemaker
> pango1.0
> perl
> php7.3
> pillow
> pinentry
> policykit-1
> poppler
> postgresql-12
> protobuf
> pulseaudio
> pupnp-1.8
> pygobject
> pypy
> pyqt5
> python2.7
> python3.7
> python3.8
> python-apt
> python-bcrypt
> python-cffi
> python-coverage
> python-cryptography
> python-evdev
> python-lz4
> python-nacl
> python-numpy
> python-scandir
> python-scipy
> py-ubjson
> pyyaml
> raqm
> re2
> re2c
> rhash
> rsync
> ruby2.5
> ruby-defaults
> ruby-json
> ruby-nokogiri
> samba
> sane-backends
> sdlgfx
> sed
> snapd-glib
> sonic
> spdlog
> stunnel4
> superlu
> svox
> swig
> systemd
> systemtap
> talloc
> tdb
> twisted
> ubuntu-drivers-common
> uchardet
> udisks2
> umockdev
> update-notifier
> upower
> util-linux
> valgrind
> vim
> vlc
> vorbis-tools
> vtk6
> vtk7
> x264
> x265
> xapian-core
> xdg-dbus-proxy
> xterm
> zope.interface
> zsh
> zvbi

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

i386-stats (1K) Download Attachment
signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: i386 autopkgtest [Re: i386 in focal: an update]

Steve Langasek-6
A further update - after some judicious pruning of the build-dependencies of
gst-plugins-bad1.0, the total number of i386 packages for autopkgtesting is
reduced from 311 to 299 (attached).

Here is the new list of packages with autopkgtests that warrant
investigation.  We now have 29% of the i386 autopkgtests passing, 24%
confirmed not testable, and 47% still needing investigation.

I have also now added 'badtest' hints for the current version of each of
these remaining packages, so as of now they should no longer be causing
proposed-migration delays for any other packages.  As you upload new
versions of these packages, please make sure to do the analysis of the
autopkgtest failures and work with the release team to either fix the tests
or have them permanently blacklisted, as appropriate.

success: 86
blacklisted: 73
failed, needs investigation: 140
acl
alsa-lib
ann
bind9
binutils
boost1.67
brotli
bubblewrap
bzip2
capnproto
cluster-glue
cmake
colord
confuse
cunit
cups
cython
datefudge
dbus
devscripts
dietlibc
dlm
docbook-to-man
double-conversion
doxygen
fakeroot
faketime
fenix
fenix-plugins
fftw3
freeglut
fribidi
game-data-packager
gcc-8
gcc-9
gcc-snapshot
gdbm
gem2deb
glibc
gnupg2
gpgme1.0
graphite2
grep
gtk+3.0
gvfs
gzip
ibus
icu
inetutils
init-system-helpers
iproute2
jemalloc
krb5
kronosnet
lapack
lbfgsb
libarchive
libassuan
libcaca
libcap2
libdaemon
libdatrie
libgd2
libgpg-error
libgphoto2
libiptcdata
libjsoncpp
libkate
libmicrohttpd
libnfs
libofa
libomxil-bellagio
libpipeline
libpsl
libqb
libraw
libsass
libsdl2
libsdl-sge
libseccomp
libsoxr
libthai
libtommath
libusb-1.0
libutempter
libuv1
libvisual
libvncserver
libxcb
libxml2
livecd-rootfs
llvm-toolchain-9
matplotlib
mir
multipath-tools
ncurses
network-manager
nghttp2
nss-wrapper
numpy
openjdk-lts
openssh
openssl
pacemaker
perl
pillow
pinentry
postgresql-12
protobuf
pupnp-1.8
python2.7
python3.7
python3.8
python-apt
python-bcrypt
python-cffi
python-coverage
python-evdev
python-numpy
python-scipy
pyyaml
re2
ruby2.5
ruby-json
ruby-nokogiri
samba
sane-backends
sdlgfx
stunnel4
systemd
systemtap
talloc
tdb
ubuntu-drivers-common
uchardet
udisks2
valgrind
vim
xterm
zsh

On Tue, Dec 10, 2019 at 11:03:30AM -0800, Steve Langasek wrote:

> Hi all,
>
> After a few days of bugfixing tests to be compatible with the new
> autopkgtest cross-arch infrastructure, I thought it would be useful to give
> folks some stats, and point you to a list of packages requiring further
> investigation that you can help drive down.
>
> Of 311 autopkgtests that we care about on i386, 24% are currently passing,
> 8% have been confirmed as not-cross-testable and blacklisted by the release
> team, and 68% still need investigation.  Details, including a list of
> packages to investigate, produced by the attached script:

>
> success: 74
> blacklisted: 26
> failed, needs investigation: 211
> acl
> alglib
> alsa-lib
> ann
> bind9
> binutils
> boost1.67
> brotli
> bubblewrap
> bzip2
> capnproto
> cfitsio
> cluster-glue
> cmake
> colord
> confuse
> coreutils
> cunit
> cups
> cython
> datefudge
> dbus
> devscripts
> dietlibc
> distro-info
> dlm
> docbook-to-man
> double-conversion
> doxygen
> faketime
> fenix
> fenix-plugins
> fftw3
> fpc
> freeglut
> fribidi
> fyba
> game-data-packager
> gcc-8
> gcc-9
> gcc-snapshot
> gdbm
> gem2deb
> gl2ps
> glibc
> gnupg2
> gpgme1.0
> graphite2
> grep
> gtk+2.0
> gtk+3.0
> gvfs
> gzip
> harfbuzz
> ibus
> icu
> inetutils
> init-system-helpers
> iproute2
> jemalloc
> json-glib
> krb5
> kronosnet
> lapack
> lbfgsb
> libanyevent-perl
> libarchive
> libassuan
> libasync-interrupt-perl
> libb-hooks-op-check-perl
> libbsd-resource-perl
> libcaca
> libcap2
> libclone-perl
> libcommon-sense-perl
> libdaemon
> libdatrie
> libdevel-callchecker-perl
> libevent-perl
> libev-perl
> libfile-libmagic-perl
> libfilter-perl
> libgd2
> libgd-perl
> libgpg-error
> libgphoto2
> libhtml-parser-perl
> libinput
> libio-pty-perl
> libiptcdata
> libjsoncpp
> libkate
> liblist-moreutils-perl
> liblocale-gettext-perl
> libmicrohttpd
> libnetaddr-ip-perl
> libnet-ssleay-perl
> libnfs
> libofa
> libomxil-bellagio
> libopenmpt
> libparams-classify-perl
> libparams-util-perl
> libparams-validate-perl
> libperlio-gzip-perl
> libpipeline
> libproxy
> libpsl
> libqb
> libraw
> libsass
> libsdl2
> libsdl-sge
> libseccomp
> libsecret
> libsocket6-perl
> libsort-key-perl
> libsoup2.4
> libsoxr
> libssh
> libsub-identify-perl
> libsub-name-perl
> libtaint-runtime-perl
> libtest-leaktrace-perl
> libtest-taint-perl
> libtext-charwidth-perl
> libtext-iconv-perl
> libthai
> libtommath
> libunicode-utf8-perl
> libusb-1.0
> libutempter
> libuuid-perl
> libuv1
> libvariable-magic-perl
> libvisual
> libvncserver
> libvorbis
> libwacom
> libwant-perl
> libxcb
> libxml2
> libxml-libxml-perl
> libxml-parser-perl
> libyaml-libyaml-perl
> linux
> livecd-rootfs
> llvm-toolchain-9
> matplotlib
> mir
> mpi4py
> multipath-tools
> ncurses
> network-manager
> nghttp2
> nss-wrapper
> numpy
> openjdk-lts
> openssh
> openssl
> pacemaker
> perl
> pillow
> pinentry
> poppler
> postgresql-12
> protobuf
> pulseaudio
> pupnp-1.8
> python2.7
> python3.7
> python3.8
> python-apt
> python-bcrypt
> python-cffi
> python-coverage
> python-cryptography
> python-evdev
> python-lz4
> python-nacl
> python-numpy
> python-scipy
> py-ubjson
> pyyaml
> raqm
> re2
> ruby2.5
> ruby-json
> ruby-nokogiri
> samba
> sane-backends
> sdlgfx
> sed
> stunnel4
> superlu
> systemd
> systemtap
> talloc
> tdb
> twisted
> ubuntu-drivers-common
> uchardet
> udisks2
> upower
> util-linux
> valgrind
> vtk6
> vtk7
> xterm
> zope.interface
> zsh
>
> Cheers,
> --
> Steve Langasek                   Give me a lever long enough and a Free OS
> Debian Developer                   to set it on, and I can move the world.
> Ubuntu Developer                                   https://www.debian.org/
> [hidden email]                                     [hidden email]
>
> On Fri, Dec 06, 2019 at 03:57:21PM -0800, Steve Langasek wrote:
> > Hi folks,
> >
> > The changes to i386 have proceeded as described in my earlier email.  The
> > autopkgtest infrastructure has been converted to run tests of i386 binaries
> > on an amd64 VM, and i386 binaries not in the whitelist are now in the
> > process of being removed.  As of this moment, there are 1,679 source
> > packages in the whitelist for i386, and 8,921 source packages with published
> > i386 binary packages in focal, vs. 14,344 source packages with published
> > amd64 binaries.  At the current rate, I expect the binary removals to
> > complete some time tomorrow.
> >
> > On the autopkgtest side, there are now a total of 311 source packages
> > producing i386 binaries which also have autopkgtests[1].  Any packages
> > reporting autopkgtest regressions on i386 which are not on this list should
> > have their test failures ignored; if you see i386 autopkgtest failures for
> > other packages blocking migrations out of focal-proposed, please ask a
> > member of the release team to override.
> >
> > It's good that the number of remaining autopkgtests for i386 is as small as
> > it is, because while the /infrastructure/ is now capable of cross-testing
> > i386 packages on amd64, the vast majority of these tests don't run correctly
> > in such an environment without modification.  I have triggered runs of all
> > of these tests on the amd64 testbed, so current status of the tests is
> > visible on autopkgtest.ubuntu.com for each package.
> >
> > Since these tests have also de facto regressed in the release pocket due to
> > changes to the infrastructure, it is appropriate for those test failures to
> > be ignored for proposed-migration purposes as well.  I intend to go through
> > and systematically mark these as bad tests, but in the meantime if you have
> > package migrations that are being blocked by one of these, please reach out.
> >
> >
> > The other thing that you can do is help fix the failing autopkgtests, so
> > that we have better test coverage on i386.  I've identified a few common
> > patterns so far in the test failures:
> >
> >  - Tests of Essential or Priority: required/import packages that require removing
> >    the amd64 package in favor of the i386 package (e.g.: bash, apt,
> >    systemd).  These tests will never work, and - outside of launchpad
> >    builds, where we are still building i386 natively instead of on an amd64
> >    host - these packages aren't relevant to the i386 experience on focal.
> >    These test failures should be permanently ignored.
> >
> >  - Tests with test dependencies on python on perl modules.  It's a
> >    longstanding known issue with multiarch that python and perl modules pose
> >    problems for cross-installation, because they depend on the interpreter
> >    of the matching architecture.  Neither perl nor python can be
> >    cross-installed on the testbed (the amd64 versions are both part of the
> >    test infrastructure), so these tests will always fail with 'badpkg'.
> >    While some of these tests may be more relevant to the behavior of our
> >    i386 runtime libraries, fixing them is largely intractable so these are a
> >    low priority.
> >
> >  - Tests with other uninstallable test dependencies ("badpkg").  Many of
> >    these are going to be fixable, but require either annotations of
> >    additional packages in the archive as Multi-Arch: foreign (if
> >    appropriate), or annotations of the test dependencies in
> >    debian/tests/control with architecture information.  The autopkgtest
> >    dependency resolver for cross-testing uses apt-get build-dep, so that the
> >    full range of multiarch build-dependency specifiers are available as
> >    described at <https://wiki.ubuntu.com/MultiarchCross>.
> >
> >  - Build tests.  These are common tests for libraries, and some of the
> >    easiest to fix, but most don't work out of the box because they don't
> >    invoke the cross-toolchain.  Here are some example patches for various
> >    types of build systems used in build tests:
> >    - pkg-config: https://bugs.debian.org/946303
> >    - meson: https://bugs.debian.org/946292
> >
> > Happy fixing,
> > --
> > Steve Langasek                   Give me a lever long enough and a Free OS
> > Debian Developer                   to set it on, and I can move the world.
> > Ubuntu Developer                                   https://www.debian.org/
> > [hidden email]                                     [hidden email]
> >
> > [1] See attachment.  List calculated by:
> > $ join -j1 <(wget -O - -q -o /dev/null \
> >                https://people.canonical.com/~ubuntu-archive/germinate-output/i386.focal/i386+build-depends.sources \
> >              | awk '{ print $1 }' | sort) \
> >            <(cat /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_focal_*Sources \
> >              | grep-dctrl -r -FTestsuite . -a '!' -FArchitecture -X all \
> >                           -sPackage -n | sort)
> >
> > On Thu, Nov 28, 2019 at 12:46:34PM -0800, Steve Langasek wrote:
> > > Thanks to thorough feedback from our community, we now have a reasonably
> > > comprehensive answer to the question of what 32-bit compatibility library
> > > packages are needed on x86 for Ubuntu 20.04.
> > >
> > > https://discourse.ubuntu.com/t/community-process-for-32-bit-compatibility/12598/46
> > >
> > > Some developers will have noticed changes this week to the behavior of focal
> > > builds in Launchpad.  Out of 30,000 source packages in focal, there is now a
> > > whitelist of about 1,700 source packages which will trigger builds on i386
> > > in Launchpad.  This means that other packages which previously built on i386
> > > will need to have the binaries from the old version of the package removed
> > > before they will be migratable from focal-proposed to focal.
> > >
> > >
> > > As a side note, the implementation of this also affects PPA builds, because
> > > the whitelist applies to the focal series as a whole.  In general, you
> > > should not expect to need i386 builds of third-party packages in PPAs for
> > > focal either, given that i386 in focal exists solely for compatibility with
> > > legacy binary software.  However, if you have a third-party package that you
> > > believe it's important to continue producing i386 binary builds of in
> > > Launchpad for Ubuntu 20.04, please contact the Ubuntu archive admins
> > > ([hidden email], or #ubuntu-devel on freenode.net for best
> > > results), and we can evaluate including your PPA package in the whitelist.
> > >
> > >
> > > At the moment, I am doing some manual removals of the i386 binaries as I see
> > > them show up as blockers on
> > > https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html
> > > and as I'm able to determine that the removals aren't going to cause
> > > near-term knock-on problems.  But if some i386 binaries aren't being removed
> > > fast enough and this is blocking your work, feel free to reach out to an
> > > archive admin to ask for their removal.
> > >
> > > In the slightly less near term, the plan is to do a mass binary removal of
> > > all of the i386 binary packages in focal built from sources other than those
> > > in the whitelist.  However, before pulling the trigger on this mass removal,
> > > there are some changes that should be landed to our autopkgtest
> > > infrastructure, so that we can continue to run autopkgtests for those
> > > remaining 1700 packages.  In summary: the plan is not to retain the test
> > > dependencies of those 1700 packages on i386, but instead to cross-test the
> > > i386 libraries on an amd64 host, which ultimately means testing them in an
> > > environment that better models the expected real-world usage.  The work is
> > > in progress for this change and I'm currently anticipating landing it next
> > > week.
> > >
> > > In the meantime, if you need any help getting packages migrating to the
> > > focal release from -proposed, please reach out on #ubuntu-release on IRC.
Thanks,
--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                   https://www.debian.org/
[hidden email]                                     [hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

i386-whitelisted-autopkgtests.txt (3K) Download Attachment
signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: i386 autopkgtest [Re: i386 in focal: an update]

Andreas Hasenack-5
In reply to this post by Steve Langasek-6
Hello,

On Tue, Dec 10, 2019 at 4:04 PM Steve Langasek
<[hidden email]> wrote:
>

> team, and 68% still need investigation.  Details, including a list of
> packages to investigate, produced by the attached script:
>
> success:                        74
> blacklisted:                    26
> failed, needs investigation:    211

> krb5

I grabbed krb5 for a look, and I think it boils down to missing
Multi-Arch in krb5-config (src: kerberos-configs) and krb5-doc (src:
krb5).

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel