Updated archive rebuild results

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Updated archive rebuild results

Lucas Nussbaum
Long time no see... And 1474 packages failed to build!

http://udd.debian.org/cgi-bin/ubuntu_ftbfs.cgi

- Lucas

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Barry Warsaw-2
On Dec 03, 2010, at 08:23 AM, Lucas Nussbaum wrote:

>Long time no see... And 1474 packages failed to build!
>
>http://udd.debian.org/cgi-bin/ubuntu_ftbfs.cgi

Thanks.  I was making some progress on the graphviz failure before my stint as
patch pilot yesterday.  Hopefully I'll have a branch to fix this later today.
Ongoing, I'll be looking at all the Python related ones, but if anybody wants
to do triage on them, it would be helpful to make sure there are bugs filed
and tagged with 'python27' when they are related to the Python 2.7 transition.

Cheers,
-Barry

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Scott Howard-5
> On Dec 03, 2010, at 08:23 AM, Lucas Nussbaum wrote:
>
>>Long time no see... And 1474 packages failed to build!

There are lots of:
[LD_ERROR] libfoo.so: could not read symbols: Invalid operation

due to binutils-gold not doing indirect linking. Is there an organized
effort to address them? They all just need to have -lfoo added to
their compiler flags. Since Debian is frozen and such failures would
not qualify as an RC bugs, do we need diffs from debian for all those
packages and just tell debian about the patch? They are usually pretty
easy fixes, I'll try to tackle some soon.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Matthias Klose-6
On 03.12.2010 18:11, Scott Howard wrote:

>> On Dec 03, 2010, at 08:23 AM, Lucas Nussbaum wrote:
>>
>>> Long time no see... And 1474 packages failed to build!
>
> There are lots of:
> [LD_ERROR] libfoo.so: could not read symbols: Invalid operation
>
> due to binutils-gold not doing indirect linking. Is there an organized
> effort to address them? They all just need to have -lfoo added to
> their compiler flags. Since Debian is frozen and such failures would
> not qualify as an RC bugs, do we need diffs from debian for all those
> packages and just tell debian about the patch? They are usually pretty
> easy fixes, I'll try to tackle some soon.

see http://wiki.debian.org/ToolChain/DSOLinking

for a list of bug reports:
http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=no-add-needed;users=peter.fritzsche@...

if you add patches, please submit these to the existing bug reports, if you
submit new patches, please use the same usertags:

Tags: patch
User: [hidden email]
Usertags: origin-ubuntu natty ubuntu-patch
User: [hidden email]
Usertags: no-add-needed


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Scott Howard-5
On Fri, Dec 3, 2010 at 12:38 PM, Matthias Klose <[hidden email]> wrote:
> see http://wiki.debian.org/ToolChain/DSOLinking

Thanks, I'm familiar with the Debian effort and agree this change is a
good thing. My question is more on the Ubuntu side: there are ~400
packages (or more) that would require diffs from Debian to fix since
Debian is frozen. A few of the fixes might make it to unstable (or
experimental), but overall that's a big chunk that will be in Ubuntu
this release and not in Debian. Ubuntu will need a decent push to get
all those packages fixed up in time for the next release or the
release would have lots of FTBFS in the archive (even though the
packages that have already been compiled are still usable.) Is than an
Ubuntu effort to address all those packages, or is it a low priority
since nothing is "broken" from an end user point of view?

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Matthias Klose-6
On 03.12.2010 19:15, Scott Howard wrote:
> On Fri, Dec 3, 2010 at 12:38 PM, Matthias Klose<[hidden email]>  wrote:
>> see http://wiki.debian.org/ToolChain/DSOLinking
>
> Thanks, I'm familiar with the Debian effort and agree this change is a
> good thing. My question is more on the Ubuntu side: there are ~400
> packages (or more) that would require diffs from Debian to fix since
> Debian is frozen.

where do you get this number?  It is definitely not 400 or more.

   Matthias

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Scott Howard-4
On Fri, Dec 3, 2010 at 4:42 PM, Matthias Klose <[hidden email]> wrote:
> where do you get this number?  It is definitely not 400 or more.

I may be completely off base here, but I got that number from two
independent methods.

First, by parsing the build logs at [1], I found a list of 434
packages which match a build failure of the following form:
/usr/bin/ld: XXXX.o: undefined reference to symbol 'XXXX'
/usr/bin/ld: note: 'XXXX' is defined in DSO /usr/lib/libXXXX.so.X so
try adding it to the linker command line

I placed a list of those packages online here [2]. Those FTBFS are
solely from the change that removes indirect linking, as they did
build in Maverick.


Second, from [3], there are 416 packages that have not been fixed in
Debian yet, so there is a high probability that those packages still
FTBFS in Natty. Those bugs include all forms of FTBFS caused by the
DSO linking changes. There are also an additional 298 packages that
have been fixed in Debian that may have been fixed in experimental
(because of the freeze) and would require manual syncing.

Those two data points, combined, leads me to believe that there are
400+ packages that are failing to build in Natty. Because of the
Debian freeze, we should not expect fixes to end up in unstable and
thus propagate to Ubuntu. Ubuntu would have to create diffs to fix
those FTBFS.

[1] http://udd.debian.org/cgi-bin/ubuntu_ftbfs.cgi
[2] https://wiki.ubuntu.com/ScottHoward/FTBFSreport
[3] http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=no-add-needed;users=peter.fritzsche@...
# Outstanding bugs -- Important bugs; Patch Available (22 bugs)
# Outstanding bugs -- Minor bugs; Patch Available (7 bugs)
# Outstanding bugs -- Minor bugs; Confirmed (2 bugs)
# Outstanding bugs -- Minor bugs; Unclassified (363 bugs)
# Outstanding bugs -- Minor bugs; More information needed (2 bugs)
# Forwarded bugs -- Important bugs (1 bug)
# Forwarded bugs -- Normal bugs (1 bug)
# Forwarded bugs -- Minor bugs (18 bugs)

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Scott Howard-5
On Sat, Dec 4, 2010 at 1:41 PM, Evan Broder <[hidden email]> wrote:

> On Sat, Dec 4, 2010 at 8:17 AM, Scott Howard <[hidden email]> wrote:
>> Those two data points, combined, leads me to believe that there are
>> 400+ packages that are failing to build in Natty. Because of the
>> Debian freeze, we should not expect fixes to end up in unstable and
>> thus propagate to Ubuntu. Ubuntu would have to create diffs to fix
>> those FTBFS.
>
> I think your assumptions are a little off. Plenty of DDs are more than
> happy to upload fixes for binutils-gold errors into unstable. I've
> fixed at least one FTBFS where I knew the Debian maintainer was
> responsive by sending the patch off, waiting for it to be uploaded to
> unstable, and then syncing it back into Debian. No Ubuntu diff needed.

My problem isn't with the diff as much as there is work that needs to
be done, and it needs to be initiated from within Ubuntu since these
are not RC bugs to Debian until after squeeze is released. I'm not
saying there isn't coordination, just that if Ubuntu waits for Debian,
it won't get done until after Squeeze is released because DD/DMs
shouldn't upload to unstable unless there is an RC bug fix during
freeze. These FTBFS are not RC bugs to Debian Squeeze (but are for
Debian Weezy). Exactly as you said, you had to do the work to fix the
FTBFS - which is great, we need more of that. Diff or no-diff, we need
to do that 400 more times before natty.

I'm a DM that received a bug report from ubuntu with a patch, which I
immediately uploaded to experimental and requested sync to ubuntu.
Even if every maintainer does that, there are still 400+ packages that
need to be reviewed, patched, uploaded to debian (most likely
experimental), and manually synced to Ubuntu (since it is in
experimental). It is doable, we just need to be aware of it and get
started. My point is that if we don't put an organized effort into it,
a large number of packages won't be fixed in time for natty.

Cheers.
Scott

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Micah Gersten-8
On 12/04/2010 01:53 PM, Scott Howard wrote:

> On Sat, Dec 4, 2010 at 1:41 PM, Evan Broder <[hidden email]> wrote:
>> On Sat, Dec 4, 2010 at 8:17 AM, Scott Howard <[hidden email]> wrote:
>>> Those two data points, combined, leads me to believe that there are
>>> 400+ packages that are failing to build in Natty. Because of the
>>> Debian freeze, we should not expect fixes to end up in unstable and
>>> thus propagate to Ubuntu. Ubuntu would have to create diffs to fix
>>> those FTBFS.
>> I think your assumptions are a little off. Plenty of DDs are more than
>> happy to upload fixes for binutils-gold errors into unstable. I've
>> fixed at least one FTBFS where I knew the Debian maintainer was
>> responsive by sending the patch off, waiting for it to be uploaded to
>> unstable, and then syncing it back into Debian. No Ubuntu diff needed.
> My problem isn't with the diff as much as there is work that needs to
> be done, and it needs to be initiated from within Ubuntu since these
> are not RC bugs to Debian until after squeeze is released. I'm not
> saying there isn't coordination, just that if Ubuntu waits for Debian,
> it won't get done until after Squeeze is released because DD/DMs
> shouldn't upload to unstable unless there is an RC bug fix during
> freeze. These FTBFS are not RC bugs to Debian Squeeze (but are for
> Debian Weezy). Exactly as you said, you had to do the work to fix the
> FTBFS - which is great, we need more of that. Diff or no-diff, we need
> to do that 400 more times before natty.
>
> I'm a DM that received a bug report from ubuntu with a patch, which I
> immediately uploaded to experimental and requested sync to ubuntu.
> Even if every maintainer does that, there are still 400+ packages that
> need to be reviewed, patched, uploaded to debian (most likely
> experimental), and manually synced to Ubuntu (since it is in
> experimental). It is doable, we just need to be aware of it and get
> started. My point is that if we don't put an organized effort into it,
> a large number of packages won't be fixed in time for natty.
>
> Cheers.
> Scott
>

What a lot of MOTUs have been doing is fixing the FTBFS in Ubuntu and
sending the patch to Debian as a wishlist bug with the appropriate
tags.  Then just watch for the package to get the fix in Debian and
request a sync when it's ready.  Sync's are processed pretty regularly
now, so that's not really an issue.

Micah

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Matthias Klose-6
In reply to this post by Scott Howard-5
On 04.12.2010 20:53, Scott Howard wrote:

> On Sat, Dec 4, 2010 at 1:41 PM, Evan Broder<[hidden email]>  wrote:
>> On Sat, Dec 4, 2010 at 8:17 AM, Scott Howard<[hidden email]>  wrote:
>>> Those two data points, combined, leads me to believe that there are
>>> 400+ packages that are failing to build in Natty. Because of the
>>> Debian freeze, we should not expect fixes to end up in unstable and
>>> thus propagate to Ubuntu. Ubuntu would have to create diffs to fix
>>> those FTBFS.
>>
>> I think your assumptions are a little off. Plenty of DDs are more than
>> happy to upload fixes for binutils-gold errors into unstable. I've
>> fixed at least one FTBFS where I knew the Debian maintainer was
>> responsive by sending the patch off, waiting for it to be uploaded to
>> unstable, and then syncing it back into Debian. No Ubuntu diff needed.
>
> My problem isn't with the diff as much as there is work that needs to
> be done, and it needs to be initiated from within Ubuntu since these
> are not RC bugs to Debian until after squeeze is released. I'm not
> saying there isn't coordination, just that if Ubuntu waits for Debian,
> it won't get done until after Squeeze is released because DD/DMs
> shouldn't upload to unstable unless there is an RC bug fix during
> freeze. These FTBFS are not RC bugs to Debian Squeeze (but are for
> Debian Weezy). Exactly as you said, you had to do the work to fix the
> FTBFS - which is great, we need more of that. Diff or no-diff, we need
> to do that 400 more times before natty.

So how long do you wait?  Until after these issues are all fixed in Debian?
Surely that is the least amount of work, but do we want to wait that long?  If
changes are made in Debian and Ubuntu starts to merge these changes, then Ubuntu
still has to go ahead for all these reports for the next Ubuntu release (if this
release is before the next Debian release).

Looking at our current toolchain "diff":

  - GCC-4.5: Natty started to use GCC-4.5.  Issues are reported in Debian
    as [1], these are reproducible by installing gcc/g++ and gcc-4.5/g++-4.5
    from experimental).

This was planned at the maverick UDS. Better code generation in 4.5 than in 4.4.

  - Build failures with the Gold linker, or with ld --no-add-needed.
    Reported as , reproducible by installing gcc/g++ and gcc-4.5/g++-4.5
    from experimental or by installing binutils-gold from experimental.

Planned and decided at the natty UDS. It's a correctness issue, and it will
enable us to further experiment/test with gold.

  - Build failures with ld --as-needed. Issues found at [3].

Discussed at the natty UDS.  Primary reason is to get dependencies down a bit
and ease library transitions (Ubuntu doesn't have binNMU's).  Before release
there's always a lot of cruft to clean up in NBS; these changes should reduce
the amount of clean-up work.

  - Build failures with glibc-2.12, currently not tracked with user tags,
    although I cannot remember more than a few issues.

Not (yet) changed in natty.

  - Build failures with more recent kernel headers, currently not tracked
    with user tags.

Changes not tested, fallout will be seen in packages using kernel headers. I
don't track these explicitly.

Large-scale changes always make more work than a simple library transition, but
there are reasons to make these changes because of the benefits.  You cannot
completely avoid this kind of work, even if Debian starts but Ubuntu needs to
finish earlier, so it's more predicatable to start these large-scale changes at
the beginning of a release cycle.

There will be one more large-scale change for natty, making python2.7 the
default.  Delaying that for 11.10 would make this version available more than
twelve months after the 2.7 release, which is a bit late. At least Ubuntu uses
Python for many distro-specific developments.  So we'll make the 2.7 switch
still in this year.

> I'm a DM that received a bug report from ubuntu with a patch, which I
> immediately uploaded to experimental and requested sync to ubuntu.
> Even if every maintainer does that, there are still 400+ packages that
> need to be reviewed, patched, uploaded to debian (most likely
> experimental), and manually synced to Ubuntu (since it is in
> experimental). It is doable, we just need to be aware of it and get
> started. My point is that if we don't put an organized effort into it,
> a large number of packages won't be fixed in time for natty.

About 100 packages are now fixed in natty, even with the holidays coming, these
looks promising to get these fixed until the feature freeze.

   Matthias

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Updated archive rebuild results

Scott Howard-4
On Mon, Dec 6, 2010 at 12:36 PM, Matthias Klose <[hidden email]> wrote:
> So how long do you wait?  Until after these issues are all fixed in Debian?
> Surely that is the least amount of work, but do we want to wait that long?  If
> changes are made in Debian and Ubuntu starts to merge these changes, then Ubuntu
> still has to go ahead for all these reports for the next Ubuntu release (if this
> release is before the next Debian release).

I never said we should wait, in fact my first email said it was the
right thing to do and should be done. I guess I wasn't clear enough.
I'm not saying we should avoid the work now, but actually the exact
opposite: I wanted to make sure people knew that this was going on so
that we would do the work now. My fear was FF arriving and there still
being many broken packages because not enough people were involved
because they were not aware of the change.

> About 100 packages are now fixed in natty, even with the holidays coming, these
> looks promising to get these fixed until the feature freeze.

That's all I wanted to know: we confident that it will get done by FF
at the current pace.

Regards,
Scott

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel