Uploading to multiple distros

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Uploading to multiple distros

Ian Jackson-2
Didier Raboud writes ("Re: Color Management in Debian"):
> Given that Debian is currently not frozen (and that the Oneiric
> release will very probably happen before Wheezy's), I really think
> that not uploading those packages to Debian first would be a shame,
> as this would only mean doubling efforts.

As a structural matter I think it's wrong to have either Debian or
Ubuntu end up being blocked on the other, if it can be avoided.  One
of the ways that free software really works well is that it enables
people to get their work done right away and worry about interacting
with others later.

One thing I have done a few times is to upload the very same package
simultaneously to (say) Debian and Ubuntu.

Where the package wants to be identical, and the person doing the
upload is the same, it would be nice if this could be made simpler.
At the moment you basically need to build the whole thing twice with
minor edits to the changelog to set the target suite.

It would be nice if this could be made simpler.  In principle it would
be nice if you could use the same .changes file for uploading to two
distros (provided that only one of them wants binaries) but that may
be too much to ask.

At the very least it should be possible to do one upload to two
distros without altering the debian/changelog.

For this to work I think the "suite" target in the changelog entry
should be qualified by the distro, so you could say (picking a random
example):

  picocom (1.4-1) debian:unstable ubuntu:oneiric; urgency=low

    * new upstream version:
      - Changed the behavior of the C-a command. Now pressing C-a
           twice sends C-a to the serial port
      - Added the C-\ command that generates a break sequence
      - Added (optional) UUCP-lockfiles support

   -- Oliver Kurth <[hidden email]>  Tue, 14 Sep 2004 11:25:07 -0700

or something.

What do people think ?

Ian.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Iain Lane-8
On Thu, Jun 02, 2011 at 01:20:08PM +0100, Ian Jackson wrote:

>Didier Raboud writes ("Re: Color Management in Debian"):
>> Given that Debian is currently not frozen (and that the Oneiric
>> release will very probably happen before Wheezy's), I really think
>> that not uploading those packages to Debian first would be a shame,
>> as this would only mean doubling efforts.
>
> [...]
>
>At the very least it should be possible to do one upload to two
>distros without altering the debian/changelog.
>
>For this to work I think the "suite" target in the changelog entry
>should be qualified by the distro, so you could say (picking a random
>example):
>
>  picocom (1.4-1) debian:unstable ubuntu:oneiric; urgency=low
>
>    * new upstream version:
>      - Changed the behavior of the C-a command. Now pressing C-a
>   twice sends C-a to the serial port
>      - Added the C-\ command that generates a break sequence
>      - Added (optional) UUCP-lockfiles support
>
>   -- Oliver Kurth <[hidden email]>  Tue, 14 Sep 2004 11:25:07 -0700
>
>or something.
You can do this already by using syncpackage(1) from ubuntu-dev-tools —
I don't think the archive software looks at the changelog, but the
distribution in the changes file.

For "normal" syncs we generally advise not using syncpackage, but it
might make sense when doing simultaneous uploads.

This at least works when uploading packages prepared for Debian to
Ubuntu. I'm not sure how well Debian's infrastructure would cope if the
changelog specifies an Ubuntu release.

Cheers,
Iain

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Ian Jackson-2
Iain Lane writes ("Re: Uploading to multiple distros"):
> You can do this already by using syncpackage(1) from ubuntu-dev-tools ?
> I don't think the archive software looks at the changelog, but the
> distribution in the changes file.

I wasn't aware of syncpackage.  The manpage is quite discouraging.

> For "normal" syncs we generally advise not using syncpackage, but it
> might make sense when doing simultaneous uploads.

Hrm.  So syncpackage generates a .changes for uploading to ubuntu from
the .dsc (which presumably came out of the Debian build).  That does
mean though that the Ubuntu target suite is not visible in the
changelog of the ultimate Ubuntu package.  And if the package is not
accepted into the Debian archive for any reason, the changelog is very
misleading because it looks like a sync from Debian.

> This at least works when uploading packages prepared for Debian to
> Ubuntu. I'm not sure how well Debian's infrastructure would cope if the
> changelog specifies an Ubuntu release.

Debian's archive tools don't look at the changelog either, I think.
But the .changes file is generated from it.  Both Debian and Ubuntu
will reject a .changes file containing unrecognised suites.

One way to enable simultaneous uploads would be to arrange for
dpkg-genchanges to filter out suites for "other" distros when
generating the .changes file.  Then you would have the same files
being uploaded but two different .changes files.

One .changes file would be nice but there are conceptual and practical
problems with it.

Ian.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Martin Owens-2
In reply to this post by Ian Jackson-2
On Thu, 2011-06-02 at 13:20 +0100, Ian Jackson wrote:
> For this to work I think the "suite" target in the changelog entry
> should be qualified by the distro, so you could say (picking a random
> example):
>
>   picocom (1.4-1) debian:unstable ubuntu:oneiric; urgency=low
>
>     * new upstream version:

Why not just make ppa uploading simpler at the same time? (if you can
change the changelog file processing):

picocom (1.4-1) debian:unstable oneiric natty maverick lucid;
urgency=low

Instead of pushing 6 different packages, we could push just the one.
Have the tools search for a match and build for that archive.

Martin,


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Peter Samuelson
In reply to this post by Ian Jackson-2

[Ian Jackson]
> Hrm.  So syncpackage generates a .changes for uploading to ubuntu from
> the .dsc (which presumably came out of the Debian build).  That does
> mean though that the Ubuntu target suite is not visible in the
> changelog of the ultimate Ubuntu package.  And if the package is not
> accepted into the Debian archive for any reason, the changelog is very
> misleading because it looks like a sync from Debian.

Since syncs from Debian are actually supposed to be the majority of
packages in Ubuntu anyway, why not just do that - a real sync, not a
fake simultaneous one.  I don't live in the Ubuntu dev universe, but
given how common of an operation this apparently is, I'd think the
tool(s) to do it would be mature and easy to use.  Sure, you have to
wait a few minutes until you get your ACCEPTED mail from dak, but what
of it?  The mail is your reminder to finish the job.

(And if the 'sync from debian' process can't pull your .dsc out of
incoming, that's something I imagine would be easy enough to add.)

> One way to enable simultaneous uploads would be to arrange for
> dpkg-genchanges to filter out suites for "other" distros when
> generating the .changes file.  Then you would have the same files
> being uploaded but two different .changes files.

I suppose you mean put that functionality into this syncpackage thing.
Because otherwise you're still going to need to run dpkg-genchanges
twice, once presumably by hand with some sort of suite override switch,
and that does rather break the abstractions of debian/rules, debhelper,
and the like.

--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Iain Lane-8
In reply to this post by Ian Jackson-2
On Thu, Jun 02, 2011 at 01:54:37PM +0100, Ian Jackson wrote:
>[...]
>Hrm.  So syncpackage generates a .changes for uploading to ubuntu from
>the .dsc (which presumably came out of the Debian build).  That does
>mean though that the Ubuntu target suite is not visible in the
>changelog of the ultimate Ubuntu package.  And if the package is not
>accepted into the Debian archive for any reason, the changelog is very
>misleading because it looks like a sync from Debian.

Yes, this is one reason why it's probably better to wait for acceptance
into Debian first before uploading anywhere else. NEW delays really
are rather short these days in my experience so in all but the most
urgent cases that would be the right thing to do.

Anyway, unless you're proposing changing the version string based on
target distribution there is going to be confusion regardless IMO. It's
simpler to just rebuild the source package with an Ubuntu changelog
entry or wait the couple of days to sync. :-)

Regards,
Iain

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Matt Zimmerman-2
In reply to this post by Ian Jackson-2
On Thu, Jun 02, 2011 at 01:54:37PM +0100, Ian Jackson wrote:
> Iain Lane writes ("Re: Uploading to multiple distros"):
> > For "normal" syncs we generally advise not using syncpackage, but it
> > might make sense when doing simultaneous uploads.
>
> Hrm.  So syncpackage generates a .changes for uploading to ubuntu from
> the .dsc (which presumably came out of the Debian build).  That does
> mean though that the Ubuntu target suite is not visible in the
> changelog of the ultimate Ubuntu package.

That's true for most Ubuntu packages anyway, since the source packages are
copied unmodified from Debian.  We've accepted this since the beginning, as
it's certainly not worth modifying the package just to have this be correct.

It's similar to how in Debian, the changelog says the package was uploaded
to unstable but now it's in testing or stable having propagated there
unmodified.

> And if the package is not accepted into the Debian archive for any reason,
> the changelog is very misleading because it looks like a sync from Debian.

True, but perhaps harmless.

> > This at least works when uploading packages prepared for Debian to
> > Ubuntu. I'm not sure how well Debian's infrastructure would cope if the
> > changelog specifies an Ubuntu release.
>
> Debian's archive tools don't look at the changelog either, I think.
> But the .changes file is generated from it.  Both Debian and Ubuntu
> will reject a .changes file containing unrecognised suites.
>
> One way to enable simultaneous uploads would be to arrange for
> dpkg-genchanges to filter out suites for "other" distros when
> generating the .changes file.  Then you would have the same files
> being uploaded but two different .changes files.

Yes, it seems like it would be straightforward enough to generate two
appropriate .changes files for this case, and it would do the right thing.

--
 - mdz

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Dustin Kirkland-5
In reply to this post by Ian Jackson-2
On Thu, Jun 2, 2011 at 8:20 AM, Ian Jackson
<[hidden email]> wrote:
...

> One thing I have done a few times is to upload the very same package
> simultaneously to (say) Debian and Ubuntu.
>
> Where the package wants to be identical, and the person doing the
> upload is the same, it would be nice if this could be made simpler.
> At the moment you basically need to build the whole thing twice with
> minor edits to the changelog to set the target suite.
>
> It would be nice if this could be made simpler.  In principle it would
> be nice if you could use the same .changes file for uploading to two
> distros (provided that only one of them wants binaries) but that may
> be too much to ask.
>
> At the very least it should be possible to do one upload to two
> distros without altering the debian/changelog.

I handle this using a set of scripts in Ubuntu's bikeshed package:
 * http://manpg.es/release-build
 * http://manpg.es/release

'release-build' builds me one binary package, which I test locally,
and gives me instructions on what to do next, if I want to proceed
with the upstream release and upload to a bunch of different places
simultaneously.  It also builds source packages for each of the
supported Ubuntu releases.  If my testing goes well, I'm informed to
run the 'release' command.

'release' then does a handful of things (like tagging the release,
bumping and opening the next version), and prints a series of commands
that I run manually, if I want to actually do the release (such as the
bzr push to the trunk, lp-project-upload to put the tarball on
Launchpad, dput of the backport changes file to that package's PPA,
and dput to Ubuntu Oneiric).  (If I had Debian Maintainer privileges,
I'd also simultaneously dput to Debian too).  I just used
release-test, release-build, and release to publish bikeshed-1.15, and
captured its log here:
 * http://paste.ubuntu.com/616969/

These helpers allow me to consistently release dozens of open source
projects simultaneously to the current development series of Ubuntu,
but also publish "backports" to PPAs of that same package for
Lucid/Maverick/Natty.  They keep me from making many of the most
common mistakes and allow me to upload pretty effortlessly.

These scripts are a little Ubuntu/Launchpad/Bzr specific, so I'm not
advocating their use to you (Debian), but acknowledging and supporting
Ian's suggestion to solve some duplication of effort!

--
:-Dustin

Dustin Kirkland
Ubuntu Core Developer

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Barry Warsaw
In reply to this post by Peter Samuelson
On Jun 02, 2011, at 10:28 AM, Peter Samuelson wrote:

>Since syncs from Debian are actually supposed to be the majority of
>packages in Ubuntu anyway, why not just do that - a real sync, not a
>fake simultaneous one.  I don't live in the Ubuntu dev universe, but
>given how common of an operation this apparently is, I'd think the
>tool(s) to do it would be mature and easy to use.  Sure, you have to
>wait a few minutes until you get your ACCEPTED mail from dak, but what
>of it?  The mail is your reminder to finish the job.

How can we help Ubuntu developers do the right thing?  Are there tools,
processes, or documentation that we can develop that would make it easy(er)
for someone who works on an Ubuntu machine, is well versed in Ubuntu culture
(processes, tools, etc), to integrate with Debian?

-Barry

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Andres Rodriguez-4
In reply to this post by Dustin Kirkland-5

I handle this using a set of scripts in Ubuntu's bikeshed package:
 * http://manpg.es/release-build
 * http://manpg.es/release

+1! I have been using these scripts extensively to do my releases, and preparing the packages to upload them into the archive or PPA's. Specifically for TestDrive and PowerNap. A common example is PowerNap, on which I use these scripts to prepare a release, test it, and once I'm satisfied, I upload them to the PPA's without having to do manual modifications to the changelogs and being able to easily release a backport in PPA for PowerNap.

These scripts are a little Ubuntu/Launchpad/Bzr specific, so I'm not
advocating their use to you (Debian), but acknowledging and supporting
Ian's suggestion to solve some duplication of effort!

I agree with you Dustin that the use of scripts like this can resolve some duplication! Even though they don't work with Debian itself, I don't see the problem of adapting them to handle both cases and then making life easier for everybody!

-- 
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
Systems Engineer

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Robert Collins
In reply to this post by Martin Owens-2
On Fri, Jun 3, 2011 at 2:40 AM, Martin Owens <[hidden email]> wrote:

> On Thu, 2011-06-02 at 13:20 +0100, Ian Jackson wrote:
>> For this to work I think the "suite" target in the changelog entry
>> should be qualified by the distro, so you could say (picking a random
>> example):
>>
>>   picocom (1.4-1) debian:unstable ubuntu:oneiric; urgency=low
>>
>>     * new upstream version:
>
> Why not just make ppa uploading simpler at the same time? (if you can
> change the changelog file processing):
>
> picocom (1.4-1) debian:unstable oneiric natty maverick lucid;
> urgency=low
>
> Instead of pushing 6 different packages, we could push just the one.
> Have the tools search for a match and build for that archive.

The challenge with ppas here  is that a package can only be built once
in a given archive: the binary package names are the same for all of
unstable/oneiric/natty/maverick/lucid : the APT archive which makes up
a PPA cannot present different binary versions for each series.

We could build only in lucid and then promote the package forward (the
same as doing a 'binary + source copy' between
lucid->maverick->natty->oneiric. For many packages though, that will
run into various dependency / toolchain hell issues - particularly
because source packages tend to be updated to build with the latest
dpkg/debhelper etc, rather than staying geared to build with lucids
dpkg.
-Rob

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Evan Broder-3
On Thu, Jun 2, 2011 at 6:05 PM, Robert Collins
<[hidden email]> wrote:

> On Fri, Jun 3, 2011 at 2:40 AM, Martin Owens <[hidden email]> wrote:
>> Why not just make ppa uploading simpler at the same time? (if you can
>> change the changelog file processing):
>>
>> picocom (1.4-1) debian:unstable oneiric natty maverick lucid;
>> urgency=low
>>
>> Instead of pushing 6 different packages, we could push just the one.
>> Have the tools search for a match and build for that archive.
>
> The challenge with ppas here  is that a package can only be built once
> in a given archive: the binary package names are the same for all of
> unstable/oneiric/natty/maverick/lucid : the APT archive which makes up
> a PPA cannot present different binary versions for each series.

Hmm...a lot of this discussion seems to be getting caught up in the
ubuntu-devel moderation queue, but I'll try to guess context as best
as I can...

If you want to upload a package to a PPA for a bunch of versions
simultaneously, I'd recommend my backportpackage script in
ubuntu-dev-tools. You can do something like

  backportpackage --destination natty --destination maverick
--destination lucid --upload ppa:broder/ppa picocom_1.4-1.dsc

Or instead of a .dsc, you can give it a --source release (e.g.
oneiric) and a package name. It handles bumping (well, dropping) the
package version number appropriately (e.g. ~natty1). It can also build
locally (--build/-b) and save the build products (-w WORKDIR)

It will still push 3 different packages, but the generation and upload
process is totally automated, and it avoids issues with version
numbers, toolchains, and upgrades.

- Evan

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Matt Zimmerman
On Thu, Jun 02, 2011 at 06:17:09PM -0700, Evan Broder wrote:
> Hmm...a lot of this discussion seems to be getting caught up in the
> ubuntu-devel moderation queue, but I'll try to guess context as best
> as I can...

The moderation queue doesn't have any outstanding messages for this thread,
though maybe it did when you looked.

It's also possible that some folks are subscribed to debian-devel but not
ubuntu-devel.

--
 - mdz

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Ian Jackson-2
In reply to this post by Peter Samuelson
Peter Samuelson writes ("Re: Uploading to multiple distros"):
> Since syncs from Debian are actually supposed to be the majority of
> packages in Ubuntu anyway, why not just do that - a real sync, not a
> fake simultaneous one.  [...]

Because that means that the Ubuntu developer needs to wait for Debian
to process the upload (which may involve waiting for NEW or other kind
of approvals).

> I suppose you mean put that functionality into this syncpackage thing.
> Because otherwise you're still going to need to run dpkg-genchanges
> twice, once presumably by hand with some sort of suite override switch,
> and that does rather break the abstractions of debian/rules, debhelper,
> and the like.

dpkg-buildpackage could run dpkg-genchanges as many times as there
were target suites.

Ian.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Ian Jackson-2
In reply to this post by Ian Jackson-2
Matt Zimmerman writes ("Re: Uploading to multiple distros"):
> On Thu, Jun 02, 2011 at 01:54:37PM +0100, Ian Jackson wrote:
> > And if the package is not accepted into the Debian archive for any reason,
> > the changelog is very misleading because it looks like a sync from Debian.
>
> True, but perhaps harmless.

Well, it would normally result in the next sync being a pure sync from
Debian (since it would look like the previous Ubuntu version was in
sync with Debian) and perhaps the changes in the upload that went only
to Ubuntu would be dropped.

> > One way to enable simultaneous uploads would be to arrange for
> > dpkg-genchanges to filter out suites for "other" distros when
> > generating the .changes file.  Then you would have the same files
> > being uploaded but two different .changes files.
>
> Yes, it seems like it would be straightforward enough to generate two
> appropriate .changes files for this case, and it would do the right thing.

Right, that's what I think I'm suggesting.

Ian.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Uploading to multiple distros

Steve Langasek-6
In reply to this post by Ian Jackson-2
On Fri, Jun 03, 2011 at 05:55:18PM +0100, Ian Jackson wrote:
> Peter Samuelson writes ("Re: Uploading to multiple distros"):
> > Since syncs from Debian are actually supposed to be the majority of
> > packages in Ubuntu anyway, why not just do that - a real sync, not a
> > fake simultaneous one.  [...]

> Because that means that the Ubuntu developer needs to wait for Debian
> to process the upload (which may involve waiting for NEW or other kind
> of approvals).

Ubuntu policy on package versioning holds that you shouldn't upload such a
package to Ubuntu before it's been approved into Debian *anyway*, lest you
end up with the package rejected in Debian and two different source packages
with the same version number in Debian vs. Ubuntu...

> > I suppose you mean put that functionality into this syncpackage thing.
> > Because otherwise you're still going to need to run dpkg-genchanges
> > twice, once presumably by hand with some sort of suite override switch,
> > and that does rather break the abstractions of debian/rules, debhelper,
> > and the like.

> dpkg-buildpackage could run dpkg-genchanges as many times as there
> were target suites.

To upload to both Debian and Ubuntu, it would also have to pass different
options since Ubuntu accepts only source uploads and Debian does not accept
source-only uploads.

--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
[hidden email]                                     [hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (845 bytes) Download Attachment