Transition of LXD from deb to snap

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Transition of LXD from deb to snap

Stéphane Graber-2
Hello,

As some of you may know, we've slowly been pushing LXD users to use the
snap rather than the deb, whenever their environment allows it.

Starting with Ubuntu 18.10, we will be replacing the deb package
traditionally shipped in Ubuntu by the LXD snap.


There are a few pieces to that migration so I'll go into details about
each of them:


== Dependency management ==
LXD currently has the following reverse dependencies:

 - adapt (depends)
 - nova-compute-lxd (depends)
 - autopkgtest (suggests)
 - snapcraft (suggests)

We'll make sure that each of those are capable of handling the LXD snap.

That typically means either using the command line tool directly or if
using the API, looking at both /var/lib/lxd/unix.socket and
/var/snap/lxd/common/lxd/unix.socket

We will keep a deb in the archive, which will provide the "lxd" binary
package and act as a shim which installs the snap and proceeds with data
migration if required.


== Data migration ==
The LXD snap ships with a "lxd.migrate" command which handles moving of
all data from the deb path (/var/lib/lxd) over to the snap path
(/var/snap/lxd/common/lxd).

The package in its preinst will attempt to contact the snap store,
holding package upgrade until it's reachable, then ask the user about
what track they'd like to use, followed by installing the LXD snap,
waiting for it to be online and automatically run the migration tool.

Should any error happen, the deb content will remain on disk (as all
this is done in preinst). We can then provide individual instructions
for failed migrations and update lxd.migrate to account for any cases
that it couldn't handled.


== Seed management ==
LXD is currently seeded in the server seed, making it included in all
Ubuntu Server installations, including cloud images.

We expect that this will continue to happen and have landed experimental
socket activation support in our edge snap to allow for this (as always
running LXD in all cloud instances is obviously unacceptable).

The main blocker we have right now is that snapd socket activation is
misbheaving on Fedora, making it impractical for us to enable socket
activation in the stable channel.

If we don't have a resolution on the Fedora side within a couple of
weeks, I expect we'll temporarily unseed LXD from Ubuntu so we can move
forward with the rest of the plan, then seed LXD as a snap as soon as
socket activation works properly for all our users.


== Channels and tracks ==
LXD has a track per upstream LTS release as well as a "latest" track.

The current plan is to have Ubuntu LTS releases default to installing
from the most recent LTS track (currently "3.0") while non-LTS Ubuntu
releases should default to installing from the "latest" track.

The debconf prompt will always be shown, so this will only affect the
default selection.


A branch of the stable channel of those tracks will be created and
closed per policy on seeded snaps (allowing to push emergency snaps to
those users bypassing the upstream).

== LXD in the LTSes ==
Nothing is going to change for LXD in existing Ubuntu releases.

We'll keep maintaining those debs in both -updates and -backports until
such time as the Ubuntu release becomes EOL.

This change applies ONLY to Ubuntu 18.10 and later.


== Timeline ==
I was hoping to have all of this done prior to Feature Freeze, but it's
clear that this will not happen.

The current plan is to have the reverse dependencies and remaining
socket activation problems sorted within the next 2 weeks, at which
point we can upload the migration deb to the archive and transition the
seeds to pointing to the snap.

This should still provide with plenty of time to deal with any issue
that shows up and Ubuntu 18.10 feels like the right time to do this work
so we can be very confident that the 18.04 to 20.04 upgrade will go
smoothly down the line.

== How to help ==
We have a Launchpad bug open to track the reverse dependencies part of
this here:

    https://bugs.launchpad.net/ubuntu/+source/lxd/+bug/1788040

And have a PPA which contains the current version of the upgrade deb here:

    ppa:stgraber/experimental-devirt


If you're running downstream software which interacts with LXD, I'd
strongly recommend you try switching to the snap, either using the
package in that PPA or manually by installing the LXD snap and then
running "lxd.migrate".


Should you have any questions or issues, feel free to respond here or
contact me directly on IRC or by e-mail.

--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Matthias Klose-6
On 20.08.2018 23:13, Stéphane Graber wrote:
> If you're running downstream software which interacts with LXD, I'd
> strongly recommend you try switching to the snap, either using the
> package in that PPA or manually by installing the LXD snap and then
> running "lxd.migrate".

there are (and for are for a while) currently failing lxd autopkg tests
triggered by other packages (see update_excuses).  What's the future of these?
Short term fixing those please, and long term?

Matthias

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Stéphane Graber-2
On Tue, Aug 21, 2018 at 11:46:34AM +0200, Matthias Klose wrote:
> On 20.08.2018 23:13, Stéphane Graber wrote:
> > If you're running downstream software which interacts with LXD, I'd
> > strongly recommend you try switching to the snap, either using the
> > package in that PPA or manually by installing the LXD snap and then
> > running "lxd.migrate".
>
> there are (and for are for a while) currently failing lxd autopkg tests
> triggered by other packages (see update_excuses).  What's the future of these?
> Short term fixing those please, and long term?

LXD 3.0.2 which we'll be uploading this week has a "fix" for those
errors (effectively shellcheck becoming more verbose).

The empty LXD deb will not contain any autopkgtest so once the
transition to the snap is done, autopkgtest will effectively always be a
no-op.

> Matthias

--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Matthias Klose-6
On 21.08.2018 16:56, Stéphane Graber wrote:

> On Tue, Aug 21, 2018 at 11:46:34AM +0200, Matthias Klose wrote:
>> On 20.08.2018 23:13, Stéphane Graber wrote:
>>> If you're running downstream software which interacts with LXD, I'd
>>> strongly recommend you try switching to the snap, either using the
>>> package in that PPA or manually by installing the LXD snap and then
>>> running "lxd.migrate".
>>
>> there are (and for are for a while) currently failing lxd autopkg tests
>> triggered by other packages (see update_excuses).  What's the future of these?
>> Short term fixing those please, and long term?
>
> LXD 3.0.2 which we'll be uploading this week has a "fix" for those
> errors (effectively shellcheck becoming more verbose).
>
> The empty LXD deb will not contain any autopkgtest so once the
> transition to the snap is done, autopkgtest will effectively always be a
> no-op.

... which is interpreted as a failing autopkg test.  You have to add an always
succeeding autopkg test.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Stéphane Graber-2
On Tue, Aug 21, 2018 at 04:58:21PM +0200, Matthias Klose wrote:

> On 21.08.2018 16:56, Stéphane Graber wrote:
> > On Tue, Aug 21, 2018 at 11:46:34AM +0200, Matthias Klose wrote:
> >> On 20.08.2018 23:13, Stéphane Graber wrote:
> >>> If you're running downstream software which interacts with LXD, I'd
> >>> strongly recommend you try switching to the snap, either using the
> >>> package in that PPA or manually by installing the LXD snap and then
> >>> running "lxd.migrate".
> >>
> >> there are (and for are for a while) currently failing lxd autopkg tests
> >> triggered by other packages (see update_excuses).  What's the future of these?
> >> Short term fixing those please, and long term?
> >
> > LXD 3.0.2 which we'll be uploading this week has a "fix" for those
> > errors (effectively shellcheck becoming more verbose).
> >
> > The empty LXD deb will not contain any autopkgtest so once the
> > transition to the snap is done, autopkgtest will effectively always be a
> > no-op.
>
> ... which is interpreted as a failing autopkg test.  You have to add an always
> succeeding autopkg test.
Sure but as I said just a paragraph above this, LXD 3.0.2, which we'll
be uploading this week, has a "fix" for this.

--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Matthias Klose-6
On 21.08.2018 17:01, Stéphane Graber wrote:

> On Tue, Aug 21, 2018 at 04:58:21PM +0200, Matthias Klose wrote:
>> On 21.08.2018 16:56, Stéphane Graber wrote:
>>> On Tue, Aug 21, 2018 at 11:46:34AM +0200, Matthias Klose wrote:
>>>> On 20.08.2018 23:13, Stéphane Graber wrote:
>>>>> If you're running downstream software which interacts with LXD, I'd
>>>>> strongly recommend you try switching to the snap, either using the
>>>>> package in that PPA or manually by installing the LXD snap and then
>>>>> running "lxd.migrate".
>>>>
>>>> there are (and for are for a while) currently failing lxd autopkg tests
>>>> triggered by other packages (see update_excuses).  What's the future of these?
>>>> Short term fixing those please, and long term?
>>>
>>> LXD 3.0.2 which we'll be uploading this week has a "fix" for those
>>> errors (effectively shellcheck becoming more verbose).
>>>
>>> The empty LXD deb will not contain any autopkgtest so once the
>>> transition to the snap is done, autopkgtest will effectively always be a
>>> no-op.
>>
>> ... which is interpreted as a failing autopkg test.  You have to add an always
>> succeeding autopkg test.
>
> Sure but as I said just a paragraph above this, LXD 3.0.2, which we'll
> be uploading this week, has a "fix" for this.

so if we have to wait for every autopkg test fix several weeks, that doesn't
work in the archive.  we have a feature freeze tomorrow, and other packages
depend on that.  From my point of view there is something very wrong if we have
to wait that long.  Multiply that time by other needed autopkg test fixes, and
we are at this point again accumulating packages in the -proposed pocket for no
reason ...

sorry, but that's not pro-active archive work.

Matthias

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Stéphane Graber-2
On Tue, Aug 21, 2018 at 05:07:47PM +0200, Matthias Klose wrote:

> On 21.08.2018 17:01, Stéphane Graber wrote:
> > On Tue, Aug 21, 2018 at 04:58:21PM +0200, Matthias Klose wrote:
> >> On 21.08.2018 16:56, Stéphane Graber wrote:
> >>> On Tue, Aug 21, 2018 at 11:46:34AM +0200, Matthias Klose wrote:
> >>>> On 20.08.2018 23:13, Stéphane Graber wrote:
> >>>>> If you're running downstream software which interacts with LXD, I'd
> >>>>> strongly recommend you try switching to the snap, either using the
> >>>>> package in that PPA or manually by installing the LXD snap and then
> >>>>> running "lxd.migrate".
> >>>>
> >>>> there are (and for are for a while) currently failing lxd autopkg tests
> >>>> triggered by other packages (see update_excuses).  What's the future of these?
> >>>> Short term fixing those please, and long term?
> >>>
> >>> LXD 3.0.2 which we'll be uploading this week has a "fix" for those
> >>> errors (effectively shellcheck becoming more verbose).
> >>>
> >>> The empty LXD deb will not contain any autopkgtest so once the
> >>> transition to the snap is done, autopkgtest will effectively always be a
> >>> no-op.
> >>
> >> ... which is interpreted as a failing autopkg test.  You have to add an always
> >> succeeding autopkg test.
> >
> > Sure but as I said just a paragraph above this, LXD 3.0.2, which we'll
> > be uploading this week, has a "fix" for this.
>
> so if we have to wait for every autopkg test fix several weeks, that doesn't
> work in the archive.  we have a feature freeze tomorrow, and other packages
> depend on that.  From my point of view there is something very wrong if we have
> to wait that long.  Multiply that time by other needed autopkg test fixes, and
> we are at this point again accumulating packages in the -proposed pocket for no
> reason ...
Can you relax a bit maybe? LXD 3.0.2 was tagged last Thursday, we're
finishing the release announcement so that it can go in the changelog
and then we'll upload it.

The LXD testsuite takes quite a while to run, so it'd be a waste of time
to have to figure out what to cherry-pick, testbuild, upload, get things
to the release pocket to then do it all over again a few hours later for
the upstream bugfix release.

> sorry, but that's not pro-active archive work.

> Matthias

--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Matthias Klose-6
On 21.08.2018 17:14, Stéphane Graber wrote:

> On Tue, Aug 21, 2018 at 05:07:47PM +0200, Matthias Klose wrote:
>> On 21.08.2018 17:01, Stéphane Graber wrote:
>>> On Tue, Aug 21, 2018 at 04:58:21PM +0200, Matthias Klose wrote:
>>>> On 21.08.2018 16:56, Stéphane Graber wrote:
>>>>> On Tue, Aug 21, 2018 at 11:46:34AM +0200, Matthias Klose wrote:
>>>>>> On 20.08.2018 23:13, Stéphane Graber wrote:
>>>>>>> If you're running downstream software which interacts with LXD, I'd
>>>>>>> strongly recommend you try switching to the snap, either using the
>>>>>>> package in that PPA or manually by installing the LXD snap and then
>>>>>>> running "lxd.migrate".
>>>>>>
>>>>>> there are (and for are for a while) currently failing lxd autopkg tests
>>>>>> triggered by other packages (see update_excuses).  What's the future of these?
>>>>>> Short term fixing those please, and long term?
>>>>>
>>>>> LXD 3.0.2 which we'll be uploading this week has a "fix" for those
>>>>> errors (effectively shellcheck becoming more verbose).
>>>>>
>>>>> The empty LXD deb will not contain any autopkgtest so once the
>>>>> transition to the snap is done, autopkgtest will effectively always be a
>>>>> no-op.
>>>>
>>>> ... which is interpreted as a failing autopkg test.  You have to add an always
>>>> succeeding autopkg test.
>>>
>>> Sure but as I said just a paragraph above this, LXD 3.0.2, which we'll
>>> be uploading this week, has a "fix" for this.
>>
>> so if we have to wait for every autopkg test fix several weeks, that doesn't
>> work in the archive.  we have a feature freeze tomorrow, and other packages
>> depend on that.  From my point of view there is something very wrong if we have
>> to wait that long.  Multiply that time by other needed autopkg test fixes, and
>> we are at this point again accumulating packages in the -proposed pocket for no
>> reason ...
>
> Can you relax a bit maybe?

No.  You are trading in your test resources against resources spent by various
people doing the archive work.  From my point of view, this is a no-go.  I am
just wondering about such a laissez-faire style, at least you are still an
Ubuntu Release Manager, knowing how much autopkg test failures can get in the
way of archive work.

> LXD 3.0.2 was tagged last Thursday, we're
> finishing the release announcement so that it can go in the changelog
> and then we'll upload it.

please notice it's not just this upload, but the lxd testsuite seems to be a bit
picky to fail on the buildds.  If people need to work-around autopkg tests, it's
their time again.

> The LXD testsuite takes quite a while to run, so it'd be a waste of time
> to have to figure out what to cherry-pick, testbuild, upload, get things
> to the release pocket to then do it all over again a few hours later for
> the upstream bugfix release.
>
>> sorry, but that's not pro-active archive work.
>
>> Matthias
>


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Steve Langasek-6
In reply to this post by Stéphane Graber-2
Hi Stéphane,

On Mon, Aug 20, 2018 at 05:13:11PM -0400, Stéphane Graber wrote:

<snip>

> The package in its preinst will attempt to contact the snap store,
> holding package upgrade until it's reachable, then ask the user about
> what track they'd like to use, followed by installing the LXD snap,
> waiting for it to be online and automatically run the migration tool.
>
> Should any error happen, the deb content will remain on disk (as all
> this is done in preinst). We can then provide individual instructions
> for failed migrations and update lxd.migrate to account for any cases
> that it couldn't handled.

<snip>

> The current plan is to have Ubuntu LTS releases default to installing
> from the most recent LTS track (currently "3.0") while non-LTS Ubuntu
> releases should default to installing from the "latest" track.

> The debconf prompt will always be shown, so this will only affect the
> default selection.

It is impossible to enforce that debconf prompts are shown in all cases.
Unattended-upgrades are a thing.  Manually setting
DEBIAN_FRONTEND=noninteractive is a thing.  DEBIAN_PRIORITY=critical is a
thing (and while it's probably reasonable for the "cannot talk to the store"
error prompt to be shown at critical priority, I think track selection
fits the definition of a high-priority debconf prompt, not critical).

This is a general rule anyway, but please be sure that the package behavior
is sane when the debconf prompts are not shown.

- What should the behavior be if the store cannot be reached and the prompt
  cannot be shown?  Should the preinst loop indefinitely (causing
  unattended-upgrades to hold the apt lock forever until the admin
  intervenes), or should the preinst abort (causing an apt transaction
  failure)?
- If it should abort, you may find /usr/sbin/update-secureboot-policy in
  shim-signed to be useful prior art. ("$seen_key")
- If it should loop forever, please ensure the maintainer script outputs
  sufficient information to stdout/stderr for the apparent hang to be
  debuggable from just the apt log.  (But maybe don't generate output for
  every loop iteration, since making /var/log ENOSPC is not the ideal way
  for the admin to discover the problem either.)

> A branch of the stable channel of those tracks will be created and
> closed per policy on seeded snaps (allowing to push emergency snaps to
> those users bypassing the upstream).

Excellent!

--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                   https://www.debian.org/
[hidden email]                                     [hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Stéphane Graber-2
On Tue, Aug 21, 2018 at 11:41:18AM -0700, Steve Langasek wrote:
> Hi Stéphane,

Hi Steve,

> On Mon, Aug 20, 2018 at 05:13:11PM -0400, Stéphane Graber wrote:
>
> <snip>
>
> > The package in its preinst will attempt to contact the snap store,
> > holding package upgrade until it's reachable, then ask the user about
> > what track they'd like to use, followed by installing the LXD snap,
> > waiting for it to be online and automatically run the migration tool.
> >
> > Should any error happen, the deb content will remain on disk (as all
> > this is done in preinst). We can then provide individual instructions
> > for failed migrations and update lxd.migrate to account for any cases
> > that it couldn't handled.
>
> <snip>
>
> > The current plan is to have Ubuntu LTS releases default to installing
> > from the most recent LTS track (currently "3.0") while non-LTS Ubuntu
> > releases should default to installing from the "latest" track.
>
> > The debconf prompt will always be shown, so this will only affect the
> > default selection.
>
> It is impossible to enforce that debconf prompts are shown in all cases.
> Unattended-upgrades are a thing.  Manually setting
> DEBIAN_FRONTEND=noninteractive is a thing.  DEBIAN_PRIORITY=critical is a
> thing (and while it's probably reasonable for the "cannot talk to the store"
> error prompt to be shown at critical priority, I think track selection
> fits the definition of a high-priority debconf prompt, not critical).
Fair enough, moved all prompts to "high" except for the unreachable
store one which remains "critical".

> This is a general rule anyway, but please be sure that the package behavior
> is sane when the debconf prompts are not shown.

The preinst both attempts to show debconf prompts AND prints relevant
messages to the terminal.

> - What should the behavior be if the store cannot be reached and the prompt
>   cannot be shown?  Should the preinst loop indefinitely (causing
>   unattended-upgrades to hold the apt lock forever until the admin
>   intervenes), or should the preinst abort (causing an apt transaction
>   failure)?

My current logic is to trigger the debconf prompt and if unusccesful to
print "Unable to contact the store, trying again in 1 minute" to the
terminal and wait a minute before trying again.

> - If it should abort, you may find /usr/sbin/update-secureboot-policy in
>   shim-signed to be useful prior art. ("$seen_key")
> - If it should loop forever, please ensure the maintainer script outputs
>   sufficient information to stdout/stderr for the apparent hang to be
>   debuggable from just the apt log.  (But maybe don't generate output for
>   every loop iteration, since making /var/log ENOSPC is not the ideal way
>   for the admin to discover the problem either.)

Good point, I'll change the logic to be a bit less aggressive logging wise.

I think I'll actually go with a hybrid of the two options:
 - If possible, show the debconf critical prompt
 - If that doesn't work, print to the terminal that we'll retry every
   minute for the next 30 minutes.
 - Log something again after 10 and 20 minutes
 - Abort at 30

> > A branch of the stable channel of those tracks will be created and
> > closed per policy on seeded snaps (allowing to push emergency snaps to
> > those users bypassing the upstream).
>
> Excellent!

I actually had a question about that part, the wiki says to create an
ubuntu-18.10 branch and use that during snap installation.

But then what's responsible for switching this to a ubuntu-19.04 branch
during the next upgrade?


Since the same version of the snap will be pushed to all Ubuntu series,
wouldn't it make more sense to have it just be "ubuntu", saving us the
trouble of having to figure out what to do on Ubuntu release upgrades
and reflecting the fact that the snap is the same for all series.

Stéphane

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Steve Langasek-6
On Tue, Aug 21, 2018 at 03:03:48PM -0400, Stéphane Graber wrote:
> > > A branch of the stable channel of those tracks will be created and
> > > closed per policy on seeded snaps (allowing to push emergency snaps to
> > > those users bypassing the upstream).

> > Excellent!

> I actually had a question about that part, the wiki says to create an
> ubuntu-18.10 branch and use that during snap installation.

> But then what's responsible for switching this to a ubuntu-19.04 branch
> during the next upgrade?

Support for this has landed in ubuntu-release-upgrader 1:18.10.8 in cosmic;
LP: #1748581.  Note that this is based on a whitelist of known seeded snaps
that are encoded in u-r-u as part of the quirks handling (which is not ideal
since it duplicates the package seeds), so this will need to be updated to
include the lxd snap here.

> Since the same version of the snap will be pushed to all Ubuntu series,
> wouldn't it make more sense to have it just be "ubuntu", saving us the
> trouble of having to figure out what to do on Ubuntu release upgrades
> and reflecting the fact that the snap is the same for all series.

This escape hatch exists precisely for the case that the upstream stable
snap does not integrate correctly in a release-agnostic fashion and
per-Ubuntu-release quirking is needed.  Better to have it and never use it
than to need it and not have it.

--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                   https://www.debian.org/
[hidden email]                                     [hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Stéphane Graber-2
On Tue, Aug 21, 2018 at 12:25:33PM -0700, Steve Langasek wrote:

> On Tue, Aug 21, 2018 at 03:03:48PM -0400, Stéphane Graber wrote:
> > > > A branch of the stable channel of those tracks will be created and
> > > > closed per policy on seeded snaps (allowing to push emergency snaps to
> > > > those users bypassing the upstream).
>
> > > Excellent!
>
> > I actually had a question about that part, the wiki says to create an
> > ubuntu-18.10 branch and use that during snap installation.
>
> > But then what's responsible for switching this to a ubuntu-19.04 branch
> > during the next upgrade?
>
> Support for this has landed in ubuntu-release-upgrader 1:18.10.8 in cosmic;
> LP: #1748581.  Note that this is based on a whitelist of known seeded snaps
> that are encoded in u-r-u as part of the quirks handling (which is not ideal
> since it duplicates the package seeds), so this will need to be updated to
> include the lxd snap here.
Hmm, interesting, though it looks like the logic in the upgrader here is
a bit lacking and may lead to data corruption or at least broken snaps.

It appears to just run "snap refresh <name>
--channel=stable/ubuntu-18.10" which means a potential track switch and
channel switch for users that have seen decided to switch to another
channel or track.


Commented in the bug, I suspect this bug needs to be re-open and the
logic revisited.

https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1748581

> > Since the same version of the snap will be pushed to all Ubuntu series,
> > wouldn't it make more sense to have it just be "ubuntu", saving us the
> > trouble of having to figure out what to do on Ubuntu release upgrades
> > and reflecting the fact that the snap is the same for all series.
>
> This escape hatch exists precisely for the case that the upstream stable
> snap does not integrate correctly in a release-agnostic fashion and
> per-Ubuntu-release quirking is needed.  Better to have it and never use it
> than to need it and not have it.

Yeah, if we fix the upgrader to handle the above properly
(and as suggested in the LP bug), then that should be fine.

--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Stéphane Graber-2
On Tue, Aug 21, 2018 at 03:35:11PM -0400, Stéphane Graber wrote:

> On Tue, Aug 21, 2018 at 12:25:33PM -0700, Steve Langasek wrote:
> > On Tue, Aug 21, 2018 at 03:03:48PM -0400, Stéphane Graber wrote:
> > > > > A branch of the stable channel of those tracks will be created and
> > > > > closed per policy on seeded snaps (allowing to push emergency snaps to
> > > > > those users bypassing the upstream).
> >
> > > > Excellent!
> >
> > > I actually had a question about that part, the wiki says to create an
> > > ubuntu-18.10 branch and use that during snap installation.
> >
> > > But then what's responsible for switching this to a ubuntu-19.04 branch
> > > during the next upgrade?
> >
> > Support for this has landed in ubuntu-release-upgrader 1:18.10.8 in cosmic;
> > LP: #1748581.  Note that this is based on a whitelist of known seeded snaps
> > that are encoded in u-r-u as part of the quirks handling (which is not ideal
> > since it duplicates the package seeds), so this will need to be updated to
> > include the lxd snap here.
>
> Hmm, interesting, though it looks like the logic in the upgrader here is
> a bit lacking and may lead to data corruption or at least broken snaps.
>
> It appears to just run "snap refresh <name>
> --channel=stable/ubuntu-18.10" which means a potential track switch and
> channel switch for users that have seen decided to switch to another
> channel or track.
>
>
> Commented in the bug, I suspect this bug needs to be re-open and the
> logic revisited.
>
> https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1748581
>
> > > Since the same version of the snap will be pushed to all Ubuntu series,
> > > wouldn't it make more sense to have it just be "ubuntu", saving us the
> > > trouble of having to figure out what to do on Ubuntu release upgrades
> > > and reflecting the fact that the snap is the same for all series.
> >
> > This escape hatch exists precisely for the case that the upstream stable
> > snap does not integrate correctly in a release-agnostic fashion and
> > per-Ubuntu-release quirking is needed.  Better to have it and never use it
> > than to need it and not have it.
>
> Yeah, if we fix the upgrader to handle the above properly
> (and as suggested in the LP bug), then that should be fine.

I've now updated the PPA with version ~ppa5.

This includes:
 - Using the ubuntu-XX.XX branches (created and closed)
 - Debconf prompts are now high except for the unreachable store one
   which is critical
 - Added logic to select the "3.0" track for LTS releaes
 - Added noninteractive logic to print a message a 0, 10 and 20 minutes,
   trying connection every minute and giving up after 30.
 - Stop & disable the old systemd services to avoid any conflicts

I've validated all combinations of those variables:
 - series: 18.04 and 18.10
 - debconf: interactive (curses), interactive (text) and noninteractive
 - connectivity: available, unavailable, available after a few minutes


--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Stéphane Graber-2
On Tue, Aug 21, 2018 at 04:39:46PM -0400, Stéphane Graber wrote:

> On Tue, Aug 21, 2018 at 03:35:11PM -0400, Stéphane Graber wrote:
> > On Tue, Aug 21, 2018 at 12:25:33PM -0700, Steve Langasek wrote:
> > > On Tue, Aug 21, 2018 at 03:03:48PM -0400, Stéphane Graber wrote:
> > > > > > A branch of the stable channel of those tracks will be created and
> > > > > > closed per policy on seeded snaps (allowing to push emergency snaps to
> > > > > > those users bypassing the upstream).
> > >
> > > > > Excellent!
> > >
> > > > I actually had a question about that part, the wiki says to create an
> > > > ubuntu-18.10 branch and use that during snap installation.
> > >
> > > > But then what's responsible for switching this to a ubuntu-19.04 branch
> > > > during the next upgrade?
> > >
> > > Support for this has landed in ubuntu-release-upgrader 1:18.10.8 in cosmic;
> > > LP: #1748581.  Note that this is based on a whitelist of known seeded snaps
> > > that are encoded in u-r-u as part of the quirks handling (which is not ideal
> > > since it duplicates the package seeds), so this will need to be updated to
> > > include the lxd snap here.
> >
> > Hmm, interesting, though it looks like the logic in the upgrader here is
> > a bit lacking and may lead to data corruption or at least broken snaps.
> >
> > It appears to just run "snap refresh <name>
> > --channel=stable/ubuntu-18.10" which means a potential track switch and
> > channel switch for users that have seen decided to switch to another
> > channel or track.
> >
> >
> > Commented in the bug, I suspect this bug needs to be re-open and the
> > logic revisited.
> >
> > https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1748581
> >
> > > > Since the same version of the snap will be pushed to all Ubuntu series,
> > > > wouldn't it make more sense to have it just be "ubuntu", saving us the
> > > > trouble of having to figure out what to do on Ubuntu release upgrades
> > > > and reflecting the fact that the snap is the same for all series.
> > >
> > > This escape hatch exists precisely for the case that the upstream stable
> > > snap does not integrate correctly in a release-agnostic fashion and
> > > per-Ubuntu-release quirking is needed.  Better to have it and never use it
> > > than to need it and not have it.
> >
> > Yeah, if we fix the upgrader to handle the above properly
> > (and as suggested in the LP bug), then that should be fine.
>
>
> I've now updated the PPA with version ~ppa5.
>
> This includes:
>  - Using the ubuntu-XX.XX branches (created and closed)
>  - Debconf prompts are now high except for the unreachable store one
>    which is critical
>  - Added logic to select the "3.0" track for LTS releaes
>  - Added noninteractive logic to print a message a 0, 10 and 20 minutes,
>    trying connection every minute and giving up after 30.
>  - Stop & disable the old systemd services to avoid any conflicts
>
> I've validated all combinations of those variables:
>  - series: 18.04 and 18.10
>  - debconf: interactive (curses), interactive (text) and noninteractive
>  - connectivity: available, unavailable, available after a few minutes
All rdepends of lxd and lxd-client have been tested to behave properly
with the LXD snap and our deb shim.

I will now be uploading the transitional LXD deb to the archive, holding
it in -proposed until Monday so anyone interested can test it that way
and we can sort out any autopkgtest issue that may arise.

On Monday, I'll remove the blocker tag which should have it migrate to
the release pocket, I'll land a seed change at the same time, removing
lxd and lxd-client from the supported server seed and replacing lxd with
snap:lxd in the server ship seed.


This will then cause our images to start shipping the LXD snap rather
than the deb, we can then sort out any issue that show up as a result
of that change (should there be any).


Note that at this time, the LXD snap isn't using socket activation yet.
We have code in place for that in the edge channel but want to do more
tests on it before we roll it out to all our users. This means that
starting on Monday, LXD will be starting up unconditionally on 18.10
images. This is a temporary situation and will be corrected by release.

--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transition of LXD from deb to snap

Stéphane Graber-2
On Wed, Sep 12, 2018 at 02:42:00PM -0400, Stéphane Graber wrote:

> On Tue, Aug 21, 2018 at 04:39:46PM -0400, Stéphane Graber wrote:
> > On Tue, Aug 21, 2018 at 03:35:11PM -0400, Stéphane Graber wrote:
> > > On Tue, Aug 21, 2018 at 12:25:33PM -0700, Steve Langasek wrote:
> > > > On Tue, Aug 21, 2018 at 03:03:48PM -0400, Stéphane Graber wrote:
> > > > > > > A branch of the stable channel of those tracks will be created and
> > > > > > > closed per policy on seeded snaps (allowing to push emergency snaps to
> > > > > > > those users bypassing the upstream).
> > > >
> > > > > > Excellent!
> > > >
> > > > > I actually had a question about that part, the wiki says to create an
> > > > > ubuntu-18.10 branch and use that during snap installation.
> > > >
> > > > > But then what's responsible for switching this to a ubuntu-19.04 branch
> > > > > during the next upgrade?
> > > >
> > > > Support for this has landed in ubuntu-release-upgrader 1:18.10.8 in cosmic;
> > > > LP: #1748581.  Note that this is based on a whitelist of known seeded snaps
> > > > that are encoded in u-r-u as part of the quirks handling (which is not ideal
> > > > since it duplicates the package seeds), so this will need to be updated to
> > > > include the lxd snap here.
> > >
> > > Hmm, interesting, though it looks like the logic in the upgrader here is
> > > a bit lacking and may lead to data corruption or at least broken snaps.
> > >
> > > It appears to just run "snap refresh <name>
> > > --channel=stable/ubuntu-18.10" which means a potential track switch and
> > > channel switch for users that have seen decided to switch to another
> > > channel or track.
> > >
> > >
> > > Commented in the bug, I suspect this bug needs to be re-open and the
> > > logic revisited.
> > >
> > > https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1748581
> > >
> > > > > Since the same version of the snap will be pushed to all Ubuntu series,
> > > > > wouldn't it make more sense to have it just be "ubuntu", saving us the
> > > > > trouble of having to figure out what to do on Ubuntu release upgrades
> > > > > and reflecting the fact that the snap is the same for all series.
> > > >
> > > > This escape hatch exists precisely for the case that the upstream stable
> > > > snap does not integrate correctly in a release-agnostic fashion and
> > > > per-Ubuntu-release quirking is needed.  Better to have it and never use it
> > > > than to need it and not have it.
> > >
> > > Yeah, if we fix the upgrader to handle the above properly
> > > (and as suggested in the LP bug), then that should be fine.
> >
> >
> > I've now updated the PPA with version ~ppa5.
> >
> > This includes:
> >  - Using the ubuntu-XX.XX branches (created and closed)
> >  - Debconf prompts are now high except for the unreachable store one
> >    which is critical
> >  - Added logic to select the "3.0" track for LTS releaes
> >  - Added noninteractive logic to print a message a 0, 10 and 20 minutes,
> >    trying connection every minute and giving up after 30.
> >  - Stop & disable the old systemd services to avoid any conflicts
> >
> > I've validated all combinations of those variables:
> >  - series: 18.04 and 18.10
> >  - debconf: interactive (curses), interactive (text) and noninteractive
> >  - connectivity: available, unavailable, available after a few minutes
>
> All rdepends of lxd and lxd-client have been tested to behave properly
> with the LXD snap and our deb shim.
>
> I will now be uploading the transitional LXD deb to the archive, holding
> it in -proposed until Monday so anyone interested can test it that way
> and we can sort out any autopkgtest issue that may arise.
>
> On Monday, I'll remove the blocker tag which should have it migrate to
> the release pocket, I'll land a seed change at the same time, removing
> lxd and lxd-client from the supported server seed and replacing lxd with
> snap:lxd in the server ship seed.
>
>
> This will then cause our images to start shipping the LXD snap rather
> than the deb, we can then sort out any issue that show up as a result
> of that change (should there be any).
>
>
> Note that at this time, the LXD snap isn't using socket activation yet.
> We have code in place for that in the edge channel but want to do more
> tests on it before we roll it out to all our users. This means that
> starting on Monday, LXD will be starting up unconditionally on 18.10
> images. This is a temporary situation and will be corrected by release.
The package is now in the release pocket and the seeds have been updated.

--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (849 bytes) Download Attachment