/usr-merge by default for new installations, with backwards compat

Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

/usr-merge by default for new installations, with backwards compat

Dimitri John Ledkov
Over the past year, there have been multiple times when I had to fix
scripts / exec calls to account for split-usr and usr-merged systems
by patching constants or adding configuration checks to detect if
certain binaries need to be called from /usr or /.

At the same time, I find certain properties of usr-merged systems
attractive. For example, the fact that all the packaged managed stuff
is scoped in /usr is very convenient, for deduplication, sharing,
snapshotting, including/excluding said dir in configs for diffs &
backups. Ability to call any binary using /bin and /sbin prefixes is
very cute too. Our initrd implementation has been fixed for a while to
mount both / and /usr before executing init.

Going forward, I believe, it is useful to enable usr-merge by default
on new installs, without forcing migration of existing installations
and continue to support split-usr systems. It will enable our users to
use their systems in more innovative ways, and reduce the amount of
any accidental /-vs-/usr mistakes and incompatibilities in the
archive, and among third-party software.

In the Ubuntu Archive in cosmic all / vs /usr conflicts are resolved,
initrd by default mounts /usr (if it is separate), and
debootstrap/dpkg have support to create usr-merged systems and ensure
conflicting binaries are prevented from installation. Therefore next
week, I would like to start the upload of debootstrap to switch cosmic
to have merged-usr by default for new installations only. This will
then include desktop / server / cloud / container installations and
images as well.

Existing installations will not be forced to merge /usr, but can opt
into doing so by installing usrmerge package.

/usr-merged installations with /usr as a seperate mountpoints will
continue to be supported, as long as they boot with initrd which does
mount both / and /usr - this is already the case with default Ubuntu
for a few releases now.

Overall the scope of changes is very small, simply /bin becomes a
symlink to /usr/bin (and a few other paths, in a similar fashion) with
everything still available and working via any path prefix.

--
Regards,

Dimitri.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: /usr-merge by default for new installations, with backwards compat

Robie Basak-4
Hi Dimitri,

On Mon, Jul 23, 2018 at 01:41:03PM +0100, Dimitri John Ledkov wrote:
> Going forward, I believe, it is useful to enable usr-merge by default
> on new installs, without forcing migration of existing installations
> and continue to support split-usr systems. It will enable our users to
> use their systems in more innovative ways, and reduce the amount of
> any accidental /-vs-/usr mistakes and incompatibilities in the
> archive, and among third-party software.

Could you please fill us in on Debian's position and status on this? How
much delta do we or would we have to maintain, and how much pain is
there in this maintenance?

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /usr-merge by default for new installations, with backwards compat

Dimitri John Ledkov
On 23 July 2018 at 13:47, Robie Basak <[hidden email]> wrote:

> Hi Dimitri,
>
> On Mon, Jul 23, 2018 at 01:41:03PM +0100, Dimitri John Ledkov wrote:
>> Going forward, I believe, it is useful to enable usr-merge by default
>> on new installs, without forcing migration of existing installations
>> and continue to support split-usr systems. It will enable our users to
>> use their systems in more innovative ways, and reduce the amount of
>> any accidental /-vs-/usr mistakes and incompatibilities in the
>> archive, and among third-party software.
>
> Could you please fill us in on Debian's position and status on this? How
> much delta do we or would we have to maintain, and how much pain is
> there in this maintenance?

The changes are in per-distro scripts in debootstrap. And the Ubuntu
ones are synced into Debian on regular basis (once we announce our
codenames and/or sync back our changes), thus in practice deltas are
short-lived and folded back into Debian uploads of debootstrap. Some
of the conflicts in Cosmic have been resolved differently to Debian,
thus there are only a small number of packages with different breaks
versions / paths to debian, e.g. usrmerge package being the largest
delta - which is still very small.

Currently, Debian has --merged-usr enabled by default in unstable and
testing. See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839046
uploaded in 1.0.102, full current changelog at
https://tracker.debian.org/media/packages/d/debootstrap/changelog-1.0.106

--
Regards,

Dimitri.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: /usr-merge by default for new installations, with backwards compat

Seth Arnold
In reply to this post by Dimitri John Ledkov
On Mon, Jul 23, 2018 at 01:41:03PM +0100, Dimitri John Ledkov wrote:
> In the Ubuntu Archive in cosmic all / vs /usr conflicts are resolved,

AppArmor profiles need to be adjusted:
https://gitlab.com/apparmor/apparmor/issues/8

Thanks

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /usr-merge by default for new installations, with backwards compat

Dimitri John Ledkov
On 24 July 2018 at 01:06, Seth Arnold <[hidden email]> wrote:
> On Mon, Jul 23, 2018 at 01:41:03PM +0100, Dimitri John Ledkov wrote:
>> In the Ubuntu Archive in cosmic all / vs /usr conflicts are resolved,
>
> AppArmor profiles need to be adjusted:
> https://gitlab.com/apparmor/apparmor/issues/8
>

Yes, I was planning to verify these. Note that we are only planning to
perform / & /usr merge, all binaries will remain the same w.r.t. bin
-vs- sbin locations. Thus in practice only /bin and /sbin things need
to be verified in the profiles to correctly take into account optional
/usr prefix. This should be a smaller subset of binaries I hope.

--
Regards,

Dimitri.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: /usr-merge by default for new installations, with backwards compat

Robie Basak-4
In reply to this post by Dimitri John Ledkov
Hi Dimitri,

I see that you blocked the SRU for debootstrap in LP: #1773496 / LP:
#1800945 on a usr-merge problem. Please can you resolve this, given that
you chose to drive this change in Ubuntu? I believe this is what is
causing the sbuild autopkgtests in stable releases to fail - because
distro-info-data now mismatches what debootstrap knows about.

Alternatively, can we just land the debootstrap SRUs anyway? Given that
users release upgrading won't be converted to usr-merge anyway and we
have to support that, I don't see it being much of a problem if users
running debootstrap for Disco while running Bionic also get
non-usr-merge. Perhaps it would even be a good thing to get better
dogfooding coverage on both supported combinations.

Thanks,

Robie

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (836 bytes) Download Attachment