Proposing a New App Developer Upload Process

classic Classic list List threaded Threaded
117 messages Options
1234 ... 6
Reply | Threaded
Open this post in threaded view
|

Proposing a New App Developer Upload Process

David Planella-4
Hi everyone,

As many of you will know, in the last few cycles we've been laying out
the foundations to make Ubuntu a target of choice for app developers. I
am not referring to building the platform here (an area in which we also
keep seeking growth), but rather to enabling app authors to create and
distribute original software, to grow a rich ecosystem of independent
apps for our users.

The resounding success of our latest initiative, the Ubuntu App
Showdown, has not only shown an explosion of high-quality applications
(created in just 3 weeks!), but more importantly, the growing interest
in developing applications for Ubuntu.

As we continue building upon these foundations, we also keep on refining
our processes to identify and improve areas in which we can provide a
better experience for app developers. While doing this, we've been
gathering metrics from different sources –the current queue of apps
pending review and the Showdown results being the main ones. They all
provide clear evidence: the current approach to submit third-party apps
in the Extras repository has already reached the limit in terms of human
review capability and application throughput.

Despite our best intentions and the Ubuntu App Review Board's epic
efforts, we're currently putting a strain on reviewers (who cannot keep
up with the incoming stream of apps) and providing an unsatisfactory
experience for app authors (who have to endure long delays to be able to
upload their apps).

During the course of the last few weeks, after having identified the key
issues and assessed the different options available, Jono Bacon, Michael
Hall and myself have been working to put forward a proposal for a new,
improved app developer upload process. We've been not alone in this
project: we've had the input and help from many others, including the
App Review Board and members of many other Ubuntu and Canonical teams
(Security, Foundations, Desktop, Consumer Apps, just to name a few).

We are now at a point where we consider the spec to be ready for public
review and comment, and we would like to ask for your own contribution
to this process. Ultimately, the goal is to discuss and scope the new
app developer upload process at UDS, but we would like to have a solid
specification to be used in advance as the basis for any UDS sessions
and blueprints. If you'd like to contribute, you can:

* Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
* Provide any feedback either on the Feedback section (preferred) or
  continue the discussion on this thread.

As the people who create Ubuntu, your input is especially valuable, and
we'd really appreciate it if you could spend some of your time to help
app development in Ubuntu succeed.

Thanks!

Regards,
David.


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (563 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Philipp Kern
David,

am Mon, Sep 03, 2012 at 07:59:15PM +0200 hast du folgendes geschrieben:
> * Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess

what happens in case of a namespace conflict with Debian? Given that apps are
not only in dpkg's namespace of all installed applications, as they also refer
to paths within /usr. Can a package in Ubuntu be updated through this process?
Will a package that appears in Ubuntu proper supersede the application
submitted through this process? Will the version for use by apt and dpkg be
prefixed, so that it does not collide? (I see todo items referencing a version
numbering scheme, but I don't see it spelled out at first glance.)

Kind regards
Philipp Kern

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (501 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Jason Taylor-2
In reply to this post by David Planella-4
Additional Permissions
Maybe add
- Sound - Microphone ?

Maybe review
http://developer.android.com/reference/android/Manifest.permission.html

Cheers

On 4 September 2012 05:59, David Planella <[hidden email]> wrote:
Hi everyone,

As many of you will know, in the last few cycles we've been laying out
the foundations to make Ubuntu a target of choice for app developers. I
am not referring to building the platform here (an area in which we also
keep seeking growth), but rather to enabling app authors to create and
distribute original software, to grow a rich ecosystem of independent
apps for our users.

The resounding success of our latest initiative, the Ubuntu App
Showdown, has not only shown an explosion of high-quality applications
(created in just 3 weeks!), but more importantly, the growing interest
in developing applications for Ubuntu.

As we continue building upon these foundations, we also keep on refining
our processes to identify and improve areas in which we can provide a
better experience for app developers. While doing this, we've been
gathering metrics from different sources –the current queue of apps
pending review and the Showdown results being the main ones. They all
provide clear evidence: the current approach to submit third-party apps
in the Extras repository has already reached the limit in terms of human
review capability and application throughput.

Despite our best intentions and the Ubuntu App Review Board's epic
efforts, we're currently putting a strain on reviewers (who cannot keep
up with the incoming stream of apps) and providing an unsatisfactory
experience for app authors (who have to endure long delays to be able to
upload their apps).

During the course of the last few weeks, after having identified the key
issues and assessed the different options available, Jono Bacon, Michael
Hall and myself have been working to put forward a proposal for a new,
improved app developer upload process. We've been not alone in this
project: we've had the input and help from many others, including the
App Review Board and members of many other Ubuntu and Canonical teams
(Security, Foundations, Desktop, Consumer Apps, just to name a few).

We are now at a point where we consider the spec to be ready for public
review and comment, and we would like to ask for your own contribution
to this process. Ultimately, the goal is to discuss and scope the new
app developer upload process at UDS, but we would like to have a solid
specification to be used in advance as the basis for any UDS sessions
and blueprints. If you'd like to contribute, you can:

* Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
* Provide any feedback either on the Feedback section (preferred) or
  continue the discussion on this thread.

As the people who create Ubuntu, your input is especially valuable, and
we'd really appreciate it if you could spend some of your time to help
app development in Ubuntu succeed.

Thanks!

Regards,
David.


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel




--
"Weekends don't count unless you spend them doing something completely pointless. " - Calven


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

bvidinli
In reply to this post by David Planella-4
I am a software developer.
I totally like to see a new Application development, deployment,
approval system.

previously I had prepared package for Ubuntu, for my software.
However, software was being denied by ubuntu people with the cause
"program touches system files". This reason was really interesting. My
software Does touches system files, its job is to do so...
my program is not in Ubuntu repos since 4-5 years, but it is being
used since 6 years in ubuntu community, server admins, through classic
install.sh installation, not apt-get command, since its package was
not approved by package maintainers + ubuntu package preparation steps
was too complicated

I mean, if we/you want to spread the word, want to make Ubuntu a
widely used software/desktop platform, we need to make Application
development/deployment/packagement processes as easy as possible.

As you remember, Win programs was widely spread when Delphi was out in
about 1996. In a similar way, we should make easy:
* app development & testing (ide tools should be improved; Delphi was
a good example, click and code.)
* package management, packaging
* deployment, putting in repos

many things can be said in this manner...

see you,

On Mon, Sep 3, 2012 at 8:59 PM, David Planella
<[hidden email]> wrote:

> Hi everyone,
>
> As many of you will know, in the last few cycles we've been laying out
> the foundations to make Ubuntu a target of choice for app developers. I
> am not referring to building the platform here (an area in which we also
> keep seeking growth), but rather to enabling app authors to create and
> distribute original software, to grow a rich ecosystem of independent
> apps for our users.
>
> The resounding success of our latest initiative, the Ubuntu App
> Showdown, has not only shown an explosion of high-quality applications
> (created in just 3 weeks!), but more importantly, the growing interest
> in developing applications for Ubuntu.
>
> As we continue building upon these foundations, we also keep on refining
> our processes to identify and improve areas in which we can provide a
> better experience for app developers. While doing this, we've been
> gathering metrics from different sources –the current queue of apps
> pending review and the Showdown results being the main ones. They all
> provide clear evidence: the current approach to submit third-party apps
> in the Extras repository has already reached the limit in terms of human
> review capability and application throughput.
>
> Despite our best intentions and the Ubuntu App Review Board's epic
> efforts, we're currently putting a strain on reviewers (who cannot keep
> up with the incoming stream of apps) and providing an unsatisfactory
> experience for app authors (who have to endure long delays to be able to
> upload their apps).
>
> During the course of the last few weeks, after having identified the key
> issues and assessed the different options available, Jono Bacon, Michael
> Hall and myself have been working to put forward a proposal for a new,
> improved app developer upload process. We've been not alone in this
> project: we've had the input and help from many others, including the
> App Review Board and members of many other Ubuntu and Canonical teams
> (Security, Foundations, Desktop, Consumer Apps, just to name a few).
>
> We are now at a point where we consider the spec to be ready for public
> review and comment, and we would like to ask for your own contribution
> to this process. Ultimately, the goal is to discuss and scope the new
> app developer upload process at UDS, but we would like to have a solid
> specification to be used in advance as the basis for any UDS sessions
> and blueprints. If you'd like to contribute, you can:
>
> * Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
> * Provide any feedback either on the Feedback section (preferred) or
>   continue the discussion on this thread.
>
> As the people who create Ubuntu, your input is especially valuable, and
> we'd really appreciate it if you could spend some of your time to help
> app development in Ubuntu succeed.
>
> Thanks!
>
> Regards,
> David.
>
>
> --
> ubuntu-devel mailing list
> [hidden email]
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
>

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Michael Hall
In reply to this post by Philipp Kern
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 09/03/2012 04:03 PM, Philipp Kern wrote:

> David,
>
> am Mon, Sep 03, 2012 at 07:59:15PM +0200 hast du folgendes
> geschrieben:
>> * Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
>
> what happens in case of a namespace conflict with Debian? Given
> that apps are not only in dpkg's namespace of all installed
> applications, as they also refer to paths within /usr. Can a
> package in Ubuntu be updated through this process? Will a package
> that appears in Ubuntu proper supersede the application submitted
> through this process? Will the version for use by apt and dpkg be
> prefixed, so that it does not collide? (I see todo items
> referencing a version numbering scheme, but I don't see it spelled
> out at first glance.)
>
> Kind regards Philipp Kern
>

Apps already in the archives won't be eligible for this process.  We
don't have any requirement on version prefixing, the work items are
there to make sure quickly and the lintian profile are using the
normal version numbering scheme.


Michael Hall
[hidden email]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBCAAGBQJQRUBSAAoJEInUYcGJgfVyTnEP/AsTbp5pNMHb2Hl4K3MVzuL6
ZuOxMDUI03xRjXxI2YzGj1fIM0ClYXfry0PZb78jy4/F952hoqDa0rbB33dRiKcL
AQgADPQ1zVAVzP5U7celPQm0zAMO9BjkdN2DkNK0TdzlJfxjHnlxxnq4f+9YBQS8
k+ksnOU3W/UAkoFShGoZeB3GyAy0IjmmXqO/PDh/PALi+tev4u3ohr7IY1G5NHkg
2L+V8rNMvtJzrPPl5dyVO56Jzp2p14swrGFgrhcAa5cdxG7MrjnE+jfzyyckoIhu
VJbXvIMg7SRGQkyuMPYyH/f9U2hqA0LKNdhKd4igz44vlV56+z5GfUOjyZDgaxg5
7mAeidfxmW2tjKmvhgAdge7atEXEXAF4lM3SejrQ5Cm3BUQWIJFHAJV7aHXlw4pu
ZVjxHwPhqXZeKAukzxdRXMBCHZHqHnozlPr1PbYtzuHOnXM7d59y1XTQh1+kOh3B
+AEFTgA7BjzBjN/bHi8pBjxk90v0OeA99xdTpaAh5kauXgJdP9glTV7MfYCRCZeu
QkAC3iwkAPYPYL5MW6+5/gqbcePC5cFh22eEg9k58jmpWlBUm4b7zWT1kOYIx1W9
T1V4wPVCNit17E2ct9vkM9eF1oLTGBzYB9HYoVK1mAXsOEA7UejIInqvJKRbtSL2
i6L2sJP08BUOeUNP1zJZ
=vPcq
-----END PGP SIGNATURE-----

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Philipp Kern
On Mon, Sep 03, 2012 at 07:42:14PM -0400, Michael Hall wrote:

> On 09/03/2012 04:03 PM, Philipp Kern wrote:
> > am Mon, Sep 03, 2012 at 07:59:15PM +0200 hast du folgendes
> > geschrieben:
> >> * Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
> > what happens in case of a namespace conflict with Debian? Given
> > that apps are not only in dpkg's namespace of all installed
> > applications, as they also refer to paths within /usr. Can a
> > package in Ubuntu be updated through this process? Will a package
> > that appears in Ubuntu proper supersede the application submitted
> > through this process? Will the version for use by apt and dpkg be
> > prefixed, so that it does not collide? (I see todo items
> > referencing a version numbering scheme, but I don't see it spelled
> > out at first glance.)
> Apps already in the archives won't be eligible for this process.  We
> don't have any requirement on version prefixing, the work items are
> there to make sure quickly and the lintian profile are using the
> normal version numbering scheme.
Hm, you dodged the issue of an app getting added to Debian/Ubuntu proper. Will
you forbid epochs so that people cannot do something silly?

Kind regards
Philipp Kern

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (501 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Chow Loong Jin-2
In reply to this post by bvidinli
On 04/09/2012 05:50, bvidinli wrote:
> previously I had prepared package for Ubuntu, for my software.
> However, software was being denied by ubuntu people with the cause
> "program touches system files". This reason was really interesting. My
> software Does touches system files, its job is to do so...
> my program is not in Ubuntu repos since 4-5 years, but it is being
> used since 6 years in ubuntu community, server admins, through classic
> install.sh installation, not apt-get command, since its package was
> not approved by package maintainers + ubuntu package preparation steps
> was too complicated

There probably is more to this picture, but it is really hard to look into what
really happened if you do not even name the software in question.

--
Kind regards,
Loong Jin


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (915 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

TJ-17
On 04/09/12 09:38, Chow Loong Jin wrote:> On 04/09/2012 05:50, bvidinli wrote:

>> previously I had prepared package for Ubuntu, for my software.
>> However, software was being denied by ubuntu people with the cause
>> "program touches system files". This reason was really interesting. My
>> software Does touches system files, its job is to do so...
>> my program is not in Ubuntu repos since 4-5 years, but it is being
>> used since 6 years in ubuntu community, server admins, through classic
>> install.sh installation, not apt-get command, since its package was
>> not approved by package maintainers + ubuntu package preparation steps
>> was too complicated
>
> There probably is more to this picture, but it is really hard to look into what
> really happened if you do not even name the software in question.
>
I suspect it is EHCP

http://ehcp.net/

Looking at it superficially it doesn't give me great confidence so I can understand a reluctance to add it:

"hi everybody, we had an attack last week. (attacker uses safe-host.info) So, please take care of your servers too,similarly (especially if you downloaded ehcp last week, if ehcp installer with file
size bigger than 10M, it may be vulnerable, you may check ehcp installer size to be sure, in your server.)"

Instantly I thought - what about the sha256 sum?

The rejection referred to is:

https://lists.ubuntu.com/archives/app-review-board/2012-February/000381.html



--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Scott Kitterman-3
In reply to this post by David Planella-4
On Monday, September 03, 2012 07:59:15 PM David Planella wrote:

> Hi everyone,
>
> As many of you will know, in the last few cycles we've been laying out
> the foundations to make Ubuntu a target of choice for app developers. I
> am not referring to building the platform here (an area in which we also
> keep seeking growth), but rather to enabling app authors to create and
> distribute original software, to grow a rich ecosystem of independent
> apps for our users.
>
> The resounding success of our latest initiative, the Ubuntu App
> Showdown, has not only shown an explosion of high-quality applications
> (created in just 3 weeks!), but more importantly, the growing interest
> in developing applications for Ubuntu.
>
> As we continue building upon these foundations, we also keep on refining
> our processes to identify and improve areas in which we can provide a
> better experience for app developers. While doing this, we've been
> gathering metrics from different sources –the current queue of apps
> pending review and the Showdown results being the main ones. They all
> provide clear evidence: the current approach to submit third-party apps
> in the Extras repository has already reached the limit in terms of human
> review capability and application throughput.
>
> Despite our best intentions and the Ubuntu App Review Board's epic
> efforts, we're currently putting a strain on reviewers (who cannot keep
> up with the incoming stream of apps) and providing an unsatisfactory
> experience for app authors (who have to endure long delays to be able to
> upload their apps).
>
> During the course of the last few weeks, after having identified the key
> issues and assessed the different options available, Jono Bacon, Michael
> Hall and myself have been working to put forward a proposal for a new,
> improved app developer upload process. We've been not alone in this
> project: we've had the input and help from many others, including the
> App Review Board and members of many other Ubuntu and Canonical teams
> (Security, Foundations, Desktop, Consumer Apps, just to name a few).
>
> We are now at a point where we consider the spec to be ready for public
> review and comment, and we would like to ask for your own contribution
> to this process. Ultimately, the goal is to discuss and scope the new
> app developer upload process at UDS, but we would like to have a solid
> specification to be used in advance as the basis for any UDS sessions
> and blueprints. If you'd like to contribute, you can:
>
> * Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
> * Provide any feedback either on the Feedback section (preferred) or
>   continue the discussion on this thread.
>
> As the people who create Ubuntu, your input is especially valuable, and
> we'd really appreciate it if you could spend some of your time to help
> app development in Ubuntu succeed.

I understand the goal of this and why it's the direction those interested in
making it easier to provide non-Ubuntu apps want things to go.  It's not at
all surprising that the volunteer reviewers were insufficient, since that's
always been the case for new Ubuntu packages as well.

I do think the proposal misses a few points though.  The purpose of /opt was
not only to limit the access that these apps to the rest of the file hierarchy,
it was also to make sure that files provided by these apps could not be in the
path of Ubuntu packages and to avoid namespace contamination, particularly in
/usr/bin.  This proposal addresses neither of those points, so I think it's
inusufficient as it stands.

See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=614907 for an example of
how much 'fun' /usr/bin namespace collisions can be.

Scott K

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

David Planella-4
Al 04/09/12 13:06, En/na Scott Kitterman ha escrit:

> On Monday, September 03, 2012 07:59:15 PM David Planella wrote:
>> Hi everyone,
>>
>> As many of you will know, in the last few cycles we've been laying out
>> the foundations to make Ubuntu a target of choice for app developers. I
>> am not referring to building the platform here (an area in which we also
>> keep seeking growth), but rather to enabling app authors to create and
>> distribute original software, to grow a rich ecosystem of independent
>> apps for our users.
>>
>> The resounding success of our latest initiative, the Ubuntu App
>> Showdown, has not only shown an explosion of high-quality applications
>> (created in just 3 weeks!), but more importantly, the growing interest
>> in developing applications for Ubuntu.
>>
>> As we continue building upon these foundations, we also keep on refining
>> our processes to identify and improve areas in which we can provide a
>> better experience for app developers. While doing this, we've been
>> gathering metrics from different sources –the current queue of apps
>> pending review and the Showdown results being the main ones. They all
>> provide clear evidence: the current approach to submit third-party apps
>> in the Extras repository has already reached the limit in terms of human
>> review capability and application throughput.
>>
>> Despite our best intentions and the Ubuntu App Review Board's epic
>> efforts, we're currently putting a strain on reviewers (who cannot keep
>> up with the incoming stream of apps) and providing an unsatisfactory
>> experience for app authors (who have to endure long delays to be able to
>> upload their apps).
>>
>> During the course of the last few weeks, after having identified the key
>> issues and assessed the different options available, Jono Bacon, Michael
>> Hall and myself have been working to put forward a proposal for a new,
>> improved app developer upload process. We've been not alone in this
>> project: we've had the input and help from many others, including the
>> App Review Board and members of many other Ubuntu and Canonical teams
>> (Security, Foundations, Desktop, Consumer Apps, just to name a few).
>>
>> We are now at a point where we consider the spec to be ready for public
>> review and comment, and we would like to ask for your own contribution
>> to this process. Ultimately, the goal is to discuss and scope the new
>> app developer upload process at UDS, but we would like to have a solid
>> specification to be used in advance as the basis for any UDS sessions
>> and blueprints. If you'd like to contribute, you can:
>>
>> * Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
>> * Provide any feedback either on the Feedback section (preferred) or
>>   continue the discussion on this thread.
>>
>> As the people who create Ubuntu, your input is especially valuable, and
>> we'd really appreciate it if you could spend some of your time to help
>> app development in Ubuntu succeed.
>
> I understand the goal of this and why it's the direction those interested in
> making it easier to provide non-Ubuntu apps want things to go.  It's not at
> all surprising that the volunteer reviewers were insufficient, since that's
> always been the case for new Ubuntu packages as well.
>
> I do think the proposal misses a few points though.  The purpose of /opt was
> not only to limit the access that these apps to the rest of the file hierarchy,
> it was also to make sure that files provided by these apps could not be in the
> path of Ubuntu packages and to avoid namespace contamination, particularly in
> /usr/bin.  This proposal addresses neither of those points, so I think it's
> inusufficient as it stands.
>
Thanks Scott for the feedback. This is indeed a good point and something
that we've discussed, but I agree that we should better flesh it out in
the spec.

However, I disagree in that we're not addressing neither of the points.
For 1. (limiting access to the rest of the file hierarchy) the proposal
covers a mechanism for whitelisting a well-known set of file system
installation locations [1]. It is only additional security in terms of
installation of system files in the same sense /opt did. Actual access
to the filesystem is limited through sandboxing, which is the main
improvement over the /opt approach.

As per 2. (file system conflicts), we're mentioning:

"will [...] rely on [...] file path conflict checking on the archive side"

And I think you're right in that that part needs further development. In
the past, we've discussed a couple of approaches to detect file system
conflicts [2]. They were mainly concerned with scanning the file
contents of packages from different archives, comparing them and
detecting the conflicts. For the scanning part, we've already got an
example on packages.ubuntu.com (reads the Contents.gz file from the
archive and stores the info on a database).

Any input on these or new alternatives would be very useful.

Thanks.

Cheers,
David.

[1]
https://wiki.ubuntu.com/AppDevUploadProcess#TableWhitelistedInstallLocations
[2] /!\ Note: this document is not related to the current spec and some
of the information is out of date or just for reference /!\
https://wiki.ubuntu.com/OptRequirement#Addressing_the_concerns_separately


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (563 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Michael Hall
In reply to this post by Philipp Kern
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 09/04/2012 04:12 AM, Philipp Kern wrote:

> On Mon, Sep 03, 2012 at 07:42:14PM -0400, Michael Hall wrote:
>> On 09/03/2012 04:03 PM, Philipp Kern wrote:
>>> am Mon, Sep 03, 2012 at 07:59:15PM +0200 hast du folgendes
>>> geschrieben:
>>>> * Review the spec at
>>>> https://wiki.ubuntu.com/AppDevUploadProcess
>>> what happens in case of a namespace conflict with Debian?
>>> Given that apps are not only in dpkg's namespace of all
>>> installed applications, as they also refer to paths within
>>> /usr. Can a package in Ubuntu be updated through this process?
>>> Will a package that appears in Ubuntu proper supersede the
>>> application submitted through this process? Will the version
>>> for use by apt and dpkg be prefixed, so that it does not
>>> collide? (I see todo items referencing a version numbering
>>> scheme, but I don't see it spelled out at first glance.)
>> Apps already in the archives won't be eligible for this process.
>> We don't have any requirement on version prefixing, the work
>> items are there to make sure quickly and the lintian profile are
>> using the normal version numbering scheme.
>
> Hm, you dodged the issue of an app getting added to Debian/Ubuntu
> proper. Will you forbid epochs so that people cannot do something
> silly?
>
> Kind regards Philipp Kern
>
>
>
If the app is added to Ubuntu proper (directly or from Debian) for a
development release after it is already in Extras for a stable
release, the Extras package will not be forward-copied to the
development release and the developer will not be able to uploaded it
to the development release.

Michael Hall
[hidden email]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBCAAGBQJQRflbAAoJEInUYcGJgfVyBvcP+wUcJpr2mmP2UpWuaQj0edzG
wRiLMZ3ezU+L/cY18FqAvbs+wPr4QSjtOszDEstuRipVu+4/1Nxh9lSil1zco0Tl
wUuBHjfFUANEBNEdYovoxgi47VcvWbRMvc5iY8HDrneS9U6vf7puOqiQgv61I5Sq
H91evexTH4C4wLZgyDM3wBFNjmDeYI4KPaleRXFNPpsQdstLaNDrYs8PhZO3dmMH
x9eEhLjOACKoeVRp64jrjbz2JHgZbl6RS7kIVEDh/cbaLMtI25/JbgNlZt0v0cxM
R921OnF5RFK8MkeyaGtitCpo/tr/EIRobX9801f0Ebm43OZDW9kgJjYimQ/0pXia
cKjTX8DnT6Qf1BtjcANyhduI3xg43ZwNmXjflq303NMT15DZ5l4BFhglnh9mHBxa
1cj0swRE91I+XGCmCwEISXt6NcYRsuT8L1btjrJ6xjqkM6gggCQoT+LT851MSmDJ
hkPnVzQfaIoskqzALyP4wpLANhZHWqagolq8NEfVaPPRq3OAc5r8Ehvug00eifkq
4XmwYBB0aKeU+jLxSXM13B2jX7rZ/roxiQgIuE8pk8xowwqvP8+btwifOxF20+/J
6/s62FVj2qH0F//r4MPp8/CzzC/vfr7iEmAUAe9TRGSuWN5viJgiqS4m/UUGMCxe
cI1nHlEC2e8nHQ2s3pkM
=hm5K
-----END PGP SIGNATURE-----

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Emmet Hikory-2
In reply to this post by David Planella-4
David Planella wrote:

> Al 04/09/12 13:06, En/na Scott Kitterman ha escrit:
> > I do think the proposal misses a few points though.  The purpose of /opt was
> > not only to limit the access that these apps to the rest of the file hierarchy,
> > it was also to make sure that files provided by these apps could not be in the
> > path of Ubuntu packages and to avoid namespace contamination, particularly in
> > /usr/bin.  This proposal addresses neither of those points, so I think it's
> > inusufficient as it stands.
> >
> As per 2. (file system conflicts), we're mentioning:
>
> "will [...] rely on [...] file path conflict checking on the archive side"
>
> And I think you're right in that that part needs further development. In
> the past, we've discussed a couple of approaches to detect file system
> conflicts [2]. They were mainly concerned with scanning the file
> contents of packages from different archives, comparing them and
> detecting the conflicts. For the scanning part, we've already got an
> example on packages.ubuntu.com (reads the Contents.gz file from the
> archive and stores the info on a database).

    The difficulty here is that there is uncontrolled scope for future
conflict.  While the Contents.gz file is useful (and the conflictchecker
system more so), if both extras and backports are enabled by default, there
is no means by which the review board can determine that a given filename
will not be provided by a backport of a new package imported from Debian.

    While it might be possible to change the backport process to exclude
such packages, and further require that any extras packages that create
such conflict be adjusted for following releases to avoid conflict, this
merely masks the confusion: there may be any number of users who have
various scripts, desktop files, custom launchers, or other local files
that reference a path for which there cannot be any expectation of
continuity between releases.  All such users may be unable to safely upgrade
to the next release without significant local modification: a significant
change to the current process where do-release-upgrade is typically a safe
operation, especially for those who have verified that any installed extras
applications have already become available for the release to which they are
upgrading.

    With the requirement that extras packages only provide files prohibited
in Debian policy-compliant packages, while there is potential for conflict in
search paths for unadorned filenames (foo.cfg, some-executable, icon.png,
etc.), there is no potential for conflict (even future conflict) for full
pathnames.

    Yes, this is exceedingly annoying in terms of providing a seamless
interface for extras packages, but the issue can be avoided safely in other
ways, by defining a common namespace-protected location into which such
packages may store files, and modifying software that uses such things to
search multiple locations for things like desktop files, executables, etc.
(some of this work has already been discussed in depth, and a smaller
portion completed).

    For those packages that require greater system integration, is there a
compelling reason not to provide them as regular packages, with immediate
backports?  Such new packages in backports should be available to users and
provide a future upgrade path.  They will also appear in the various systems
used to track package coordination between Debian and Ubuntu, and may well be
included in Debian directly (or at least considered by those potentially able
to introduce a future namespace collision).

--
Emmet HIKORY

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Scott Kitterman-3
In reply to this post by David Planella-4
On Tuesday, September 04, 2012 02:32:04 PM David Planella wrote:

> Al 04/09/12 13:06, En/na Scott Kitterman ha escrit:
> > On Monday, September 03, 2012 07:59:15 PM David Planella wrote:
> >> Hi everyone,
> >>
> >> As many of you will know, in the last few cycles we've been laying out
> >> the foundations to make Ubuntu a target of choice for app developers. I
> >> am not referring to building the platform here (an area in which we also
> >> keep seeking growth), but rather to enabling app authors to create and
> >> distribute original software, to grow a rich ecosystem of independent
> >> apps for our users.
> >>
> >> The resounding success of our latest initiative, the Ubuntu App
> >> Showdown, has not only shown an explosion of high-quality applications
> >> (created in just 3 weeks!), but more importantly, the growing interest
> >> in developing applications for Ubuntu.
> >>
> >> As we continue building upon these foundations, we also keep on refining
> >> our processes to identify and improve areas in which we can provide a
> >> better experience for app developers. While doing this, we've been
> >> gathering metrics from different sources –the current queue of apps
> >> pending review and the Showdown results being the main ones. They all
> >> provide clear evidence: the current approach to submit third-party apps
> >> in the Extras repository has already reached the limit in terms of human
> >> review capability and application throughput.
> >>
> >> Despite our best intentions and the Ubuntu App Review Board's epic
> >> efforts, we're currently putting a strain on reviewers (who cannot keep
> >> up with the incoming stream of apps) and providing an unsatisfactory
> >> experience for app authors (who have to endure long delays to be able to
> >> upload their apps).
> >>
> >> During the course of the last few weeks, after having identified the key
> >> issues and assessed the different options available, Jono Bacon, Michael
> >> Hall and myself have been working to put forward a proposal for a new,
> >> improved app developer upload process. We've been not alone in this
> >> project: we've had the input and help from many others, including the
> >> App Review Board and members of many other Ubuntu and Canonical teams
> >> (Security, Foundations, Desktop, Consumer Apps, just to name a few).
> >>
> >> We are now at a point where we consider the spec to be ready for public
> >> review and comment, and we would like to ask for your own contribution
> >> to this process. Ultimately, the goal is to discuss and scope the new
> >> app developer upload process at UDS, but we would like to have a solid
> >> specification to be used in advance as the basis for any UDS sessions
> >> and blueprints. If you'd like to contribute, you can:
> >>
> >> * Review the spec at https://wiki.ubuntu.com/AppDevUploadProcess
> >> * Provide any feedback either on the Feedback section (preferred) or
> >>
> >>   continue the discussion on this thread.
> >>
> >> As the people who create Ubuntu, your input is especially valuable, and
> >> we'd really appreciate it if you could spend some of your time to help
> >> app development in Ubuntu succeed.
> >
> > I understand the goal of this and why it's the direction those interested
> > in making it easier to provide non-Ubuntu apps want things to go.  It's
> > not at all surprising that the volunteer reviewers were insufficient,
> > since that's always been the case for new Ubuntu packages as well.
> >
> > I do think the proposal misses a few points though.  The purpose of /opt
> > was not only to limit the access that these apps to the rest of the file
> > hierarchy, it was also to make sure that files provided by these apps
> > could not be in the path of Ubuntu packages and to avoid namespace
> > contamination, particularly in /usr/bin.  This proposal addresses neither
> > of those points, so I think it's inusufficient as it stands.
>
> Thanks Scott for the feedback. This is indeed a good point and something
> that we've discussed, but I agree that we should better flesh it out in
> the spec.
>
> However, I disagree in that we're not addressing neither of the points.
> For 1. (limiting access to the rest of the file hierarchy) the proposal
> covers a mechanism for whitelisting a well-known set of file system
> installation locations [1]. It is only additional security in terms of
> installation of system files in the same sense /opt did. Actual access
> to the filesystem is limited through sandboxing, which is the main
> improvement over the /opt approach.
>
> As per 2. (file system conflicts), we're mentioning:
>
> "will [...] rely on [...] file path conflict checking on the archive side"
>
> And I think you're right in that that part needs further development. In
> the past, we've discussed a couple of approaches to detect file system
> conflicts [2]. They were mainly concerned with scanning the file
> contents of packages from different archives, comparing them and
> detecting the conflicts. For the scanning part, we've already got an
> example on packages.ubuntu.com (reads the Contents.gz file from the
> archive and stores the info on a database).
>
> Any input on these or new alternatives would be very useful.
>
> Thanks.
>
> Cheers,
> David.
>
> [1]
> https://wiki.ubuntu.com/AppDevUploadProcess#TableWhitelistedInstallLocations
> [2] /!\ Note: this document is not related to the current spec and some of
> the information is out of date or just for reference /!\
> https://wiki.ubuntu.com/OptRequirement#Addressing_the_concerns_separately

I'm afraid I wasn't clear about my concerns.

The problem isn't just with file conflicts with current packages, it's that
these packages will now start using up distro namespace.  If some app
developer package ships the file /usr/games/bird-game, even though there's no
current conflict, there is a package sync'ed from Debian that also ships
/usr/games/bird-game then there's a conflict we have to resolve.  In /opt in a
proper vendor namespace this can never happen.

As far as I can tell, this is a fundamental problem with the proposal that I
don't think can be resolved short of installing in /opt (which is why we
started off doing that).

My concern about files being in the path is not about the access that the app
developer package has, but that they may inadvertently or maliciously cause
Ubuntu packages to use files from the app developer package.

A fairly contrived example derived from the above is if the Debian/Ubuntu
package that shipped /usr/games/bird-game was in the archive and an app
developer package was uploaded that shipped /usr/bin/bird-game, it could be
run instead since /usr/bin precedes /usr/games on the standard user path.  Or
maybe a Python based app ships an embedded copy of a module since they've
tested with that version and don't want to test with the distro version and
they write their setup.py to install it in /usr/local/lib/python2.7/dist-
packages.  Suddenly every user of that module is now using the app developer
package version and not the distro version.

Apparmor doesn't help with this problem.

Once again, if you're installing in /opt, this can't happen and I don't see
how to solve it other than installing in /opt (which is another reason the
/opt requirement exists).

The /opt requirement wasn't imposed just to be difficult.  Until all the reasons
for it are addressed, I think it needs to be maintained.

Scott K

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Michael Terry-5
In reply to this post by Michael Hall
On 04/09/12 08:51, Michael Hall wrote:
> If the app is added to Ubuntu proper (directly or from Debian) for a
> development release after it is already in Extras for a stable
> release, the Extras package will not be forward-copied to the
> development release and the developer will not be able to uploaded it
> to the development release.

What about the following scenario?

We add an Extras package "foo" in precise.  Then in quantal, we sync in
some new, unrelated package "foo" from Debian that happens to share the
same name.  Now users that installed "foo" in precise that upgrade to
quantal will be replacing their "foo" package with a completely
unrelated one.

I imagine the easiest way to avoid this is to namespace the package
names too.

(Though with namespaced packages, the user can end up in a weird
situation if the Extras package "foo" does find its way into Debian and
Ubuntu quantal.  Then, they really do want the "foo" package from Debian
but are left with the non-maintained "extras-foo" from precise.)

-mt

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Matt Wheeler-2

On Sep 4, 2012 3:00 PM, "Michael Terry" <[hidden email]> wrote:
[snip]
> (Though with namespaced packages, the user can end up in a weird situation if the Extras package "foo" does find its way into Debian and Ubuntu quantal.  Then, they really do want the "foo" package from Debian but are left with the non-maintained "extras-foo" from precise.)

That can be handled the usual way package name transitions happen (e.g. git-core), perhaps it could even be totally automated?

--
Matt Wheeler
[hidden email]


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Emmet Hikory-2
In reply to this post by Michael Terry-5
Michael Terry wrote:
> I imagine the easiest way to avoid this is to namespace the package
> names too.
>
> (Though with namespaced packages, the user can end up in a weird
> situation if the Extras package "foo" does find its way into Debian
> and Ubuntu quantal.  Then, they really do want the "foo" package
> from Debian but are left with the non-maintained "extras-foo" from
> precise.)

    One way to work around these cases would be to have a transitional
extras-foo in quantal-extras, which Depends: on foo, so that upgrading
users would migrate from extras-foo to foo during the upgrade process.
At some point the extras-foo package ought be removed from the system,
but this is true for all the historical transitional packages, and so
better handled as part of a general solution for that class of problem.

--
Emmet HIKORY

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Michael Terry-5
On 04/09/12 10:12, Emmet Hikory wrote:

> Michael Terry wrote:
>> I imagine the easiest way to avoid this is to namespace the package
>> names too.
>>
>> (Though with namespaced packages, the user can end up in a weird
>> situation if the Extras package "foo" does find its way into Debian
>> and Ubuntu quantal.  Then, they really do want the "foo" package
>> from Debian but are left with the non-maintained "extras-foo" from
>> precise.)
>      One way to work around these cases would be to have a transitional
> extras-foo in quantal-extras, which Depends: on foo, so that upgrading
> users would migrate from extras-foo to foo during the upgrade process.
> At some point the extras-foo package ought be removed from the system,
> but this is true for all the historical transitional packages, and so
> better handled as part of a general solution for that class of problem.

Good point.  That's a way around the (likely not overwhelming) number of
Extras packages that make it into Debian if we namespace the packages.

So I'm even more certain that we should namespace the package names
themselves.  Without doing so, we can't really guarantee that the
package won't switch completely out from under the user.  We'd both be
removing an app they thought they had as well as installing a package
they didn't request (which could have quite bad side effects).

-mt

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Michael Hall
In reply to this post by Scott Kitterman-3

On 09/04/2012 09:39 AM, Scott Kitterman wrote:
> The problem isn't just with file conflicts with current packages, it's that
> these packages will now start using up distro namespace.  If some app
> developer package ships the file /usr/games/bird-game, even though there's no
> current conflict, there is a package sync'ed from Debian that also ships
> /usr/games/bird-game then there's a conflict we have to resolve.  In /opt in a
> proper vendor namespace this can never happen.

If bird-game already exists in Extras, and then a different package is
allowed into backports that will install files into the same location,
then yes there is a possibility for a conflict.  But I assume part of
the backports approval process already checks for conflicts, as they may
exist with another package in the stable release already, so that
process could easily be extended to include Extras packages as well.


> A fairly contrived example derived from the above is if the Debian/Ubuntu
> package that shipped /usr/games/bird-game was in the archive and an app
> developer package was uploaded that shipped /usr/bin/bird-game, it could be
> run instead since /usr/bin precedes /usr/games on the standard user path.

This would only happen if two different apps were both called
"bird-game" but otherwise different.  This situation is also possible
already, even between two packages in Universe.  How is that currently
handled?

> maybe a Python based app ships an embedded copy of a module since they've
> tested with that version and don't want to test with the distro version and
> they write their setup.py to install it in /usr/local/lib/python2.7/dist-
> packages.  Suddenly every user of that module is now using the app developer
> package version and not the distro version.

Installing python modules in the system directories is not allowed under
this process.  Check the whitelist defined in the "App Preparation"
section of the spec.


Michael Hall
[hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Daniel Holbach-2
In reply to this post by Emmet Hikory-2
Hello,

On 04.09.2012 15:28, Emmet Hikory wrote:
>     The difficulty here is that there is uncontrolled scope for future
> conflict.  While the Contents.gz file is useful (and the conflictchecker
> system more so), if both extras and backports are enabled by default, there
> is no means by which the review board can determine that a given filename
> will not be provided by a backport of a new package imported from Debian.

The fairest solution to this problem would be to turn on a
conflict-check-before-publish for all parts of the archive. This would
help us find these (and pre-existing) issues immediately and resolve
them amongst the maintainers and upstreams.

My current expectation is only a very tiny fraction of uploads would get
blocked due to this and the general amount of work to resolve them would
be small too.

The /opt requirement on the other hand unfortunately imposes a huge
amount of work on 1) app developers because our tools don't work this
way very easily and 2) Ubuntu maintainers who have to enable path
lookups in tools which don't know about /opt yet.

Have a great day,
 Daniel

--
Get involved in Ubuntu development! developer.ubuntu.com/packaging
And follow @ubuntudev on identi.ca/twitter.com/facebook.com/gplus.to

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Proposing a New App Developer Upload Process

Michael Hall
In reply to this post by Michael Terry-5

> What about the following scenario?
>
> We add an Extras package "foo" in precise.  Then in quantal, we sync in
> some new, unrelated package "foo" from Debian that happens to share the
> same name.  Now users that installed "foo" in precise that upgrade to
> quantal will be replacing their "foo" package with a completely
> unrelated one.
>

It's possible for me to upload a package to Universe in Precise's
development phase, then for an unrelated package by the same name to be
in Debian's archives when we do the Quantal sync.

How is this currently handled?


Michael Hall
[hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
1234 ... 6