More diagnostics data from desktop

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

More diagnostics data from desktop

Will Cooke-2
Dear all,

We want to be able to focus our engineering efforts on the things that matter most to our users, and in order to do that we need to get some more data about sort of setups our users have and which software they are running on it.

We would like to add a checkbox to the installer, exact wording TBD, but along the lines of “Send diagnostics information to help improve Ubuntu”.  This would be checked by default.

The result of having that box checked would be:

* Information from the installation would be sent over HTTPS to a service run by Canonical’s IS team.  This would be saved to disk and sent on first boot once there is a network connection.  The file containing this data would be available for the user to inspect.

That data would include:
   * Ubuntu Flavour
   * Ubuntu Version
   * Network connectivity or not
   * CPU family
   * RAM
   * Disk(s) size
   * Screen(s) resolution
   * GPU vendor and model
   * OEM Manufacturer
   * Location (based on the location selection made by the user at install).  No IP information would be gathered
   * Installation duration (time taken)
   * Auto login enabled or not
   * Disk layout selected
   * Third party software selected or not
   * Download updates during install or not
   * LivePatch enabled or not

* Popcon would be installed.  This will allow us to spot trends in package usage and help us to  focus on the packages which are of most value to our users.

* Apport would be configured to automatically send anonymous crash reports without user interruption.

The results of this data would be made public.  E.g. People would be able to see that X% of Ubuntu users are based in .de vs Y% in .za.  Z% of our users run Dell hardware, and so on.
The Ubuntu privacy policy would be updated to reflect this change.

Any user can simply opt out by unchecking the box, which triggers one simple POST stating, “diagnostics=false”.  There will be a corresponding checkbox in the Privacy panel of GNOME Settings to toggle the state of this.

And to reiterate, the service which stores this data would *never* store IP addresses.

We value your feedback and comments!

Cheers, Will
On behalf of the Ubuntu Desktop Team


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Cassidy James Blaede
This makes sense from Ubuntu's perspective, and it will certainly be interesting to see the resulting data. I have a few concerns, but nothing insurmountable:

How will this affect downstreams? Downstreams/non-official-flavors may want to disable or remove any diagnostics. Keep them in mind when designing the implementation.

Users are not always installers. Will additional users be prompted of the diagnostics upon first login? Are these diagnostics intended to be system-wide or user-wide? If Ubuntu uses GNOME Initial Setup for new users, that would be a great place for this.

Cassidy

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Ian Bruntlett
In reply to this post by Will Cooke-2
Hi Will,

On 14 February 2018 at 15:22, Will Cooke <[hidden email]> wrote:
We want to be able to focus our engineering efforts on the things that matter most to our users, and in order to do that we need to get some more data about sort of setups our users have and which software they are running on it.

We would like to add a checkbox to the installer, exact wording TBD, but along the lines of “Send diagnostics information to help improve Ubuntu”.  This would be checked by default.

Fine by me.I refurbish old donated laptops and PCs and pass them on to people with mental health problems, their family and carers - this is the Computer Wombling Project. Enabling the above by default would be a good idea as quite a few of the end users would struggle to cope with this sort of thing.

BW,


Ian

--
-- ACCU - Professionalism in programming - http://www.accu.org
-- My writing - https://sites.google.com/site/ianbruntlett/


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Juerg Haefliger
In reply to this post by Will Cooke-2
On 02/14/2018 04:22 PM, Will Cooke wrote:

> Dear all,
>
> We want to be able to focus our engineering efforts on the things that
> matter most to our users, and in order to do that we need to get some
> more data about sort of setups our users have and which software they
> are running on it.
>
> We would like to add a checkbox to the installer, exact wording TBD, but
> along the lines of “Send diagnostics information to help improve
> Ubuntu”.  This would be checked by default.
Please make this an opt-in rather than an opt-out. This just smells like
a trend towards a Windows/Android installation where you have to unset
gazillions of check boxes to prevent the machine from posting your life
to the vendor. We shouldn't go there.


> The result of having that box checked would be:
>
> * Information from the installation would be sent over HTTPS to a
> service run by Canonical’s IS team.  This would be saved to disk and
> sent on first boot once there is a network connection.

So sent only once or after every reboot?


> The file
> containing this data would be available for the user to inspect.
>
> That data would include:
>    * Ubuntu Flavour
>    * Ubuntu Version
>    * Network connectivity or not
>    * CPU family
>    * RAM
>    * Disk(s) size
>    * Screen(s) resolution
>    * GPU vendor and model
>    * OEM Manufacturer
>    * Location (based on the location selection made by the user at
> install).  No IP information would be gathered
>    * Installation duration (time taken)
>    * Auto login enabled or not
>    * Disk layout selected
>    * Third party software selected or not
>    * Download updates during install or not
>    * LivePatch enabled or not
>
> * Popcon would be installed.  This will allow us to spot trends in
> package usage and help us to  focus on the packages which are of most
> value to our users.
Are you saying that popcon is automatically installed and enabled? I
haven't performed an Ubuntu install lately but isn't there an install
question asking whether to enable popcon or not (with the default being
no). Or is that Debian?


> * Apport would be configured to automatically send anonymous crash
> reports without user interruption.

I hope this will be clearly articulated during install time.


> The results of this data would be made public.

Same here. People need to know that their data is publicly (yet
anonymously) visible.


> E.g. People would be
> able to see that X% of Ubuntu users are based in .de vs Y% in .za.  Z%
> of our users run Dell hardware, and so on.
> The Ubuntu privacy policy would be updated to reflect this change.
>
> Any user can simply opt out by unchecking the box, which triggers one
> simple POST stating, “diagnostics=false”.

Why does this require a POST (over the network)?


> There will be a corresponding
> checkbox in the Privacy panel of GNOME Settings to toggle the state of this.
>
> And to reiterate, the service which stores this data would *never* store
> IP addresses.
>
> We value your feedback and comments!

I don't believe that sending data by default is 'a thing that matters
most to our users'. Quite the opposite in fact. MS was/is getting a lot
of heat for their data collection and we shouldn't go down that very
same route by making data gathering the default.

...Juerg


> Cheers, Will
> On behalf of the Ubuntu Desktop Team
>
>
>


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (861 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Will Cooke-2
In reply to this post by Will Cooke-2
On 14 February 2018 at 18:37, Alistair Buxton <[hidden email]> wrote:
> * Information from the installation would be sent over HTTPS to a service
> run by Canonical’s IS team.  This would be saved to disk and sent on first
> boot once there is a network connection.  The file containing this data
> would be available for the user to inspect.

So you ask the user during install. Then the data is sent on first
boot. At what point can the user inspect the data, given that some of
it can't be collected until after installation is finished? It seems
like the first opportunity will be after it has been sent, unless you
ask the user a second time. So why not just ask them on first boot,
when you have already gathered all the data? That way user can inspect
the data there and then before deciding how to answer.

Yes, I think the first opportunity would be after it has been sent.  I'm generally against asking more questions on login though, I think it would be clunky.
We're considering ways to make the tool a stand-alone snap which could be much more interactive and could be installed and run independently of the "Send diagnostics" checkbox.

 
>    * Screen(s) resolution

This won't be correct after first boot on my system. I have to reboot
at least one more time to install working graphics drivers.

Good point, thanks.  We're more interested in general trends than the specifics, so this might skew the data a bit, I think it will work out in the end.




--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Ernst Persson
In reply to this post by Will Cooke-2
Hi,

"Send diagnostics information to help improve Ubuntu" sounds like
you're continuously reporting things, but your proposal looks more
like a single "installation ping" or something.
I guess the apport and popcon reports would be continuous though?

If you do this you should really make sure everything has http proxy
(set in user session only?) support, so you can pick up some extra
corporate users.
Sending it from the user session fits well with the privacy settings.

Sounds reasonable otherwise. Are trackpads enought trouble to collect data on?

Regards
//Ernst

2018-02-14 16:22 GMT+01:00 Will Cooke <[hidden email]>:

> Dear all,
>
> We want to be able to focus our engineering efforts on the things that
> matter most to our users, and in order to do that we need to get some more
> data about sort of setups our users have and which software they are running
> on it.
>
> We would like to add a checkbox to the installer, exact wording TBD, but
> along the lines of “Send diagnostics information to help improve Ubuntu”.
> This would be checked by default.
>
> The result of having that box checked would be:
>
> * Information from the installation would be sent over HTTPS to a service
> run by Canonical’s IS team.  This would be saved to disk and sent on first
> boot once there is a network connection.  The file containing this data
> would be available for the user to inspect.
>
> That data would include:
>    * Ubuntu Flavour
>    * Ubuntu Version
>    * Network connectivity or not
>    * CPU family
>    * RAM
>    * Disk(s) size
>    * Screen(s) resolution
>    * GPU vendor and model
>    * OEM Manufacturer
>    * Location (based on the location selection made by the user at install).
> No IP information would be gathered
>    * Installation duration (time taken)
>    * Auto login enabled or not
>    * Disk layout selected
>    * Third party software selected or not
>    * Download updates during install or not
>    * LivePatch enabled or not
>
> * Popcon would be installed.  This will allow us to spot trends in package
> usage and help us to  focus on the packages which are of most value to our
> users.
>
> * Apport would be configured to automatically send anonymous crash reports
> without user interruption.
>
> The results of this data would be made public.  E.g. People would be able to
> see that X% of Ubuntu users are based in .de vs Y% in .za.  Z% of our users
> run Dell hardware, and so on.
> The Ubuntu privacy policy would be updated to reflect this change.
>
> Any user can simply opt out by unchecking the box, which triggers one simple
> POST stating, “diagnostics=false”.  There will be a corresponding checkbox
> in the Privacy panel of GNOME Settings to toggle the state of this.
>
> And to reiterate, the service which stores this data would *never* store IP
> addresses.
>
> We value your feedback and comments!
>
> Cheers, Will
> On behalf of the Ubuntu Desktop Team
>
>
> --
> ubuntu-devel mailing list
> [hidden email]
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
>

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Will Cooke-2
In reply to this post by Juerg Haefliger
On 15 February 2018 at 07:10, Juerg Haefliger <[hidden email]> wrote:
On 02/14/2018 04:22 PM, Will Cooke wrote:
> Dear all,
>
> We want to be able to focus our engineering efforts on the things that
> matter most to our users, and in order to do that we need to get some
> more data about sort of setups our users have and which software they
> are running on it.
>
> We would like to add a checkbox to the installer, exact wording TBD, but
> along the lines of “Send diagnostics information to help improve
> Ubuntu”.  This would be checked by default.

Please make this an opt-in rather than an opt-out. This just smells like
a trend towards a Windows/Android installation where you have to unset
gazillions of check boxes to prevent the machine from posting your life
to the vendor. We shouldn't go there.

I can understand why you would feel that way, but I honestly believe it's not the same.  We want to be as transparent as we can be, and making data available, and indeed the source code to show what's been gathered, and that we posted the proposed list of data here I hope would assure you that this isn't the same as other platforms.  If we make it opt-in I think we will lose out on a lot of valuable data.  We're working with design to make sure that the message is clear and understable to everyone so people can make an informed decision.

> * Information from the installation would be sent over HTTPS to a
> service run by Canonical’s IS team.  This would be saved to disk and
> sent on first boot once there is a network connection.

So sent only once or after every reboot?

Only once.
 
> * Popcon would be installed.  This will allow us to spot trends in
> package usage and help us to  focus on the packages which are of most
> value to our users.

Are you saying that popcon is automatically installed and enabled? I
haven't performed an Ubuntu install lately but isn't there an install
question asking whether to enable popcon or not (with the default being
no). Or is that Debian?

If the box is left checked, then popcon would get installed.  In the normal desktop installer there isn't an option for that any more.  There used to be in the past, and there was an option to turn it on in Software Sources, but that went away ages ago.  10.04 I think.
 

> * Apport would be configured to automatically send anonymous crash
> reports without user interruption.

I hope this will be clearly articulated during install time.


We're working with design to get the wording just right.
 

> The results of this data would be made public.

Same here. People need to know that their data is publicly (yet
anonymously) visible.


> E.g. People would be
> able to see that X% of Ubuntu users are based in .de vs Y% in .za.  Z%
> of our users run Dell hardware, and so on.
> The Ubuntu privacy policy would be updated to reflect this change.
>
> Any user can simply opt out by unchecking the box, which triggers one
> simple POST stating, “diagnostics=false”.

Why does this require a POST (over the network)?


This will allow us to gauge participation rates and to better quantify the data we do get.

 

> There will be a corresponding
> checkbox in the Privacy panel of GNOME Settings to toggle the state of this.
>
> And to reiterate, the service which stores this data would *never* store
> IP addresses.
>
> We value your feedback and comments!

I don't believe that sending data by default is 'a thing that matters
most to our users'. Quite the opposite in fact. MS was/is getting a lot
of heat for their data collection and we shouldn't go down that very
same route by making data gathering the default.


Noted.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Will Cooke-2
In reply to this post by Ernst Persson

On 15 February 2018 at 10:16, Ernst Sjöstrand <[hidden email]> wrote:
Hi,

"Send diagnostics information to help improve Ubuntu" sounds like
you're continuously reporting things, but your proposal looks more
like a single "installation ping" or something.
I guess the apport and popcon reports would be continuous though?


Yes, installation data would be one shot, but popcon and apport would be recurring.  We need to work on the exact wording to make this clear.

 
If you do this you should really make sure everything has http proxy
(set in user session only?) support, so you can pick up some extra
corporate users.
Sending it from the user session fits well with the privacy settings.

Noted, thanks for the advice.
 
Sounds reasonable otherwise. Are trackpads enought trouble to collect data on?


Interesting point, we will look in to this some more and update the list with our findings.


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Will Cooke-2
In reply to this post by Cassidy James Blaede
Hi Cassidy,


On 14 February 2018 at 16:00, Cassidy James Blaede <[hidden email]> wrote:
This makes sense from Ubuntu's perspective, and it will certainly be interesting to see the resulting data. I have a few concerns, but nothing insurmountable:

How will this affect downstreams? Downstreams/non-official-flavors may want to disable or remove any diagnostics. Keep them in mind when designing the implementation.

Noted.
 

Users are not always installers. Will additional users be prompted of the diagnostics upon first login? Are these diagnostics intended to be system-wide or user-wide? If Ubuntu uses GNOME Initial Setup for new users, that would be a great place for this.


I'm generally against asking more questions on log in if it can be avoided.  Apport and popcon would be system wide.  The installer data would naturally be based the system.  We aren't going to be using Initial Setup in 18.04, but I will bear that in mind for the near future, it does make sense to include it in Initial Setup if/when we use it.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Boris Pek
In reply to this post by Will Cooke-2
Hi there,

Just a small note about Popcon:

> * Popcon would be installed.  This will allow us to spot trends in package
> usage and help us to  focus on the packages which are of most value to our
> users.

It is absolutely useless until its problems are not fixed. For example:
https://bugs.launchpad.net/ubuntu/+source/popularity-contest/+bug/1067277
https://bugs.launchpad.net/ubuntu/+source/popularity-contest/+bug/1711420

Best regards,
Boris


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Sven Schwedas
In reply to this post by Will Cooke-2
> We want to be able to focus our engineering efforts on the things that
> matter most to our users

Getting turned into Analytics fodder does not matter to us. That you're
too clueless to know what your job is does not matter to us.

> This would be checked by default.

Please go back to your legal department and ask them what the GDPR is,
and why you need to make this opt in.

--
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas, Systemadministrator
[hidden email] | ☎ +43 680 301 7167
TAO Digital   | Teil der TAO Beratungs- & Management GmbH
Lendplatz 45  | FN 213999f/Klagenfurt, FB-Gericht Villach
A8020 Graz    | https://www.tao-digital.at


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

J Fernyhough
In reply to this post by Will Cooke-2
On 15/02/18 10:05, Will Cooke wrote:

> On 14 February 2018 at 18:37, Alistair Buxton <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     > * Information from the installation would be sent over HTTPS to a service
>     > run by Canonical’s IS team.  This would be saved to disk and sent on first
>     > boot once there is a network connection.  The file containing this data
>     > would be available for the user to inspect.
>
>     So you ask the user during install. Then the data is sent on first
>     boot. At what point can the user inspect the data, given that some of
>     it can't be collected until after installation is finished? It seems
>     like the first opportunity will be after it has been sent, unless you
>     ask the user a second time. So why not just ask them on first boot,
>     when you have already gathered all the data? That way user can inspect
>     the data there and then before deciding how to answer.
>
>
> Yes, I think the first opportunity would be after it has been sent.  I'm
> generally against asking more questions on login though, I think it
> would be clunky.
Am I reading it correctly that you will allow the user to see what data
had been gathered from the system only _after_ it has been sent? That
comes across as needlessly sneaky.

Surely it could be deferred until the after the user has had the
opportunity to agree properly?

As an existing implementation, the Steam client has a perfectly good way
of doing this - it pops up a dialogue box, asks whether it can send
system data, shows the data that would be sent, and explains why it is
useful and why you should consider allowing it.


On 14/02/18 15:22, Will Cooke wrote:
> Any user can simply opt out by unchecking the box, which triggers one
> simple POST stating, “diagnostics=false”.

This doesn't scan right either - you're collecting data about someone
opting out of data collection?


I can see the reasons behind collecting data but let's not make the
collection process needlessly aggressive. That's just going to make
people defensive and find ways of disabling/avoiding it entirely (e.g.
network blocks) instead of considering how it can help Ubuntu in the
long-run.

An overly-aggressive approach also makes it much more difficult for
other projects to implement statistics collection without users equating
it with user tracking/telemetry/spying/etc. and complaining vociferously
(even without any real understanding of what the process means - just
the presence of the words "data collection" is enough to generate an
awful lot of noise).


J


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (883 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

flocculant
In reply to this post by Cassidy James Blaede
On 14/02/18 16:00, Cassidy James Blaede wrote:
... If Ubuntu uses GNOME Initial Setup for new users, that would be a great place for this.

Cassidy


That's great if they are only interested in Ubuntu - flavours who might want to see the results of it done via Ubiquity won't be having gnome initial setup ...

regards


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Dustin Kirkland-5
In reply to this post by Ernst Persson
On Thu, Feb 15, 2018 at 4:16 AM, Ernst Sjöstrand <[hidden email]> wrote:
> Hi,
>
> "Send diagnostics information to help improve Ubuntu" sounds like
> you're continuously reporting things, but your proposal looks more
> like a single "installation ping" or something.
> I guess the apport and popcon reports would be continuous though?

The list that Will shared would be a one-time shot, after install,
asynchronously after networking is up on first boot (unless
opted-out).

Popcon would report periodically, and Apport would report crashes
(again, unless opted out).

> If you do this you should really make sure everything has http proxy
> (set in user session only?) support, so you can pick up some extra
> corporate users.

For sure!  Good catch.

> Sending it from the user session fits well with the privacy settings.
>
> Sounds reasonable otherwise. Are trackpads enought trouble to collect data on?

Potentially.  That's the exact sort thing that we need data on, to
help make Ubuntu better :-)

Cheers,
Dustin

> Regards
> //Ernst
>
> 2018-02-14 16:22 GMT+01:00 Will Cooke <[hidden email]>:
>> Dear all,
>>
>> We want to be able to focus our engineering efforts on the things that
>> matter most to our users, and in order to do that we need to get some more
>> data about sort of setups our users have and which software they are running
>> on it.
>>
>> We would like to add a checkbox to the installer, exact wording TBD, but
>> along the lines of “Send diagnostics information to help improve Ubuntu”.
>> This would be checked by default.
>>
>> The result of having that box checked would be:
>>
>> * Information from the installation would be sent over HTTPS to a service
>> run by Canonical’s IS team.  This would be saved to disk and sent on first
>> boot once there is a network connection.  The file containing this data
>> would be available for the user to inspect.
>>
>> That data would include:
>>    * Ubuntu Flavour
>>    * Ubuntu Version
>>    * Network connectivity or not
>>    * CPU family
>>    * RAM
>>    * Disk(s) size
>>    * Screen(s) resolution
>>    * GPU vendor and model
>>    * OEM Manufacturer
>>    * Location (based on the location selection made by the user at install).
>> No IP information would be gathered
>>    * Installation duration (time taken)
>>    * Auto login enabled or not
>>    * Disk layout selected
>>    * Third party software selected or not
>>    * Download updates during install or not
>>    * LivePatch enabled or not
>>
>> * Popcon would be installed.  This will allow us to spot trends in package
>> usage and help us to  focus on the packages which are of most value to our
>> users.
>>
>> * Apport would be configured to automatically send anonymous crash reports
>> without user interruption.
>>
>> The results of this data would be made public.  E.g. People would be able to
>> see that X% of Ubuntu users are based in .de vs Y% in .za.  Z% of our users
>> run Dell hardware, and so on.
>> The Ubuntu privacy policy would be updated to reflect this change.
>>
>> Any user can simply opt out by unchecking the box, which triggers one simple
>> POST stating, “diagnostics=false”.  There will be a corresponding checkbox
>> in the Privacy panel of GNOME Settings to toggle the state of this.
>>
>> And to reiterate, the service which stores this data would *never* store IP
>> addresses.
>>
>> We value your feedback and comments!
>>
>> Cheers, Will
>> On behalf of the Ubuntu Desktop Team
>>
>>
>> --
>> ubuntu-devel mailing list
>> [hidden email]
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
>>
>
> --
> ubuntu-devel mailing list
> [hidden email]
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Dustin Kirkland-5
In reply to this post by Juerg Haefliger
On Thu, Feb 15, 2018 at 1:10 AM, Juerg Haefliger
<[hidden email]> wrote:

> On 02/14/2018 04:22 PM, Will Cooke wrote:
>> Dear all,
>>
>> We want to be able to focus our engineering efforts on the things that
>> matter most to our users, and in order to do that we need to get some
>> more data about sort of setups our users have and which software they
>> are running on it.
>>
>> We would like to add a checkbox to the installer, exact wording TBD, but
>> along the lines of “Send diagnostics information to help improve
>> Ubuntu”.  This would be checked by default.
>
> Please make this an opt-in rather than an opt-out. This just smells like
> a trend towards a Windows/Android installation where you have to unset
> gazillions of check boxes to prevent the machine from posting your life
> to the vendor. We shouldn't go there.
>
>
>> The result of having that box checked would be:
>>
>> * Information from the installation would be sent over HTTPS to a
>> service run by Canonical’s IS team.  This would be saved to disk and
>> sent on first boot once there is a network connection.
>
> So sent only once or after every reboot?

Only once.

>
>> The file
>> containing this data would be available for the user to inspect.
>>
>> That data would include:
>>    * Ubuntu Flavour
>>    * Ubuntu Version
>>    * Network connectivity or not
>>    * CPU family
>>    * RAM
>>    * Disk(s) size
>>    * Screen(s) resolution
>>    * GPU vendor and model
>>    * OEM Manufacturer
>>    * Location (based on the location selection made by the user at
>> install).  No IP information would be gathered
>>    * Installation duration (time taken)
>>    * Auto login enabled or not
>>    * Disk layout selected
>>    * Third party software selected or not
>>    * Download updates during install or not
>>    * LivePatch enabled or not
>>
>> * Popcon would be installed.  This will allow us to spot trends in
>> package usage and help us to  focus on the packages which are of most
>> value to our users.
>
> Are you saying that popcon is automatically installed and enabled? I
> haven't performed an Ubuntu install lately but isn't there an install
> question asking whether to enable popcon or not (with the default being
> no). Or is that Debian?

There is currently no "enable popcon" install question in Ubuntu
(maybe there is in Debian)?

In Will's proposal, there's a single, simple, clear checkbox, along
the lines of "[X] Send diagnostic information", which, if enabled,
would:
 (1) post this initial list of installation options and hardware capability,
 (2) enable popcon to periodically report lists of installed packages, and
 (3) enable Apport to post crash reports

Declining to send diagnostics, would disable all 3.

>> * Apport would be configured to automatically send anonymous crash
>> reports without user interruption.
>
> I hope this will be clearly articulated during install time.
>
>
>> The results of this data would be made public.
>
> Same here. People need to know that their data is publicly (yet
> anonymously) visible.

Indeed.  DockerHub does a nice job of making their statistics publicly
available, and many, interesting 3rd party tools exist to treat that
data.

Here, you can see how many times the Ubuntu (or any other) Docker
image has been pulled, updated in real time:

$ wget -q -O- https://hub.docker.com/v2/repositories/library/ubuntu/ |
python -m json.tool

>> E.g. People would be
>> able to see that X% of Ubuntu users are based in .de vs Y% in .za.  Z%
>> of our users run Dell hardware, and so on.
>> The Ubuntu privacy policy would be updated to reflect this change.
>>
>> Any user can simply opt out by unchecking the box, which triggers one
>> simple POST stating, “diagnostics=false”.
>
> Why does this require a POST (over the network)?

Understanding the sample size, as a ratio to the total population, is
an essential characteristic in determining statistical validity, and
required to draw accurate inferences based on the data.

>> There will be a corresponding
>> checkbox in the Privacy panel of GNOME Settings to toggle the state of this.
>>
>> And to reiterate, the service which stores this data would *never* store
>> IP addresses.
>>
>> We value your feedback and comments!
>
> I don't believe that sending data by default is 'a thing that matters
> most to our users'. Quite the opposite in fact. MS was/is getting a lot
> of heat for their data collection and we shouldn't go down that very
> same route by making data gathering the default.

I do believe that quality, stability, security, usability, and free
availability are what matters most to Ubuntu users.

We can drastically improve the quality of Ubuntu through tastefully,
anonymously gathered diagnostics.  We can significantly improve the
stability of Ubuntu by automatically processing crash reports.  We can
seriously improve the security of Ubuntu by analyzing the package sets
most frequently installed and focusing our resources on the software
that matters most.

Cheers!
Dustin

> ...Juerg
>
>
>> Cheers, Will
>> On behalf of the Ubuntu Desktop Team
>>
>>
>>
>
>
> --
> ubuntu-devel mailing list
> [hidden email]
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
>

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Etienne Papegnies
In reply to this post by Will Cooke-2

Hello

I'm already seeing some blowback from the community, so I'd like to suggest that a few things be made clear.

There is three collection methods: 1) Post-Install report; 2) Popcon; and 3) Apport.

- It'd be nice to have some insurance that Canonical does not have the ability to match these three report sources by means for instance of a machine identifier.

- 1) if there is no chance for the user to inspect this data before it is sent at first boot, one should make sure it does not include Hardware UIDs (For disks, Mac Addresses for network interfaces, etc)

- 2) I don't know the popcon backend, but it'd be nice to know that the full list of packages for a given report is not stored (because such information can be used for tracking)

- 3) If Apport automatically submits a report a StackTrace is OK but no coredump data should be included

Cheers

On 14/02/2018 16:22, Will Cooke wrote:
Dear all,

We want to be able to focus our engineering efforts on the things that matter most to our users, and in order to do that we need to get some more data about sort of setups our users have and which software they are running on it.

We would like to add a checkbox to the installer, exact wording TBD, but along the lines of “Send diagnostics information to help improve Ubuntu”.  This would be checked by default.

The result of having that box checked would be:

* Information from the installation would be sent over HTTPS to a service run by Canonical’s IS team.  This would be saved to disk and sent on first boot once there is a network connection.  The file containing this data would be available for the user to inspect.

That data would include:
   * Ubuntu Flavour
   * Ubuntu Version
   * Network connectivity or not
   * CPU family
   * RAM
   * Disk(s) size
   * Screen(s) resolution
   * GPU vendor and model
   * OEM Manufacturer
   * Location (based on the location selection made by the user at install).  No IP information would be gathered
   * Installation duration (time taken)
   * Auto login enabled or not
   * Disk layout selected
   * Third party software selected or not
   * Download updates during install or not
   * LivePatch enabled or not

* Popcon would be installed.  This will allow us to spot trends in package usage and help us to  focus on the packages which are of most value to our users.

* Apport would be configured to automatically send anonymous crash reports without user interruption.

The results of this data would be made public.  E.g. People would be able to see that X% of Ubuntu users are based in .de vs Y% in .za.  Z% of our users run Dell hardware, and so on.
The Ubuntu privacy policy would be updated to reflect this change.

Any user can simply opt out by unchecking the box, which triggers one simple POST stating, “diagnostics=false”.  There will be a corresponding checkbox in the Privacy panel of GNOME Settings to toggle the state of this.

And to reiterate, the service which stores this data would *never* store IP addresses.

We value your feedback and comments!

Cheers, Will
On behalf of the Ubuntu Desktop Team





--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Matthew Paul Thomas
In reply to this post by Will Cooke-2
Will Cooke wrote on 14/02/18 15:22:
>…
> We want to be able to focus our engineering efforts on the things that
> matter most to our users, and in order to do that we need to get some
> more data about sort of setups our users have and which software they
> are running on it.
>
> We would like to add a checkbox to the installer, exact wording TBD,
> but along the lines of “Send diagnostics information to help improve
> Ubuntu”.  This would be checked by default.

I’ve just drafted a design for this. <https://goo.gl/yJ86Qa> It’s
basically a subset of the System Settings screen.

> The result of having that box checked would be:
>
> * Information from the installation would be sent over HTTPS to a
> service run by Canonical’s IS team.  This would be saved to disk and
> sent on first boot once there is a network connection.

Information from the installation would be fascinating, for improvement
of the installer in particular. However, I don’t think it would give you
an accurate idea about the “sort of setups our users have”, for
improvement of Ubuntu in general. It could lead you to think that, for
example:

*   Internet connection is less common than it really is. (Because of
    things like proxies, as mentioned by Ernst Sjöstrand, or
    not-yet-installed wi-fi drivers. And because if people still can’t
    get online later, they might uninstall Ubuntu and you’ll never get
    a report.)

*   Wired Internet connections are more common than they really are.
    (Because they’re being used temporarily during installation, while
    wi-fi isn’t working.)

*   Typical screen resolution is lower than it really is. (Because
    people don’t tweak the resolution until after installation. And
    because if they fail to do so, they might uninstall Ubuntu,
    resulting in a report for a system that soon stops existing.)

*   Bluetooth devices are much less common than they really are.

I think it would be much more interesting to measure these things month
by month.

>                                                         The file
> containing this data would be available for the user to inspect.
>
> That data would include:
>    * Ubuntu Flavour
>    * Ubuntu Version
>    * Network connectivity or not

If I understand “on first boot once there is a network connection”, that
would exclude devices that were offline until the second startup or later.

>    * CPU family
>    * RAM
>    * Disk(s) size
>    * Screen(s) resolution
>    * GPU vendor and model
>    * OEM Manufacturer
>    * Location (based on the location selection made by the user at
> install).  No IP information would be gathered
>    * Installation duration (time taken)
>    * Auto login enabled or not
>    * Disk layout selected
>    * Third party software selected or not
>    * Download updates during install or not
>    * LivePatch enabled or not
>
> * Popcon would be installed.  This will allow us to spot trends in
> package usage and help us to  focus on the packages which are of most
> value to our users.

This effectively singles out .deb package installation as the only thing
that should be reported periodically, with everything else reported
one-off. Is that just for ease of implementation, or is there a reason
not to report the other things periodically too?

For example, if we could see how often people change their reported
location, we’d have info on how accessible the time zone UI should be.
And if it turns out that only a tiny fraction of Livepatch users turn it
on during install, vs. afterwards, that would influence future installer
design.

>…
> Any user can simply opt out by unchecking the box, which triggers one
> simple POST stating, “diagnostics=false”.

What is the purpose of this?

>                                           There will be a
> corresponding checkbox in the Privacy panel of GNOME Settings to
> toggle the state of this.
>…

This checkbox was implemented in Ubuntu 13.10 and later. I’ve just
tweaked the design to update the examples of metrics collected.
<https://wiki.ubuntu.com/ErrorTracker?action=diff&rev2=186&rev1=185>

Juerg Haefliger wrote on 15/02/18 07:10:
>…
> Please make this an opt-in rather than an opt-out. This just smells
> like a trend towards a Windows/Android installation where you have to
> unset gazillions of check boxes to prevent the machine from posting
> your life to the vendor. We shouldn't go there.

The diagnostics checkbox was introduced in System Settings five years
ago. So if this “smells like a trend”, it’s a glacially slow trend.

One characteristic of “big data” is that users often can’t be expected
to foresee the kinds of ways the data can be combined in future. So if
there is a privacy problem, making collection opt-in won’t necessarily
solve it.

On the other hand, making it opt-in would give us results that were just
as useful as making it opt-out, unless the resulting sample was (a) too
small to be useful or (b) too biased in some way.

Cheers
--
mpt

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Will Cooke-2
On 19 February 2018 at 13:55, Matthew Paul Thomas <[hidden email]> wrote:
Will Cooke wrote on 14/02/18 15:22:
>…
> We want to be able to focus our engineering efforts on the things that
> matter most to our users, and in order to do that we need to get some
> more data about sort of setups our users have and which software they
> are running on it.
>
> We would like to add a checkbox to the installer, exact wording TBD,
> but along the lines of “Send diagnostics information to help improve
> Ubuntu”.  This would be checked by default.

I’ve just drafted a design for this. <https://goo.gl/yJ86Qa> It’s
basically a subset of the System Settings screen.


Thanks, we will take a look at that, but on first scan it looks good.
 
> The result of having that box checked would be:
>
> * Information from the installation would be sent over HTTPS to a
> service run by Canonical’s IS team.  This would be saved to disk and
> sent on first boot once there is a network connection.

Information from the installation would be fascinating, for improvement
of the installer in particular. However, I don’t think it would give you
an accurate idea about the “sort of setups our users have”, for
improvement of Ubuntu in general. It could lead you to think that, for
example:

*   Internet connection is less common than it really is. (Because of
    things like proxies, as mentioned by Ernst Sjöstrand, or
    not-yet-installed wi-fi drivers. And because if people still can’t
    get online later, they might uninstall Ubuntu and you’ll never get
    a report.)

*   Wired Internet connections are more common than they really are.
    (Because they’re being used temporarily during installation, while
    wi-fi isn’t working.)

*   Typical screen resolution is lower than it really is. (Because
    people don’t tweak the resolution until after installation. And
    because if they fail to do so, they might uninstall Ubuntu,
    resulting in a report for a system that soon stops existing.)

*   Bluetooth devices are much less common than they really are.

I think it would be much more interesting to measure these things month
by month.


Yes, general trending is what we're aiming for here.  Thanks for the input on those specifics.

 
>                                                         The file
> containing this data would be available for the user to inspect.
>
> That data would include:
>    * Ubuntu Flavour
>    * Ubuntu Version
>    * Network connectivity or not

If I understand “on first boot once there is a network connection”, that
would exclude devices that were offline until the second startup or later.



We were thinking along the lines of something which would try to send the data at login a number of times, let's say.... 10, and then give up.  So if the machine never comes on line, then the data never gets sent.  If the machine travels between various locations before arriving at a working internet connection, then it should eventually be able to send it.  I think that would cover the vast majority of cases.

 
>    * CPU family
>    * RAM
>    * Disk(s) size
>    * Screen(s) resolution
>    * GPU vendor and model
>    * OEM Manufacturer
>    * Location (based on the location selection made by the user at
> install).  No IP information would be gathered
>    * Installation duration (time taken)
>    * Auto login enabled or not
>    * Disk layout selected
>    * Third party software selected or not
>    * Download updates during install or not
>    * LivePatch enabled or not
>
> * Popcon would be installed.  This will allow us to spot trends in
> package usage and help us to  focus on the packages which are of most
> value to our users.

This effectively singles out .deb package installation as the only thing
that should be reported periodically, with everything else reported
one-off. Is that just for ease of implementation, or is there a reason
not to report the other things periodically too?

Mainly ease of implementation.

 

For example, if we could see how often people change their reported
location, we’d have info on how accessible the time zone UI should be.
And if it turns out that only a tiny fraction of Livepatch users turn it
on during install, vs. afterwards, that would influence future installer
design.

Wouldn't that involve us being able to track a person/machine to know when it had been changed?  Or would something in the location picker send a signal?  I don't like either of these options, I've probably misunderstood the idea.

 

>…
> Any user can simply opt out by unchecking the box, which triggers one
> simple POST stating, “diagnostics=false”.

What is the purpose of this?


To try and measure engagement rates.  This would be important in the "opt-in" case I think, how representative of users is the data?  < 10% of people are submitting data, then probably not very useful.

Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel


--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Robie Basak-4
On Tue, Feb 20, 2018 at 11:10:36AM +0000, Will Cooke wrote:
> We were thinking along the lines of something which would try to send the
> data at login a number of times, let's say.... 10, and then give up.

Note that if the user has defined a wifi connection that isn't available
to all users, then I think the connection can't be made until the user
has logged on, and that might take a few seconds to set up. So you'll
want to try some time after logging in, rather than immediately on
login, to avoid racing it.

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: More diagnostics data from desktop

Matthew Paul Thomas
In reply to this post by Will Cooke-2
Will Cooke wrote on 20/02/18 11:10:
>…
> We were thinking along the lines of something which would try to send
> the data at login a number of times, let's say.... 10, and then give
> up.  So if the machine never comes on line, then the data never gets
> sent.  If the machine travels between various locations before
> arriving at a working internet connection, then it should eventually
> be able to send it.  I think that would cover the vast majority of
> cases.

That seems reasonable.

>…
>> For example, if we could see how often people change their reported
>> location, we’d have info on how accessible the time zone UI should
>> be. And if it turns out that only a tiny fraction of Livepatch users
>> turn it on during install, vs. afterwards, that would influence
>> future installer design.
>
> Wouldn't that involve us being able to track a person/machine to know
> when it had been changed?  Or would something in the location picker
> send a signal?  I don't like either of these options, I've probably
> misunderstood the idea.

Neither, I think. The location example would require Ubuntu to count how
often the setting had changed (for example, “2 changes in the past
month”). And the Livepatch example would require Livepatch to record
whether it had been configured in the installer or System Settings.

>>> Any user can simply opt out by unchecking the box, which triggers
>>> one simple POST stating, “diagnostics=false”.
>>
>> What is the purpose of this?
>
> To try and measure engagement rates.  This would be important in the
> "opt-in" case I think, how representative of users is the data?  < 10%
> of people are submitting data, then probably not very useful.
>…

A 10% response rate would be necessary, as far as I can tell, only if
there were fewer than 3460 Ubuntu desktop users in the world. Assuming
that you’re going for a 5% margin of error with 95% confidence level.

If there were, say, 4 million Ubuntu desktop users in the world, you’d
need only 385 submissions to reach that level of confidence. Even for a
more precise 2% margin of error, you’d need only 2400 submissions — that
is, you’d need only a 0.06% response rate. (This is why someone polling
a state/country, which has a million voters, doesn’t need 100 000
responses. Often they collect just 1000.)

As long as the number of Ubuntu desktop users is anything more than half
a million, the response rate is basically irrelevant: the required
sample size stays almost constant. For example, to get that 2% margin of
error from 500 000 Ubuntu users would require 2390 submissions, while
from 100 million Ubuntu users it would require 2401 submissions.

Now, I’m not a statistician, so maybe I’ve made a silly miscalculation
or misunderstanding. If you were planning to do any sub-sample analysis,
or reweighting for known biases, then the original sample would need to
be bigger. But if you were proposing that this be opt-out merely because
you thought we’d need a ≥10% response rate — or if you were proposing
“diagnostics=false” because you thought we’d need to measure the
response rate at all — then I’d strongly encourage consulting a
statistician first.

Cheers
--
mpt

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
12