[kteam-tools][PATCH 0/5] implement snap-related tasks handling

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[kteam-tools][PATCH 0/5] implement snap-related tasks handling

Kleber Souza
Snap-related tasks have been added to Kernel SRU Workflow project to
keep track of the steps needed to get the builds tested and published on
snapcraft.io store.

These patches implement/fix the addition of these tasks to the relevant
tracking bugs and their state transitions done by swm.

Kleber Sacilotto de Souza (5):
  ktl: assign snap-certification-testing task to hw-cert team
  ktl/kernel-series-info: add more snap info and document it
  ktl/tracking_bug.py: fix snap tasks lookup
  wfl: add a helper class for snap store
  wfl: implement snap tasks handlers

 ktl/kernel-series-info.yaml   |  82 +++++++++++
 ktl/tracking_bug.py           |  34 ++++-
 ktl/workflow.py               |   4 +-
 stable/wfl/snap.py            | 119 ++++++++++++++++
 stable/wfl/wft/kernel_snap.py | 324 +++++++++++++++++++++++++++++++++++++++---
 5 files changed, 537 insertions(+), 26 deletions(-)
 create mode 100644 stable/wfl/snap.py

--
2.14.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[kteam-tools][PATCH 1/5] ktl: assign snap-certification-testing task to hw-cert team

Kleber Souza
Signed-off-by: Kleber Sacilotto de Souza <[hidden email]>
---
 ktl/workflow.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ktl/workflow.py b/ktl/workflow.py
index ed79443f..e361b5ac 100644
--- a/ktl/workflow.py
+++ b/ktl/workflow.py
@@ -128,7 +128,7 @@ class Workflow:
                 'snap-release-to-stable'     : 'canonical-kernel-team',
                 'snap-qa-testing'            : 'canonical-kernel-team',
                 'snap-publish'               : 'canonical-kernel-team',
-                'snap-certification-testing' : 'canonical-kernel-team',
+                'snap-certification-testing' : 'canonical-hw-cert',
                 'prepare-package'            : 'canonical-kernel-team',
                 'prepare-package-lbm'        : 'canonical-kernel-team',
                 'prepare-package-lrm'        : 'canonical-kernel-team',
@@ -210,7 +210,7 @@ class Workflow:
                 'snap-release-to-stable'     : 'canonical-kernel-team',
                 'snap-qa-testing'            : 'canonical-kernel-team',
                 'snap-publish'               : 'canonical-kernel-team',
-                'snap-certification-testing' : 'canonical-kernel-team',
+                'snap-certification-testing' : 'canonical-hw-cert',
                 'prepare-package'            : 'canonical-kernel-team',
                 'prepare-package-lbm'        : 'canonical-kernel-team',
                 'prepare-package-lrm'        : 'canonical-kernel-team',
--
2.14.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[kteam-tools][PATCH 2/5] ktl/kernel-series-info: add more snap info and document it

Kleber Souza
In reply to this post by Kleber Souza
Add some additional fields for the snaps to support handling the
tracking bugs tasks and document it.

Signed-off-by: Kleber Sacilotto de Souza <[hidden email]>
---
 ktl/kernel-series-info.yaml | 82 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/ktl/kernel-series-info.yaml b/ktl/kernel-series-info.yaml
index c6f9872b..19360c10 100644
--- a/ktl/kernel-series-info.yaml
+++ b/ktl/kernel-series-info.yaml
@@ -123,6 +123,51 @@
     #           oneiric backport, the list means, we are backporting from
     #           'linux' package present on '11.10'.
     #
+    # * dependent-snaps (dictionary)
+    #       Dictionary used to list the packages which should have snap related
+    #       tasks added to the same tracking bug used for the debian packages.
+    #       This information will be used to create and process the tracking
+    #       bugs.
+    #
+    #       Format:
+    #           'dependent-snaps': {
+    #             <source package name A>: {
+    #               'snap': <snap name>
+    #               'gated': (bool)
+    #               'stable': (bool)
+    #               'arches': [<arch 1>[,arch2...]]
+    #             }
+    #             <source package name B>: ...
+    #           }
+    #
+    #           Fields:
+    #             - gated:   whether the snap is gated by another snap. This controls if the
+    #                        snap will be refreshed automatically by the users once it
+    #                        hits the stable channel. (bool)
+    #             - stable:  whether the snap should be published to the stable channel. (bool)
+    #             - qa:      whether the snap needs QA testing (snap-qa-testing task). (bool)
+    #             - hw-cert: whether the snap needs HW certification tests
+    #                        (snap-certification-testing). (bool)
+    #             - arches:  list of architectures which the snap is built for.
+    #
+    #       Example:
+    #           dependent-snaps:
+    #               linux:
+    #                   snap: pc-kernel
+    #                   gated: false
+    #                   stable: true
+    #                   qa: true
+    #                   hw-cert: true
+    #                   arches:
+    #                   - amd64
+    #                   - i386
+    #
+    # * derivative-snaps (dictionary)
+    #       Similar to 'dependent-snaps', but used to list the snaps which should have
+    #       separate tracking bugs from the master debian package. This is useful when
+    #       the same debian package will be used to generate several debian packages
+    #       and need a tracking bug for each of the snaps.
+    #
     '18.04':
         dependent-packages:
             linux:
@@ -282,16 +327,53 @@
         dependent-snaps:
             linux:
                 snap: pc-kernel
+                gated: false
+                stable: true
+                qa: false
+                hw-cert: true
+                arches:
+                - amd64
+                - i386
             linux-raspi2:
                 snap: pi2-kernel
+                gated: false
+                stable: true
+                qa: false
+                hw-cert: true
+                arches:
+                - armhf
             linux-snapdragon:
                 snap: dragonboard-kernel
+                gated: false
+                stable: true
+                qa: false
+                hw-cert: true
+                arches:
+                - arm64
             linux-aws:
                 snap: aws-kernel
+                gated: false
+                stable: false
+                qa: false
+                hw-cert: false
+                arches:
+                - amd64
             linux-gke:
                 snap: gke-kernel
+                gated: false
+                stable: false
+                qa: false
+                hw-cert: false
+                arches:
+                - amd64
             linux-gcp:
                 snap: gcp-kernel
+                gated: false
+                stable: false
+                qa: false
+                hw-cert: false
+                arches:
+                - amd64
         derivative-packages:
             linux:
             - linux-raspi2
--
2.14.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[kteam-tools][PATCH 3/5] ktl/tracking_bug.py: fix snap tasks lookup

Kleber Souza
In reply to this post by Kleber Souza
The layout of the snap-related information has been changed on
kernel-series-info.yaml. So change ktl/tracking_bug.py for the new
layout.

Also prevent creating the snap-publish, snap-qa-testing and
snap-certification-testing tasks depending on the snap properties on
kernel-series-info.yaml.

Signed-off-by: Kleber Sacilotto de Souza <[hidden email]>
---
 ktl/tracking_bug.py | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/ktl/tracking_bug.py b/ktl/tracking_bug.py
index b32637c1..5820e6ca 100644
--- a/ktl/tracking_bug.py
+++ b/ktl/tracking_bug.py
@@ -17,6 +17,7 @@ class TrackingBug:
         self.wf = Workflow()
         self.ub = Ubuntu()
         self.__dependency_list = None
+        self.__dependency_snap_list = None
         self._targeted_series_name = None
         self._series_specified = None
         self._series_target = None
@@ -41,6 +42,20 @@ class TrackingBug:
         retval = dependent_package in self.__dependency_list
         return retval
 
+    # _get_dependent_snap_prop
+    def _get_dependent_snap_prop(self, series, main_package, prop):
+        '''
+        Return the value of the property of the dependent snap for the
+        specified series/package. Return None if not found.
+        '''
+        if self.__dependency_snap_list is None:
+            try:
+                record = self.ub.lookup(series)
+                self.__dependency_snap_list = record['dependent-snaps'][main_package]
+            except KeyError:
+                self.__dependency_snap_list = {}
+        return self.__dependency_snap_list.get(prop, None)
+
     # valid_series
     #
     def valid_series(s, lp_series, series_specified, targeted_series_name, package):
@@ -65,7 +80,7 @@ class TrackingBug:
             if lp_series.name == 'prepare-package-signed' and not s.has_dependent_package(targeted_series_name, package, 'signed'):
                 cdebug('    no prepare-package-signed', 'yellow')
                 break
-            if not s.has_dependent_package(targeted_series_name, package, 'snap'):
+            if s._get_dependent_snap_prop(targeted_series_name, package, 'snap') is None:
                 if lp_series.name == 'snap-release-to-edge':
                     cdebug('    no snap-release-to-edge', 'yellow')
                     break
@@ -87,6 +102,22 @@ class TrackingBug:
                 if lp_series.name == 'snap-certification-testing':
                     cdebug('    no snap-certification-testing', 'yellow')
                     break
+            else:
+                if lp_series.name == 'snap-certification-testing':
+                    hw_cert = s._get_dependent_snap_prop(targeted_series_name, package, 'hw-cert')
+                    if hw_cert is None or not hw_cert:
+                        cdebug('    no snap-certification-testing', 'yellow')
+                        break
+                if lp_series.name == 'snap-qa-testing':
+                    qa = s._get_dependent_snap_prop(targeted_series_name, package, 'qa')
+                    if qa is None or not qa:
+                        cdebug('    no snap-qa-testing', 'yellow')
+                        break
+                if lp_series.name == 'snap-publish':
+                    gated = s._get_dependent_snap_prop(targeted_series_name, package, 'gated')
+                    if gated is None or not gated:
+                        cdebug('    no snap-publish', 'yellow')
+                        break
             retval = True
             break
         return retval
@@ -310,6 +341,7 @@ class TrackingBug:
         cdebug('    series_specified: %s' % series_specified)
 
         self.__dependency_list = None
+        self.__dependency_snap_list = None
 
         self.package = package
         self.version = version
--
2.14.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[kteam-tools][PATCH 4/5] wfl: add a helper class for snap store

Kleber Souza
In reply to this post by Kleber Souza
Add a helper class to handle the snap store operations. For now it loads
the versioning info for a snap associated with a tracking bug on first
use and provides a method to check if the kernel package version matches
the snap version for a given channel.

Signed-off-by: Kleber Sacilotto de Souza <[hidden email]>
---
 stable/wfl/snap.py | 119 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 119 insertions(+)
 create mode 100644 stable/wfl/snap.py

diff --git a/stable/wfl/snap.py b/stable/wfl/snap.py
new file mode 100644
index 00000000..7afcbf76
--- /dev/null
+++ b/stable/wfl/snap.py
@@ -0,0 +1,119 @@
+#!/usr/bin/env python
+
+try:
+    from urllib.request import urlopen, Request
+    from urllib.parse import urlencode, urljoin
+    from urllib.error import URLError, HTTPError
+except ImportError:
+    from urllib2 import urlopen, urlencode, Request, URLError, HTTPError
+
+import json
+from .errors import ShankError
+
+
+# SnapStoreError
+#
+class SnapStoreError(ShankError):
+    """
+    Thrown when something goes wrong with the snap store (e.g. snap not found).
+    """
+    def __init__(s, emsg):
+        super(SnapStoreError, s).__init__(emsg)
+
+    def __str__(s):
+        return s.message
+
+
+# SnapStore
+#
+class SnapStore:
+    """
+    A helper class to handle Snapcraft store operations.
+    """
+    base_url = "https://search.apps.ubuntu.com/api/v1/snaps/details/"
+    common_headers = {'X-Ubuntu-Series': '16'}
+
+    # __init__
+    #
+    def __init__(s, bug):
+        """
+        :param bug: WorkflowBug object
+        """
+        s.bug = bug
+        s._dependent_snap = None
+        s._snap_store_versions = {}  # dictionary with {<channel>: {<arch>: <version>[, ...]}[, ...]}
+
+    # _get_snap_info
+    #
+    def _get_snap_info(s, channel):
+        """
+        Query the snap store URL to get the information about the kernel snap
+        on the provided channel and store it in the private variables.
+
+        :return: nothing
+        """
+        if s._dependent_snap is None:
+            try:
+                record = s.bug.ubuntu.lookup(s.bug.series)
+                s._dependent_snap = record['dependent-snaps'][s.bug.pkg_name]
+            except KeyError:
+                raise SnapStoreError('Could not find snap for series/package \'%s/%s\'' %
+                                     (s.bug.series, s.bug.pkg_name))
+
+        s._snap_store_versions[channel] = {}
+        for arch in s._dependent_snap['arches']:
+            try:
+                headers = s.common_headers
+                headers['X-Ubuntu-Architecture'] = arch
+                params = urlencode({'fields': 'version', 'channel': channel})
+                url = "%s?%s" % (urljoin(s.base_url, s._dependent_snap['snap']), params)
+                req = Request(url, headers=headers)
+                with urlopen(req) as resp:
+                    version = json.loads(resp.read())['version']
+                    s._snap_store_versions[channel][arch] = version
+            except HTTPError as e:
+                # Error 404 is returned if the snap has never been published
+                # to the given channel.
+                store_err = False
+                if hasattr(e, 'code') and e.code == 404:
+                    ret_body = e.read().decode()
+                    store_err_str = 'has no published revisions in the given context'
+                    if store_err_str in ret_body:
+                        store_err = True
+                        s._snap_store_versions[channel][arch] = None
+                if not store_err:
+                    raise SnapStoreError('failed to retrieve store URL (%s)' % str(e))
+            except (URLError, KeyError) as e:
+                raise SnapStoreError('failed to retrieve store URL (%s: %s)' %
+                                     (type(e), str(e)))
+
+    # match_version
+    #
+    def match_version(s, channel):
+        """
+        Check if the snap version on the store for the given channel corresponds
+        to the kernel version and abi numbers of the kernel package.
+
+        :param channel: store channel name
+        :return: True if the snap version for all arches on the given channel match the
+        kernel version and ABI from the tracking bug, False otherwise
+        """
+
+        try:
+            s._get_snap_info(channel)
+        except:
+            raise
+
+        # Loop over the store version for all arches and check if they are consistent.
+        store_version = None
+        for arch in s._snap_store_versions[channel].keys():
+            if s._snap_store_versions[channel][arch] is None:
+                # Return False if there are no published version for one of the arches
+                return False
+            if store_version is None:
+                store_version = s._snap_store_versions[channel][arch]
+            elif store_version != s._snap_store_versions[channel][arch]:
+                return False
+
+        return (store_version is not None and
+                store_version.rpartition('.')[0] == "%s.%s" % (s.bug.kernel_version, s.bug.abi))
--
2.14.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[kteam-tools][PATCH 5/5] wfl: implement snap tasks handlers

Kleber Souza
In reply to this post by Kleber Souza
Signed-off-by: Kleber Sacilotto de Souza <[hidden email]>
---
 stable/wfl/wft/kernel_snap.py | 324 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 301 insertions(+), 23 deletions(-)

diff --git a/stable/wfl/wft/kernel_snap.py b/stable/wfl/wft/kernel_snap.py
index 0fbbdda8..3f7ff7c5 100644
--- a/stable/wfl/wft/kernel_snap.py
+++ b/stable/wfl/wft/kernel_snap.py
@@ -1,7 +1,9 @@
 
-from wfl.log                                    import center, cleave
+from wfl.log                                    import center, cleave, cinfo, cerror
+from wfl.snap                                   import SnapStore, SnapStoreError
 from .base                                      import TaskHandler
 
+
 class KernelSnapBase(TaskHandler):
     '''
     '''
@@ -12,33 +14,25 @@ class KernelSnapBase(TaskHandler):
         center(s.__class__.__name__ + '.__init__')
         super(KernelSnapBase, s).__init__(lp, task, bug)
 
-        # The tracking bug should start this task out as 'Confirmed'.
-        #
-        s.jumper['New']           = s._new
-        s.jumper['Confirmed']     = s._common
-        s.jumper['Triaged']       = s._common
-        s.jumper['In Progress']   = s._common
-        s.jumper['Fix Committed'] = s._common
+        s.snap_store = SnapStore(bug)
+        s._snap_info = None
 
         cleave(s.__class__.__name__ + '.__init__')
 
-    # _new
-    #
-    def _new(s):
-        center(s.__class__.__name__ + '._new')
-        retval = False
-        cleave(s.__class__.__name__ + '._new (%s)' % (retval))
-        return retval
-
-    # _common
-    #
-    def _common(s):
+    @property
+    def snap_info(s):
         '''
+        Return a dictionary with the snap info from kernel-series-info.yaml.
+        If not found, an empty dictionary is returned.
         '''
-        center(s.__class__.__name__ + '._common')
-        retval = False
-        cleave(s.__class__.__name__ + '._common (%s)' % (retval))
-        return retval
+        if s._snap_info is None:
+            try:
+                record = s.bug.ubuntu.lookup(s.bug.series)
+                s._snap_info = record['dependent-snaps'][s.bug.pkg_name]
+            except KeyError:
+                s._snap_info = {}
+
+        return s._snap_info
 
 
 class SnapReleaseToEdge(KernelSnapBase):
@@ -50,8 +44,52 @@ class SnapReleaseToEdge(KernelSnapBase):
     def __init__(s, lp, task, bug):
         center(s.__class__.__name__ + '.__init__')
         super(SnapReleaseToEdge, s).__init__(lp, task, bug)
+
+        s.jumper['New']           = s._new
+        s.jumper['Confirmed']     = s._verify_release
+        s.jumper['Triaged']       = s._verify_release
+        s.jumper['In Progress']   = s._verify_release
+        s.jumper['Fix Committed'] = s._verify_release
+
         cleave(s.__class__.__name__ + '.__init__')
 
+    # _new
+    #
+    def _new(s):
+        center(s.__class__.__name__ + '._new')
+        retval = False
+
+        # The snap should be released to edge and beta channels after
+        # the package hits -proposed.
+        if s.bug.tasks_by_name['promote-to-proposed'].status == 'Fix Released':
+            s.task.status = 'Confirmed'
+            s.task.timestamp('started')
+            retval = True
+        else:
+            cinfo('    task promote-to-proposed is not \'Fix Released\'', 'yellow')
+
+        cleave(s.__class__.__name__ + '._new (%s)' % (retval))
+        return retval
+
+    # _verify_release
+    #
+    def _verify_release(s):
+        center(s.__class__.__name__ + '._verify_release')
+        retval = False
+
+        try:
+            if s.snap_store.match_version('edge'):
+                s.task.status = 'Fix Released'
+                s.task.timestamp('finished')
+                retval = True
+            else:
+                cinfo('    snap not in edge channel', 'yellow')
+        except SnapStoreError as e:
+            cerror('    failed to query snap store (%s)' % str(e))
+
+        cleave(s.__class__.__name__ + '._verify_release (%s)' % (retval))
+        return retval
+
 
 class SnapReleaseToBeta(KernelSnapBase):
     '''
@@ -62,8 +100,52 @@ class SnapReleaseToBeta(KernelSnapBase):
     def __init__(s, lp, task, bug):
         center(s.__class__.__name__ + '.__init__')
         super(SnapReleaseToBeta, s).__init__(lp, task, bug)
+
+        s.jumper['New']           = s._new
+        s.jumper['Confirmed']     = s._verify_release
+        s.jumper['Triaged']       = s._verify_release
+        s.jumper['In Progress']   = s._verify_release
+        s.jumper['Fix Committed'] = s._verify_release
+
         cleave(s.__class__.__name__ + '.__init__')
 
+    # _new
+    #
+    def _new(s):
+        center(s.__class__.__name__ + '._new')
+        retval = False
+
+        # The snap should be released to edge and beta channels after
+        # the package hits -proposed.
+        if s.bug.tasks_by_name['promote-to-proposed'].status == 'Fix Released':
+            s.task.status = 'Confirmed'
+            s.task.timestamp('started')
+            retval = True
+        else:
+            cinfo('    task promote-to-proposed is not \'Fix Released\'', 'yellow')
+
+        cleave(s.__class__.__name__ + '._new (%s)' % (retval))
+        return retval
+
+    # _verify_release
+    #
+    def _verify_release(s):
+        center(s.__class__.__name__ + '._verify_release')
+        retval = False
+
+        try:
+            if s.snap_store.match_version('beta'):
+                s.task.status = 'Fix Released'
+                s.task.timestamp('finished')
+                retval = True
+            else:
+                cinfo('    snap not in beta channel', 'yellow')
+        except SnapStoreError as e:
+            cerror('    failed to query snap store (%s)' % str(e))
+
+        cleave(s.__class__.__name__ + '._verify_release (%s)' % (retval))
+        return retval
+
 
 class SnapReleaseToCandidate(KernelSnapBase):
     '''
@@ -74,8 +156,61 @@ class SnapReleaseToCandidate(KernelSnapBase):
     def __init__(s, lp, task, bug):
         center(s.__class__.__name__ + '.__init__')
         super(SnapReleaseToCandidate, s).__init__(lp, task, bug)
+
+        s.jumper['New']           = s._new
+        s.jumper['Confirmed']     = s._verify_release
+        s.jumper['Triaged']       = s._verify_release
+        s.jumper['In Progress']   = s._verify_release
+        s.jumper['Fix Committed'] = s._verify_release
+
         cleave(s.__class__.__name__ + '.__init__')
 
+    # _new
+    #
+    def _new(s):
+        center(s.__class__.__name__ + '._new')
+        retval = False
+
+        # The snap is released to candidate channel after it's on beta channel
+        # and passes HW certification tests (or the task is set to invalid).
+        while not retval:
+            if s.bug.tasks_by_name['snap-release-to-beta'].status != 'Fix Released':
+                cinfo('    task snap-release-to-beta is not \'Fix Released\'', 'yellow')
+                break
+
+            if (s.bug.tasks_by_name.get('snap-certification-testing', None) is not None
+                    and s.bug.tasks_by_name['snap-certification-testing'].status not in ['Fix Released', 'Invalid']):
+                cinfo('    task snap-certification-testing is neither \'Fix Released\' nor \'Invalid\'', 'yellow')
+                break
+
+            s.task.status = 'Confirmed'
+            s.task.timestamp('started')
+
+            retval = True
+            break
+
+        cleave(s.__class__.__name__ + '._new (%s)' % (retval))
+        return retval
+
+    # _verify_release
+    #
+    def _verify_release(s):
+        center(s.__class__.__name__ + '._verify_release')
+        retval = False
+
+        try:
+            if s.snap_store.match_version('candidate'):
+                s.task.status = 'Fix Released'
+                s.task.timestamp('finished')
+                retval = True
+            else:
+                cinfo('    snap not in candidate channel', 'yellow')
+        except SnapStoreError as e:
+            cerror('    failed to query snap store (%s)' % str(e))
+
+        cleave(s.__class__.__name__ + '._verify_release (%s)' % (retval))
+        return retval
+
 
 class SnapReleaseToStable(KernelSnapBase):
     '''
@@ -86,8 +221,79 @@ class SnapReleaseToStable(KernelSnapBase):
     def __init__(s, lp, task, bug):
         center(s.__class__.__name__ + '.__init__')
         super(SnapReleaseToStable, s).__init__(lp, task, bug)
+
+        s.jumper['New']           = s._new
+        s.jumper['Confirmed']     = s._verify_release
+        s.jumper['Triaged']       = s._verify_release
+        s.jumper['In Progress']   = s._verify_release
+        s.jumper['Fix Committed'] = s._verify_release
+
         cleave(s.__class__.__name__ + '.__init__')
 
+    # _new
+    #
+    def _new(s):
+        center(s.__class__.__name__ + '._new')
+        retval = False
+
+        # Set the task to invalid if 'stable' is not set on kernel-series-info.yaml
+        stable = s.snap_info.get('stable', None)
+        if stable is None or not stable:
+            cinfo('    not a stable snap', 'yellow')
+            s.task.status = 'Invalid'
+            retval = True
+
+        # The snap is released to stable channel after it's on candidate channel,
+        # passes QA tests (or the task is set to invalid) and the deb is promoted
+        # to -updates or -security.
+        while not retval:
+            if s.bug.tasks_by_name['snap-release-to-candidate'].status != 'Fix Released':
+                cinfo('    task snap-release-to-candidate is not \'Fix Released\'', 'yellow')
+                break
+
+            if (s.bug.tasks_by_name.get('snap-qa-testing', None) is not None
+                    and s.bug.tasks_by_name['snap-qa-testing'].status not in ['Fix Released', 'Invalid']):
+                cinfo('    task snap-qa-testing is neither \'Fix Released\' nor \'Invalid\'', 'yellow')
+                break
+
+            if s.bug.tasks_by_name['promote-to-updates'].status not in ['Fix Released', 'Invalid']:
+                cinfo('    task promote-to-updates is neither \'Fix Released\' nor \'Invalid\'', 'yellow')
+                break
+
+            if (s.bug.tasks_by_name['promote-to-updates'].status == 'Invalid'
+                    and s.bug.tasks_by_name['promote-to-security'].status not in ['Fix Released', 'Invalid']):
+                cinfo('    task promote-to-updates is \'Invalid\' and promote-to-security is neither \'Fix Released\''
+                      ' nor \'Invalid\'', 'yellow')
+                break
+
+            s.task.status = 'Confirmed'
+            s.task.timestamp('started')
+
+            retval = True
+            break
+
+        cleave(s.__class__.__name__ + '._new (%s)' % (retval))
+        return retval
+
+    # _verify_release
+    #
+    def _verify_release(s):
+        center(s.__class__.__name__ + '._verify_release')
+        retval = False
+
+        try:
+            if s.snap_store.match_version('stable'):
+                s.task.status = 'Fix Released'
+                s.task.timestamp('finished')
+                retval = True
+            else:
+                cinfo('    snap not in stable channel', 'yellow')
+        except SnapStoreError as e:
+            cerror('    failed to query snap store (%s)' % str(e))
+
+        cleave(s.__class__.__name__ + '._verify_release (%s)' % (retval))
+        return retval
+
 
 class SnapQaTesting(KernelSnapBase):
     '''
@@ -98,8 +304,27 @@ class SnapQaTesting(KernelSnapBase):
     def __init__(s, lp, task, bug):
         center(s.__class__.__name__ + '.__init__')
         super(SnapQaTesting, s).__init__(lp, task, bug)
+
+        s.jumper['New'] = s._new
+
         cleave(s.__class__.__name__ + '.__init__')
 
+    def _new(s):
+        center(s.__class__.__name__ + '._new')
+        retval = False
+
+        # We only care about setting the task to 'Confirmed' when the
+        # snap is published to the candidate channel.
+        if s.bug.tasks_by_name['snap-release-to-candidate'].status == 'Fix Released':
+            s.task.status = 'Confirmed'
+            s.task.timestamp('started')
+            retval = True
+        else:
+            cinfo('    task snap-release-to-candidate is not \'Fix Released\'', 'yellow')
+
+        cleave(s.__class__.__name__ + '._new (%s)' % (retval))
+        return retval
+
 
 class SnapCertificationTesting(KernelSnapBase):
     '''
@@ -110,8 +335,27 @@ class SnapCertificationTesting(KernelSnapBase):
     def __init__(s, lp, task, bug):
         center(s.__class__.__name__ + '.__init__')
         super(SnapCertificationTesting, s).__init__(lp, task, bug)
+
+        s.jumper['New'] = s._new
+
         cleave(s.__class__.__name__ + '.__init__')
 
+    def _new(s):
+        center(s.__class__.__name__ + '._new')
+        retval = False
+
+        # We only care about setting the task to 'Confirmed' when the
+        # snap is published to the beta channel.
+        if s.bug.tasks_by_name['snap-release-to-beta'].status == 'Fix Released':
+            s.task.status = 'Confirmed'
+            s.task.timestamp('started')
+            retval = True
+        else:
+            cinfo('    task snap-release-to-beta is not \'Fix Released\'', 'yellow')
+
+        cleave(s.__class__.__name__ + '._new (%s)' % (retval))
+        return retval
+
 
 class SnapPublish(KernelSnapBase):
         '''
@@ -122,6 +366,40 @@ class SnapPublish(KernelSnapBase):
         def __init__(s, lp, task, bug):
             center(s.__class__.__name__ + '.__init__')
             super(SnapPublish, s).__init__(lp, task, bug)
+
+            s.jumper['New']           = s._new
+            s.jumper['Confirmed']     = s._verify_release
+            s.jumper['Triaged']       = s._verify_release
+            s.jumper['In Progress']   = s._verify_release
+            s.jumper['Fix Committed'] = s._verify_release
+
             cleave(s.__class__.__name__ + '.__init__')
 
+        # _new
+        #
+        def _new(s):
+            center(s.__class__.__name__ + '._new')
+            retval = False
+
+            # If the snap has update control set up, the original publisher of the snap
+            # needs to validate the new snap after it hits the stable channel.
+            if s.bug.tasks_by_name['snap-release-to-stable'].status == 'Fix Released':
+                s.task.status = 'Confirmed'
+                s.task.timestamp('started')
+                retval = True
+            else:
+                cinfo('    task snap-release-to-stable is not \'Fix Released\'', 'yellow')
+
+            cleave(s.__class__.__name__ + '._new (%s)' % (retval))
+            return retval
+
+        # _verify_release
+        #
+        def _verify_release(s):
+            center(s.__class__.__name__ + '._verify_release')
+            retval = False
+            # TODO: check if the snap has been un-gated
+            cleave(s.__class__.__name__ + '._verify_release (%s)' % (retval))
+            return retval
+
 # vi: set ts=4 sw=4 expandtab syntax=python
--
2.14.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

Re: [kteam-tools][PATCH 2/5] ktl/kernel-series-info: add more snap info and document it

Andy Whitcroft-3
In reply to this post by Kleber Souza
On Fri, Nov 10, 2017 at 09:18:14AM +0100, Kleber Sacilotto de Souza wrote:

> Add some additional fields for the snaps to support handling the
> tracking bugs tasks and document it.
>
> Signed-off-by: Kleber Sacilotto de Souza <[hidden email]>
> ---
>  ktl/kernel-series-info.yaml | 82 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 82 insertions(+)
>
> diff --git a/ktl/kernel-series-info.yaml b/ktl/kernel-series-info.yaml
> index c6f9872b..19360c10 100644
> --- a/ktl/kernel-series-info.yaml
> +++ b/ktl/kernel-series-info.yaml
> @@ -123,6 +123,51 @@
>      #           oneiric backport, the list means, we are backporting from
>      #           'linux' package present on '11.10'.
>      #
> +    # * dependent-snaps (dictionary)
> +    #       Dictionary used to list the packages which should have snap related
> +    #       tasks added to the same tracking bug used for the debian packages.
> +    #       This information will be used to create and process the tracking
> +    #       bugs.
> +    #
> +    #       Format:
> +    #           'dependent-snaps': {
> +    #             <source package name A>: {
> +    #               'snap': <snap name>
> +    #               'gated': (bool)
> +    #               'stable': (bool)
> +    #               'arches': [<arch 1>[,arch2...]]
> +    #             }
> +    #             <source package name B>: ...
> +    #           }
> +    #
> +    #           Fields:
> +    #             - gated:   whether the snap is gated by another snap. This controls if the
> +    #                        snap will be refreshed automatically by the users once it
> +    #                        hits the stable channel. (bool)
> +    #             - stable:  whether the snap should be published to the stable channel. (bool)
> +    #             - qa:      whether the snap needs QA testing (snap-qa-testing task). (bool)
> +    #             - hw-cert: whether the snap needs HW certification tests
> +    #                        (snap-certification-testing). (bool)
> +    #             - arches:  list of architectures which the snap is built for.
> +    #
> +    #       Example:
> +    #           dependent-snaps:
> +    #               linux:
> +    #                   snap: pc-kernel
> +    #                   gated: false
> +    #                   stable: true
> +    #                   qa: true
> +    #                   hw-cert: true
> +    #                   arches:
> +    #                   - amd64
> +    #                   - i386

This limits us to one single snap package per primary package.
However the contents seem like the sorts of things we would want.
As we are planning on rejigging this I will pull these fields into
that redesign and not worry overly at this point.

> +    #
> +    # * derivative-snaps (dictionary)
> +    #       Similar to 'dependent-snaps', but used to list the snaps which should have
> +    #       separate tracking bugs from the master debian package. This is useful when
> +    #       the same debian package will be used to generate several debian packages
> +    #       and need a tracking bug for each of the snaps.
> +    #
>      '18.04':
>          dependent-packages:
>              linux:
> @@ -282,16 +327,53 @@
>          dependent-snaps:
>              linux:
>                  snap: pc-kernel
> +                gated: false
> +                stable: true
> +                qa: false
> +                hw-cert: true
> +                arches:
> +                - amd64
> +                - i386
>              linux-raspi2:
>                  snap: pi2-kernel
> +                gated: false
> +                stable: true
> +                qa: false
> +                hw-cert: true
> +                arches:
> +                - armhf
>              linux-snapdragon:
>                  snap: dragonboard-kernel
> +                gated: false
> +                stable: true
> +                qa: false
> +                hw-cert: true
> +                arches:
> +                - arm64
>              linux-aws:
>                  snap: aws-kernel
> +                gated: false
> +                stable: false
> +                qa: false
> +                hw-cert: false
> +                arches:
> +                - amd64
>              linux-gke:
>                  snap: gke-kernel
> +                gated: false
> +                stable: false
> +                qa: false
> +                hw-cert: false
> +                arches:
> +                - amd64
>              linux-gcp:
>                  snap: gcp-kernel
> +                gated: false
> +                stable: false
> +                qa: false
> +                hw-cert: false
> +                arches:
> +                - amd64
>          derivative-packages:
>              linux:
>              - linux-raspi2
> --
> 2.14.1
>
>
> --
> kernel-team mailing list
> [hidden email]
> https://lists.ubuntu.com/mailman/listinfo/kernel-team

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

Re: [kteam-tools][PATCH 4/5] wfl: add a helper class for snap store

Andy Whitcroft-3
In reply to this post by Kleber Souza
On Fri, Nov 10, 2017 at 09:18:16AM +0100, Kleber Sacilotto de Souza wrote:
> Add a helper class to handle the snap store operations. For now it loads
> the versioning info for a snap associated with a tracking bug on first
> use and provides a method to check if the kernel package version matches
> the snap version for a given channel.
>
> Signed-off-by: Kleber Sacilotto de Souza <[hidden email]>
> ---
[...]
> +# SnapStore
> +#
> +class SnapStore:
> +    """
> +    A helper class to handle Snapcraft store operations.
> +    """
> +    base_url = "https://search.apps.ubuntu.com/api/v1/snaps/details/"
> +    common_headers = {'X-Ubuntu-Series': '16'}

I assume this should be recorded somewhere ?  Should it in fact be a key
on the dependent-snap record?  With BB opening I assume we will have an
18 in here soon?

-apw

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[APPLIED] [kteam-tools][PATCH 0/5] implement snap-related tasks handling

Andy Whitcroft-3
In reply to this post by Kleber Souza
Applied to kteam-tools and pushed to production.  Will monitor.

-apw

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team