[B][SRU][PATCH 0/3] selftests: user: return Kselftest Skip code for skipped tests

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[B][SRU][PATCH 0/3] selftests: user: return Kselftest Skip code for skipped tests

Po-Hsu Lin (Sam)
BugLink: https://bugs.launchpad.net/bugs/1812198

== Justification ==
This user_copy test has failed while attempting to insert the
test_user_copy module, but this module does not exist in the Bionic KVM
kernel as the CONFIG_TEST_USER_COPY was not enabled.

# modprobe test_user_copy
modprobe: FATAL: Module test_user_copy not found in directory /lib/modules/4.15.0-1028-kvm

== Fix ==
d7d5311 (selftests: user: return Kselftest Skip code for skipped tests)
3c07aae (selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS)
c31d02d (selftests: kselftest: Remove outdated comment)

This fix will make the test return a skipped retun code if the
test_user_copy module does not exist.

This patch set can be cherry-picked into Bionic, and we already have it
in Cosmic / Disco.

For Xenial, it already got d7d5311 and does not need the other two.

== Test ==
With this patch, the test will return "skipped" as expected:
    $ sudo ./test_user_copy.sh
    user: module test_user_copy is not found [SKIP]
    $ echo $?
    4

Note that the test framework will need another fix to treat the SKIP
return code properly. (bug 1812352)

== Regression potential ==
None,
this patch just change the code for the testing tool.


Shuah Khan (Samsung OSG) (2):
  selftests: user: return Kselftest Skip code for skipped tests
  selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS

Thiago Jung Bauermann (1):
  selftests: kselftest: Remove outdated comment

 tools/testing/selftests/kselftest.h            | 3 +--
 tools/testing/selftests/user/test_user_copy.sh | 7 +++++++
 2 files changed, 8 insertions(+), 2 deletions(-)

--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[B][SRU][PATCH 1/3] selftests: user: return Kselftest Skip code for skipped tests

Po-Hsu Lin (Sam)
From: "Shuah Khan (Samsung OSG)" <[hidden email]>

BugLink: https://bugs.launchpad.net/bugs/1812198

When user test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as
a fail by the Kselftest framework. This leads to false negative result
even when the test could not be run.

Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run. Add an explicit check
for module presence and return skip code if module isn't present.

Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.

Signed-off-by: Shuah Khan (Samsung OSG) <[hidden email]>
(cherry picked from commit d7d5311d4aa9611fe1a5a851e6f75733237a668a)
Signed-off-by: Po-Hsu Lin <[hidden email]>
---
 tools/testing/selftests/user/test_user_copy.sh | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/user/test_user_copy.sh b/tools/testing/selftests/user/test_user_copy.sh
index d60506f..f9b31a5 100755
--- a/tools/testing/selftests/user/test_user_copy.sh
+++ b/tools/testing/selftests/user/test_user_copy.sh
@@ -2,6 +2,13 @@
 # SPDX-License-Identifier: GPL-2.0
 # Runs copy_to/from_user infrastructure using test_user_copy kernel module
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
+if ! /sbin/modprobe -q -n test_user_copy; then
+ echo "user: module test_user_copy is not found [SKIP]"
+ exit $ksft_skip
+fi
 if /sbin/modprobe -q test_user_copy; then
  /sbin/modprobe -q -r test_user_copy
  echo "user_copy: ok"
--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[B][SRU][PATCH 2/3] selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS

Po-Hsu Lin (Sam)
In reply to this post by Po-Hsu Lin (Sam)
From: "Shuah Khan (Samsung OSG)" <[hidden email]>

BugLink: https://bugs.launchpad.net/bugs/1812198

KSFT_SKIP points to KSFT_PASS resulting in reporting skipped tests as
Passed, when test programs exit with KSFT_SKIP or call ksft_exit_skip().
If tests are skipped because of unmet dependencies and/or unsupported
configuration, reporting them as passed leads to too many false positives.

Fix it to return a skip code of 4 to clearly differentiate the skipped
tests.

Signed-off-by: Shuah Khan (Samsung OSG) <[hidden email]>
(cherry picked from commit 3c07aaef65988473c6cea5bd194125f905953fcc)
Signed-off-by: Po-Hsu Lin <[hidden email]>
---
 tools/testing/selftests/kselftest.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h
index 1ae565e..974853e 100644
--- a/tools/testing/selftests/kselftest.h
+++ b/tools/testing/selftests/kselftest.h
@@ -20,7 +20,7 @@
 #define KSFT_XFAIL 2
 #define KSFT_XPASS 3
 /* Treat skip as pass */
-#define KSFT_SKIP  KSFT_PASS
+#define KSFT_SKIP  4
 
 /* counters */
 struct ksft_count {
--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[B][SRU][PATCH 3/3] selftests: kselftest: Remove outdated comment

Po-Hsu Lin (Sam)
In reply to this post by Po-Hsu Lin (Sam)
From: Thiago Jung Bauermann <[hidden email]>

BugLink: https://bugs.launchpad.net/bugs/1812198

Commit 3c07aaef6598 ("selftests: kselftest: change KSFT_SKIP=4 instead of
KSFT_PASS") reverted commit 11867a77eb85 ("selftests: kselftest framework:
change skip exit code to 0") but missed removing the comment which that
commit added, so do that now.

Signed-off-by: Thiago Jung Bauermann <[hidden email]>
Signed-off-by: Shuah Khan (Samsung OSG) <[hidden email]>
(cherry picked from commit c31d02d1290e1e82a08015199e408228e152991f)
Signed-off-by: Po-Hsu Lin <[hidden email]>
---
 tools/testing/selftests/kselftest.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h
index 974853e..e2714d0 100644
--- a/tools/testing/selftests/kselftest.h
+++ b/tools/testing/selftests/kselftest.h
@@ -19,7 +19,6 @@
 #define KSFT_FAIL  1
 #define KSFT_XFAIL 2
 #define KSFT_XPASS 3
-/* Treat skip as pass */
 #define KSFT_SKIP  4
 
 /* counters */
--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

ACK/Cmnt: [B][SRU][PATCH 0/3] selftests: user: return Kselftest Skip code for skipped tests

Stefan Bader-2
In reply to this post by Po-Hsu Lin (Sam)
On 18.01.19 10:11, Po-Hsu Lin wrote:

> BugLink: https://bugs.launchpad.net/bugs/1812198
>
> == Justification ==
> This user_copy test has failed while attempting to insert the
> test_user_copy module, but this module does not exist in the Bionic KVM
> kernel as the CONFIG_TEST_USER_COPY was not enabled.
>
> # modprobe test_user_copy
> modprobe: FATAL: Module test_user_copy not found in directory /lib/modules/4.15.0-1028-kvm
>
> == Fix ==
> d7d5311 (selftests: user: return Kselftest Skip code for skipped tests)
> 3c07aae (selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS)
> c31d02d (selftests: kselftest: Remove outdated comment)
>
> This fix will make the test return a skipped retun code if the
> test_user_copy module does not exist.
>
> This patch set can be cherry-picked into Bionic, and we already have it
> in Cosmic / Disco.
>
> For Xenial, it already got d7d5311 and does not need the other two.
>
> == Test ==
> With this patch, the test will return "skipped" as expected:
>     $ sudo ./test_user_copy.sh
>     user: module test_user_copy is not found [SKIP]
>     $ echo $?
>     4
>
> Note that the test framework will need another fix to treat the SKIP
> return code properly. (bug 1812352)
>
> == Regression potential ==
> None,
> this patch just change the code for the testing tool.
>
>
> Shuah Khan (Samsung OSG) (2):
>   selftests: user: return Kselftest Skip code for skipped tests
>   selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS
>
> Thiago Jung Bauermann (1):
>   selftests: kselftest: Remove outdated comment
>
>  tools/testing/selftests/kselftest.h            | 3 +--
>  tools/testing/selftests/user/test_user_copy.sh | 7 +++++++
>  2 files changed, 8 insertions(+), 2 deletions(-)
>
I accepted all nominations and set the status of xenial and cosmic to
fix-released according to you comments here. For Bionic:

Acked-by: Stefan Bader <[hidden email]>


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

ACK: [B][SRU][PATCH 0/3] selftests: user: return Kselftest Skip code for skipped tests

Kleber Souza
In reply to this post by Po-Hsu Lin (Sam)
On 1/18/19 10:11 AM, Po-Hsu Lin wrote:

> BugLink: https://bugs.launchpad.net/bugs/1812198
>
> == Justification ==
> This user_copy test has failed while attempting to insert the
> test_user_copy module, but this module does not exist in the Bionic KVM
> kernel as the CONFIG_TEST_USER_COPY was not enabled.
>
> # modprobe test_user_copy
> modprobe: FATAL: Module test_user_copy not found in directory /lib/modules/4.15.0-1028-kvm
>
> == Fix ==
> d7d5311 (selftests: user: return Kselftest Skip code for skipped tests)
> 3c07aae (selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS)
> c31d02d (selftests: kselftest: Remove outdated comment)
>
> This fix will make the test return a skipped retun code if the
> test_user_copy module does not exist.
>
> This patch set can be cherry-picked into Bionic, and we already have it
> in Cosmic / Disco.
>
> For Xenial, it already got d7d5311 and does not need the other two.
>
> == Test ==
> With this patch, the test will return "skipped" as expected:
>     $ sudo ./test_user_copy.sh
>     user: module test_user_copy is not found [SKIP]
>     $ echo $?
>     4
>
> Note that the test framework will need another fix to treat the SKIP
> return code properly. (bug 1812352)
>
> == Regression potential ==
> None,
> this patch just change the code for the testing tool.
>
>
> Shuah Khan (Samsung OSG) (2):
>   selftests: user: return Kselftest Skip code for skipped tests
>   selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS
>
> Thiago Jung Bauermann (1):
>   selftests: kselftest: Remove outdated comment
>
>  tools/testing/selftests/kselftest.h            | 3 +--
>  tools/testing/selftests/user/test_user_copy.sh | 7 +++++++
>  2 files changed, 8 insertions(+), 2 deletions(-)
>
Acked-by: Kleber Sacilotto de Souza <[hidden email]>


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

APPLIED: [B][SRU][PATCH 0/3] selftests: user: return Kselftest Skip code for skipped tests

Khaled Elmously
In reply to this post by Po-Hsu Lin (Sam)
On 2019-01-18 17:11:16 , Po-Hsu Lin wrote:

> BugLink: https://bugs.launchpad.net/bugs/1812198
>
> == Justification ==
> This user_copy test has failed while attempting to insert the
> test_user_copy module, but this module does not exist in the Bionic KVM
> kernel as the CONFIG_TEST_USER_COPY was not enabled.
>
> # modprobe test_user_copy
> modprobe: FATAL: Module test_user_copy not found in directory /lib/modules/4.15.0-1028-kvm
>
> == Fix ==
> d7d5311 (selftests: user: return Kselftest Skip code for skipped tests)
> 3c07aae (selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS)
> c31d02d (selftests: kselftest: Remove outdated comment)
>
> This fix will make the test return a skipped retun code if the
> test_user_copy module does not exist.
>
> This patch set can be cherry-picked into Bionic, and we already have it
> in Cosmic / Disco.
>
> For Xenial, it already got d7d5311 and does not need the other two.
>
> == Test ==
> With this patch, the test will return "skipped" as expected:
>     $ sudo ./test_user_copy.sh
>     user: module test_user_copy is not found [SKIP]
>     $ echo $?
>     4
>
> Note that the test framework will need another fix to treat the SKIP
> return code properly. (bug 1812352)
>
> == Regression potential ==
> None,
> this patch just change the code for the testing tool.
>
>
> Shuah Khan (Samsung OSG) (2):
>   selftests: user: return Kselftest Skip code for skipped tests
>   selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS
>
> Thiago Jung Bauermann (1):
>   selftests: kselftest: Remove outdated comment
>
>  tools/testing/selftests/kselftest.h            | 3 +--
>  tools/testing/selftests/user/test_user_copy.sh | 7 +++++++
>  2 files changed, 8 insertions(+), 2 deletions(-)
>
> --
> 2.7.4
>
>
> --
> kernel-team mailing list
> [hidden email]
> https://lists.ubuntu.com/mailman/listinfo/kernel-team

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team