[F][G][OEM-5.6][Unstable][SRU][PATCH 0/1] Improve descriptions for XFAIL cases in kselftests/net/psock_snd

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[F][G][OEM-5.6][Unstable][SRU][PATCH 0/1] Improve descriptions for XFAIL cases in kselftests/net/psock_snd

Po-Hsu Lin (Sam)
BugLink: https://bugs.launchpad.net/bugs/1900088

== SRU Justification ==
The test output in psock_snd.sh is confusing sometimes.
Here is an output example:
  raw csum_off with bad offset (fails)
  ./psock_snd: write: Invalid argument

The first line is the test case description, and the second is the
test execution output.

It feels like this test has failed, but it has passed instead.
The "fails" here means this test is expected to fail. But you will
have to check the source code to know this fact.

== Fix ==
* 30ae801746ea selftests/net: improve descriptions for XFAIL cases in
psock_snd.sh

Change "fails" in the test case description to "expected to fail", so
that the test output can be more understandable.

This test case exists after Bionic, and it can be cherry-picked.

== Test ==
Patch tested by applying to the tree directly, the test description
will be changed as expected.

== Regression Potential ==
Low, minor changes for test case description only.

Po-Hsu Lin (1):
  selftests/net: improve descriptions for XFAIL cases in psock_snd.sh

 tools/testing/selftests/net/psock_snd.sh | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[F][G][OEM-5.6][Unstable][SRU][PATCH 1/1] selftests/net: improve descriptions for XFAIL cases in psock_snd.sh

Po-Hsu Lin (Sam)
BugLink: https://bugs.launchpad.net/bugs/1900088

Before changing this it's a bit confusing to read test output:
  raw csum_off with bad offset (fails)
  ./psock_snd: write: Invalid argument

Change "fails" in the test case description to "expected to fail", so
that the test output can be more understandable.

Signed-off-by: Po-Hsu Lin <[hidden email]>
Signed-off-by: David S. Miller <[hidden email]>
(cherry picked from commit 30ae801746ea0e099246318d373691a9e3238d40)
Signed-off-by: Po-Hsu Lin <[hidden email]>
---
 tools/testing/selftests/net/psock_snd.sh | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tools/testing/selftests/net/psock_snd.sh b/tools/testing/selftests/net/psock_snd.sh
index 6331d91..170be65 100755
--- a/tools/testing/selftests/net/psock_snd.sh
+++ b/tools/testing/selftests/net/psock_snd.sh
@@ -45,7 +45,7 @@ echo "raw vnet hdr"
 echo "raw csum_off"
 ./in_netns.sh ./psock_snd -v -c
 
-echo "raw csum_off with bad offset (fails)"
+echo "raw csum_off with bad offset (expected to fail)"
 (! ./in_netns.sh ./psock_snd -v -c -C)
 
 
@@ -57,7 +57,7 @@ echo "raw min size"
 echo "raw mtu size"
 ./in_netns.sh ./psock_snd -l "${mss}"
 
-echo "raw mtu size + 1 (fails)"
+echo "raw mtu size + 1 (expected to fail)"
 (! ./in_netns.sh ./psock_snd -l "${mss_exceeds}")
 
 # fails due to ARPHRD_ETHER check in packet_extra_vlan_len_allowed
@@ -65,19 +65,19 @@ echo "raw mtu size + 1 (fails)"
 # echo "raw vlan mtu size"
 # ./in_netns.sh ./psock_snd -V -l "${mss}"
 
-echo "raw vlan mtu size + 1 (fails)"
+echo "raw vlan mtu size + 1 (expected to fail)"
 (! ./in_netns.sh ./psock_snd -V -l "${mss_exceeds}")
 
 echo "dgram mtu size"
 ./in_netns.sh ./psock_snd -d -l "${mss}"
 
-echo "dgram mtu size + 1 (fails)"
+echo "dgram mtu size + 1 (expected to fail)"
 (! ./in_netns.sh ./psock_snd -d -l "${mss_exceeds}")
 
-echo "raw truncate hlen (fails: does not arrive)"
+echo "raw truncate hlen (expected to fail: does not arrive)"
 (! ./in_netns.sh ./psock_snd -t "$((${vnet_hlen} + ${eth_hlen}))")
 
-echo "raw truncate hlen - 1 (fails: EINVAL)"
+echo "raw truncate hlen - 1 (expected to fail: EINVAL)"
 (! ./in_netns.sh ./psock_snd -t "$((${vnet_hlen} + ${eth_hlen} - 1))")
 
 
@@ -86,13 +86,13 @@ echo "raw truncate hlen - 1 (fails: EINVAL)"
 echo "raw gso min size"
 ./in_netns.sh ./psock_snd -v -c -g -l "${mss_exceeds}"
 
-echo "raw gso min size - 1 (fails)"
+echo "raw gso min size - 1 (expected to fail)"
 (! ./in_netns.sh ./psock_snd -v -c -g -l "${mss}")
 
 echo "raw gso max size"
 ./in_netns.sh ./psock_snd -v -c -g -l "${max_mss}"
 
-echo "raw gso max size + 1 (fails)"
+echo "raw gso max size + 1 (expected to fail)"
 (! ./in_netns.sh ./psock_snd -v -c -g -l "${max_mss_exceeds}")
 
 echo "OK. All tests passed"
--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

ACK: [F][G][OEM-5.6][Unstable][SRU][PATCH 1/1] selftests/net: improve descriptions for XFAIL cases in psock_snd.sh

Stefan Bader-2
On 16.10.20 09:15, Po-Hsu Lin wrote:

> BugLink: https://bugs.launchpad.net/bugs/1900088
>
> Before changing this it's a bit confusing to read test output:
>   raw csum_off with bad offset (fails)
>   ./psock_snd: write: Invalid argument
>
> Change "fails" in the test case description to "expected to fail", so
> that the test output can be more understandable.
>
> Signed-off-by: Po-Hsu Lin <[hidden email]>
> Signed-off-by: David S. Miller <[hidden email]>
> (cherry picked from commit 30ae801746ea0e099246318d373691a9e3238d40)
> Signed-off-by: Po-Hsu Lin <[hidden email]>
Acked-by: Stefan Bader <[hidden email]>

> ---
>  tools/testing/selftests/net/psock_snd.sh | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/tools/testing/selftests/net/psock_snd.sh b/tools/testing/selftests/net/psock_snd.sh
> index 6331d91..170be65 100755
> --- a/tools/testing/selftests/net/psock_snd.sh
> +++ b/tools/testing/selftests/net/psock_snd.sh
> @@ -45,7 +45,7 @@ echo "raw vnet hdr"
>  echo "raw csum_off"
>  ./in_netns.sh ./psock_snd -v -c
>  
> -echo "raw csum_off with bad offset (fails)"
> +echo "raw csum_off with bad offset (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -v -c -C)
>  
>  
> @@ -57,7 +57,7 @@ echo "raw min size"
>  echo "raw mtu size"
>  ./in_netns.sh ./psock_snd -l "${mss}"
>  
> -echo "raw mtu size + 1 (fails)"
> +echo "raw mtu size + 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -l "${mss_exceeds}")
>  
>  # fails due to ARPHRD_ETHER check in packet_extra_vlan_len_allowed
> @@ -65,19 +65,19 @@ echo "raw mtu size + 1 (fails)"
>  # echo "raw vlan mtu size"
>  # ./in_netns.sh ./psock_snd -V -l "${mss}"
>  
> -echo "raw vlan mtu size + 1 (fails)"
> +echo "raw vlan mtu size + 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -V -l "${mss_exceeds}")
>  
>  echo "dgram mtu size"
>  ./in_netns.sh ./psock_snd -d -l "${mss}"
>  
> -echo "dgram mtu size + 1 (fails)"
> +echo "dgram mtu size + 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -d -l "${mss_exceeds}")
>  
> -echo "raw truncate hlen (fails: does not arrive)"
> +echo "raw truncate hlen (expected to fail: does not arrive)"
>  (! ./in_netns.sh ./psock_snd -t "$((${vnet_hlen} + ${eth_hlen}))")
>  
> -echo "raw truncate hlen - 1 (fails: EINVAL)"
> +echo "raw truncate hlen - 1 (expected to fail: EINVAL)"
>  (! ./in_netns.sh ./psock_snd -t "$((${vnet_hlen} + ${eth_hlen} - 1))")
>  
>  
> @@ -86,13 +86,13 @@ echo "raw truncate hlen - 1 (fails: EINVAL)"
>  echo "raw gso min size"
>  ./in_netns.sh ./psock_snd -v -c -g -l "${mss_exceeds}"
>  
> -echo "raw gso min size - 1 (fails)"
> +echo "raw gso min size - 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -v -c -g -l "${mss}")
>  
>  echo "raw gso max size"
>  ./in_netns.sh ./psock_snd -v -c -g -l "${max_mss}"
>  
> -echo "raw gso max size + 1 (fails)"
> +echo "raw gso max size + 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -v -c -g -l "${max_mss_exceeds}")
>  
>  echo "OK. All tests passed"
>


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

APPLIED [OEM-5.6] Re: [F][G][OEM-5.6][Unstable][SRU][PATCH 0/1] Improve descriptions for XFAIL cases in kselftests/net/psock_snd

Timo Aaltonen-6
In reply to this post by Po-Hsu Lin (Sam)
On 16.10.2020 10.15, Po-Hsu Lin wrote:

> BugLink: https://bugs.launchpad.net/bugs/1900088
>
> == SRU Justification ==
> The test output in psock_snd.sh is confusing sometimes.
> Here is an output example:
>    raw csum_off with bad offset (fails)
>    ./psock_snd: write: Invalid argument
>
> The first line is the test case description, and the second is the
> test execution output.
>
> It feels like this test has failed, but it has passed instead.
> The "fails" here means this test is expected to fail. But you will
> have to check the source code to know this fact.
>
> == Fix ==
> * 30ae801746ea selftests/net: improve descriptions for XFAIL cases in
> psock_snd.sh
>
> Change "fails" in the test case description to "expected to fail", so
> that the test output can be more understandable.
>
> This test case exists after Bionic, and it can be cherry-picked.
>
> == Test ==
> Patch tested by applying to the tree directly, the test description
> will be changed as expected.
>
> == Regression Potential ==
> Low, minor changes for test case description only.
>
> Po-Hsu Lin (1):
>    selftests/net: improve descriptions for XFAIL cases in psock_snd.sh
>
>   tools/testing/selftests/net/psock_snd.sh | 16 ++++++++--------
>   1 file changed, 8 insertions(+), 8 deletions(-)
>

applied to oem-5.6, thanks

--
t

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

ACK: [F][G][OEM-5.6][Unstable][SRU][PATCH 1/1] selftests/net: improve descriptions for XFAIL cases in psock_snd.sh

Kleber Souza
In reply to this post by Po-Hsu Lin (Sam)
On 16.10.20 09:15, Po-Hsu Lin wrote:

> BugLink: https://bugs.launchpad.net/bugs/1900088
>
> Before changing this it's a bit confusing to read test output:
>   raw csum_off with bad offset (fails)
>   ./psock_snd: write: Invalid argument
>
> Change "fails" in the test case description to "expected to fail", so
> that the test output can be more understandable.
>
> Signed-off-by: Po-Hsu Lin <[hidden email]>
> Signed-off-by: David S. Miller <[hidden email]>
> (cherry picked from commit 30ae801746ea0e099246318d373691a9e3238d40)
> Signed-off-by: Po-Hsu Lin <[hidden email]>

Acked-by: Kleber Sacilotto de Souza <[hidden email]>

Thanks for fixing it!

> ---
>  tools/testing/selftests/net/psock_snd.sh | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/tools/testing/selftests/net/psock_snd.sh b/tools/testing/selftests/net/psock_snd.sh
> index 6331d91..170be65 100755
> --- a/tools/testing/selftests/net/psock_snd.sh
> +++ b/tools/testing/selftests/net/psock_snd.sh
> @@ -45,7 +45,7 @@ echo "raw vnet hdr"
>  echo "raw csum_off"
>  ./in_netns.sh ./psock_snd -v -c
>  
> -echo "raw csum_off with bad offset (fails)"
> +echo "raw csum_off with bad offset (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -v -c -C)
>  
>  
> @@ -57,7 +57,7 @@ echo "raw min size"
>  echo "raw mtu size"
>  ./in_netns.sh ./psock_snd -l "${mss}"
>  
> -echo "raw mtu size + 1 (fails)"
> +echo "raw mtu size + 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -l "${mss_exceeds}")
>  
>  # fails due to ARPHRD_ETHER check in packet_extra_vlan_len_allowed
> @@ -65,19 +65,19 @@ echo "raw mtu size + 1 (fails)"
>  # echo "raw vlan mtu size"
>  # ./in_netns.sh ./psock_snd -V -l "${mss}"
>  
> -echo "raw vlan mtu size + 1 (fails)"
> +echo "raw vlan mtu size + 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -V -l "${mss_exceeds}")
>  
>  echo "dgram mtu size"
>  ./in_netns.sh ./psock_snd -d -l "${mss}"
>  
> -echo "dgram mtu size + 1 (fails)"
> +echo "dgram mtu size + 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -d -l "${mss_exceeds}")
>  
> -echo "raw truncate hlen (fails: does not arrive)"
> +echo "raw truncate hlen (expected to fail: does not arrive)"
>  (! ./in_netns.sh ./psock_snd -t "$((${vnet_hlen} + ${eth_hlen}))")
>  
> -echo "raw truncate hlen - 1 (fails: EINVAL)"
> +echo "raw truncate hlen - 1 (expected to fail: EINVAL)"
>  (! ./in_netns.sh ./psock_snd -t "$((${vnet_hlen} + ${eth_hlen} - 1))")
>  
>  
> @@ -86,13 +86,13 @@ echo "raw truncate hlen - 1 (fails: EINVAL)"
>  echo "raw gso min size"
>  ./in_netns.sh ./psock_snd -v -c -g -l "${mss_exceeds}"
>  
> -echo "raw gso min size - 1 (fails)"
> +echo "raw gso min size - 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -v -c -g -l "${mss}")
>  
>  echo "raw gso max size"
>  ./in_netns.sh ./psock_snd -v -c -g -l "${max_mss}"
>  
> -echo "raw gso max size + 1 (fails)"
> +echo "raw gso max size + 1 (expected to fail)"
>  (! ./in_netns.sh ./psock_snd -v -c -g -l "${max_mss_exceeds}")
>  
>  echo "OK. All tests passed"
>


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team