blktests failures with v6.11-rc1 kernel

[Nilay Shroff]

On 8/13/24 12:36, Yi Zhang wrote:
> On Sat, Aug 3, 2024 at 12:49 AM Nilay Shroff <nilay at linux.ibm.com> wrote:
> 
> There are no simultaneous tests during the CKI tests running.
> I reproduced the failure on that server and always can be reproduced
> within 5 times:
> # sh a.sh
> ==============================0
> nvme/052 (tr=loop) (Test file-ns creation/deletion under one subsystem) [passed]
>     runtime  21.496s  ...  21.398s
> ==============================1
> nvme/052 (tr=loop) (Test file-ns creation/deletion under one subsystem) [failed]
>     runtime  21.398s  ...  21.974s
>     --- tests/nvme/052.out 2024-08-10 00:30:06.989814226 -0400
>     +++ /root/blktests/results/nodev_tr_loop/nvme/052.out.bad
> 2024-08-13 02:53:51.635047928 -0400
>     @@ -1,2 +1,5 @@
>      Running nvme/052
>     +cat: /sys/block/nvme1n2/uuid: No such file or directory
>     +cat: /sys/block/nvme1n2/uuid: No such file or directory
>     +cat: /sys/block/nvme1n2/uuid: No such file or directory
>      Test complete
> # uname -r
> 6.11.0-rc3

We may need to debug this further. Is it possible to patch blktest and 
collect some details when this issue manifests? If yes then can you please
apply the below diff and re-run your test? This patch would capture output 
of "nvme list" and "sysfs attribute tree created under namespace head node"
and store those details in 052.full file. 

diff --git a/common/nvme b/common/nvme
index 9e78f3e..780b5e3 100644
--- a/common/nvme
+++ b/common/nvme
@@ -589,8 +589,23 @@ _find_nvme_ns() {
                if ! [[ "${ns}" =~ nvme[0-9]+n[0-9]+ ]]; then
                        continue
                fi
+               echo -e "\nBefore ${ns}/uuid check:\n" >> ${FULL}
+               echo -e "\n`nvme list -v`\n" >> ${FULL}
+               echo -e "\n`tree ${ns}`\n" >> ${FULL}
+
                [ -e "${ns}/uuid" ] || continue
                uuid=$(cat "${ns}/uuid")
+
+               if [ "$?" = "1" ]; then
+                       echo -e "\nFailed to read $ns/uuid\n" >> ${FULL}
+                       echo "`nvme list -v`" >> ${FULL}
+                       if [ -d "${ns}" ]; then
+                               echo -e "\n`tree ${ns}`\n" >> ${FULL}
+                       else
+                               echo -e "\n${ns} doesn't exist!\n" >> ${FULL}
+                       fi
+               fi
+
                if [[ "${subsys_uuid}" == "${uuid}" ]]; then
                        basename "${ns}"
                fi


After applying the above diff, when this issue occurs on your system copy this 
file "</path/to/blktests>/results/nodev_tr_loop/nvme/052.full" and send it across. 
This may give us some clue about what might be going wrong. 

Thanks,
--Nilay