[PATCH] dragonfly: Fix legendre symbol calculation failure handling

Chaitanya Tata chaitanya.mgit at gmail.com
Mon Nov 6 09:19:15 PST 2023


In case of low-memory conditions, the computation for legendre symbol
can fail and return -2 as per documentation, but the check for that
was missed here. And this can can cause an infinite loop searching for
qr and qnr.

Break the loop if calculation fails, we can leave retry to the callers
or user.

Signed-off-by: Chaitanya Tata <Chaitanya.Tata at nordicsemi.no>
---
 src/common/dragonfly.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/src/common/dragonfly.c b/src/common/dragonfly.c
index 1e8427166..a0befe537 100644
--- a/src/common/dragonfly.c
+++ b/src/common/dragonfly.c
@@ -71,8 +71,11 @@ int dragonfly_get_random_qr_qnr(const struct crypto_bignum *prime,
 			*qr = tmp;
 		else if (res == -1 && !(*qnr))
 			*qnr = tmp;
-		else
+		else {
 			crypto_bignum_deinit(tmp, 0);
+			if (res == -2)
+				break;
+		}
 	}
 
 	if (*qr && *qnr)
-- 
2.25.1




More information about the Hostap mailing list