AI code review (Claude, maybe Codex)

[Hauke Mehrtens]

Hi,

Here are some updates on this topic.

I do not plan to replace human reviewers with an AI. I think that would 
not work. The plan is to use AI in addition. I see AI as a tool humans 
use to work more efficiently. LLMs are currently good at finding general 
problems. They find buffer overflows, use-after-free bugs, and so on. In 
the way we use them, they are not good at finding architecture problems.

The LLM review bot should look for the common problems so the human only 
has to take care of the complex or uncommon ones. For me it is already 
helpful when the common problems are already fixed by the time I look at 
the PR.
When I find something in a PR it is more work for me compared to when it 
is totally fine. When I find something, I write a comment, the PR author 
fixes it, and I have to look at it again. When pull requests have fewer 
problems when I look at them, it is less work for me.

I also let it review and fix some of the OpenWrt components.
libubox: https://github.com/openwrt/libubox/pull/42/changes
ubus: https://github.com/openwrt/ubus/pull/20/changes

This works with the normal subscription and is relatively cheap. 
Scanning the code base, creating commits, and later reviewing the 
commits again costs probably less than €1, because the subscriptions are 
"heavily subsidized" compared to API usage.

I did a manual review of all the changes before creating the PRs.

I plan to run this on all OpenWrt components. I assume that this would 
have found at least 50% of the CVEs assigned to OpenWrt in the past.


I merged some fixes for the Claude Code PR review action:
https://github.com/openwrt/openwrt/pull/22897

It is now creating inline comments and should no longer pick up any 
extra commits. It should also be less verbose, but I am not sure if my 
prompting worked. Maybe I need help from a senior prompt engineer. ;-)

Here are some examples:
https://github.com/openwrt/openwrt/pull/22808#issuecomment-4236917721
https://github.com/openwrt/openwrt/pull/22791#issuecomment-4236886567

I applied for the open-source sponsorship at Anthropic (Claude Code) and 
OpenAI (Codex), but haven't heard back yet.

I also used the open-weights model gemma4:26b running locally in Ollama 
on my laptop (AMD 7840U with integrated GPU) to review an old PR by 
pasting it:
https://github.com/openwrt/mdnsd/pull/13
It ran for about 3 minutes and found the bug behind CVE-2026-30871. The 
big models are even better at this.

Hauke