I recommend picking a random sample of commits to review more thoroughly by hand in parallel with the harness review.
That way, you get the opportunity to measure how many and what kind of issues your harness method misses, and with an estimation of their cost, you can make an informed decision about what magnitude speedup pays for the change in detection rate.
This looks great, I need to try it. Reviewing is such a bottleneck now and this may help.
I currently use a combination of a self-written git browser (in the command line, relying mostly on fzf) that allows me to easily navigate files and diffs across worktree, staged changes and earlier commits. When looking at the code isn't enough, I reach for vim with fugitive and lsp for inspection and poking at the code.
The key bindings seems very vim-inspired, so I should feel at home with it.
I have been looking for an ergonomic way to review generated code and format the comments as a prompt. I don't want to give the harness access to the real remote repository, and setting up a separate forge just for the review side seems excessive.
This, although vibe-coded, seems like good inspiration for a general concept that might work. Now if only I could take the time to make some Emacs commands that integrate this functionality with Magit and Ediff...
I’m wondering, is there anything about the interface or UX that “feels” vibe-coded to you? While it’s true that we’re relying heavily on agents to build this (i’m not a full stack person), we put a lot of effort into getting the details right. I personally also hate sloppy interfaces that all look alike and have weird paper cuts that agents don’t spot.
Your question seems sincere so I'll give it a sincere response.
The documentation and website are both obviously not written by a human.[1] This doesn't speak to me, in part because I'm the sort of person who thinks those are the interface and UX; but also because I reflexively end up thinking, "If they can't explain to me how it works, then why should I believe they even know how it works?"
I'm not saying you aren't intimate with every little detail, but writing the descriptions yourself is a costly signal and thus valuable for me as a potential user.
I don't see how I could keep up with the volume of code my team now produces without this. How do other people do it?
That way, you get the opportunity to measure how many and what kind of issues your harness method misses, and with an estimation of their cost, you can make an informed decision about what magnitude speedup pays for the change in detection rate.
I currently use a combination of a self-written git browser (in the command line, relying mostly on fzf) that allows me to easily navigate files and diffs across worktree, staged changes and earlier commits. When looking at the code isn't enough, I reach for vim with fugitive and lsp for inspection and poking at the code.
The key bindings seems very vim-inspired, so I should feel at home with it.
This, although vibe-coded, seems like good inspiration for a general concept that might work. Now if only I could take the time to make some Emacs commands that integrate this functionality with Magit and Ediff...
I’m wondering, is there anything about the interface or UX that “feels” vibe-coded to you? While it’s true that we’re relying heavily on agents to build this (i’m not a full stack person), we put a lot of effort into getting the details right. I personally also hate sloppy interfaces that all look alike and have weird paper cuts that agents don’t spot.
The documentation and website are both obviously not written by a human.[1] This doesn't speak to me, in part because I'm the sort of person who thinks those are the interface and UX; but also because I reflexively end up thinking, "If they can't explain to me how it works, then why should I believe they even know how it works?"
I'm not saying you aren't intimate with every little detail, but writing the descriptions yourself is a costly signal and thus valuable for me as a potential user.
[1]: https://xkqr.org/aicomment/#lang=c&prior=50&c=PZIxjtwwDEX7Pc...