business

Verdict

Submitted 7/25/2026, 4:03:19 AM · Completed 7/25/2026, 4:06:03 AM

6.5
pivot
The idea

Ask HN: How would you harden AI changes to a 1M-line legacy SaaS before review?

Show original source text →
I’m not a software engineer, but I’ve been running an experiment to see whether agentic development could produce a useful prototype on top of an existing SaaS codebase. The codebase is 1M+ lines, 15 years old, hosted on Azure, and primarily written in C# and React. The prototype needs to be available for customer testing in September. No developers were available to work on it full-time, although I could occasionally get help with specific technical issues. An engineer will evaluate the implementation in August and decide how much confidence we can have in the AI-generated code so that we can decide how to “convert” the code to production-grade. My question is what I can do before August, primarily using AI tools and manual testing, to make the code as robust and reviewable as possible. I want to increase the likelihood that the AI-generated code would be so good that the path to production-grade would be closer to “copy-paste” than building everything again from scratch. Environment The prototype is being developed in a separate branch, deployed to a separate internal environment, and connected to its own database and schema. Planning process We interviewed customers in June and turned the resuls into an MVP spec. The process was approximately: 1. Write a PRD. 2. Use an LLM to convert the PRD into an architecture document, which was reviewed by an architect. 3. Create product designs consisting of screen images and md files containing interaction details ans edge cases with Claude Design. 4. Use an agent to break the work into epics using the PRD and architecture document as guardrails. The epics were the most granular planning artifacts that received review by me and the architect. Development process The development flow was intended to run with little intervention: 1. A planner agent converted epics into md story files and Jira stories 2. A coding agent implemented the stories including tests and opened PRs 3. A reviewer agent reviewed the PRs, requested changes, and merged them into the prototype branch The coding agent polled PRs for review comments and could escalate issues back to the planner. Agents also had a “stop and ask” list for decisions they were not allowed to make autonomously. I (and a few times an engineer) were involved by resolving those escalations and by manually testing the accumulated changes end to end each day. Most implementation and initial review were done with Claude-based agents. For riskier PRs, I also used Codex as a reviewer. The second-model review found substantially more relevant issues in the Claude-generated code, but token quotas limited the use. I had separate refactoring and harden runs for the code as well. Results The planning and setting up the environment and agentic flow took about two weeks and then the agents built the whole MVP in about two weeks. Size-wise it was 13k lines of functional code + the same amount for tests. What I would like advice on Assuming that I cannot get substantial developer involvement before the review, how can I increase the likelihood that the code is as close to production-grade as possible? Here are some of the questions I have been thinking about: 1. What checks or development loops would give the largest improvement in confidence before an engineer reviews the code? 2. How would you use independent agents or models to reduce the risk that the coder and reviewer make the same incorrect assumptions? 3. Should tests be generated by a separate agent from the one that wrote the implementation? 4. What documentation or evidence would make the eventual engineering review faster and more reliable? 5. If you had only a few weeks to improve this prototype before handing it to an engineer, what would you prioritize?
TRIZ inventive level: 3/5· Principles: mechanical interaction, parameter changes
Synthesis verdict
**Pivot**: The agentic development approach shows promise, but significant risks and challenges need to be addressed to make it a viable money-making venture. The current prototype has been developed quickly, but its production readiness is uncertain. To improve its chances, focus on enhancing the testing and review process, generating high-quality documentation, and prioritizing independent test generation and static analysis.

Strengths

  • • Novel use of a multi-agent AI workflow on a legacy codebase
  • • Impressive development speed (13k lines of code + tests in 2 weeks)
  • • Clear guardrails and multi-model validation (Claude + Codex)

Weaknesses

  • • Unproven capability to deliver production-ready code for complex legacy integrations
  • • Insufficient human oversight and looming critical flaws
  • • Token quotas limiting multi-model review

Best angle

Refine the agentic development pipeline to prioritize independent test generation, static analysis, and exhaustive testing artifacts to bridge the gap from prototype to production-grade confidence.

Panel verdicts

Viability

meta/llama-4-maverick-17b-128e-instruct

7.0

“Using multiple, independent AI agents and models can help identify and mitigate errors and incorrect assumptions in the generated code.”

The idea of using agentic development to produce a prototype on top of an existing SaaS codebase is feasible, but its success heavily relies on the quality of the AI-generated code and the effectiveness of the manual testing and review process. The fact that the codebase is large and complex (1M+ lines, 15 years old) adds to the challenge. However, the team has made a good start by setting up an agentic flow, using multiple agents and models (Claude and Codex), and performing daily manual testing. To increase the likelihood of producing near-production-grade code, it's essential to focus on improving the testing and review process, using independent agents to reduce the risk of incorrect assumptions, and generating high-quality documentation. Given the limited timeframe (until August), prioritizing these activities is crucial. The key areas to focus on are: using separate agents for test generation, implementing additional review loops, and creating detailed documentation.

Competition

nvidia/nemotron-3-nano-omni-30b-a3b-reasoning

6.0

“Differentiation hinges on a purpose‑built, multi‑agent pipeline for legacy code rather than generic AI coding assistants.”

The concept leverages a multi‑agent AI workflow on a legacy 1M‑line Azure SaaS codebase, which is more novel than generic AI code assistants but faces competition from established AI pair‑programming tools (GitHub Copilot, Tabnine, Cursor), low‑code platforms (Bubble, OutSystems, Retool), and dedicated AI test/documentation generators (Testim, Sourcery). While the separation of planning, coding, and review agents and the use of independent environments add a layer of process differentiation, many of these capabilities are being incorporated into broader AI development suites, reducing the durability of a unique moat. The main defensible edge lies in the end‑to‑end agentic pipeline tailored to a large, older codebase, yet this niche may be replicable as the market matures.

Risk

nvidia/llama-3.3-nemotron-super-49b-v1

2.0

“Agentic development's unproven capability to deliver production-ready code for complex legacy integrations within tight timelines makes the project highly risky.”

The project's viability is severely threatened by its reliance on untested agentic development for a complex, legacy codebase integration, coupled with insufficient human oversight and a looming review that will likely uncover critical flaws. The 2-week MVP development by agents raises concerns about depth of testing, security, and compatibility with the 15-year-old C# and React base. Without substantial developer involvement, the gap to 'production-grade' is vast. Regulatory and platform risks are less immediate threats compared to the technical and quality control challenges.

Monetization

mistralai/mistral-medium-3.5-128b

8.0

“Multi-agent validation, static analysis, and exhaustive testing artifacts are the fastest way to bridge the gap from prototype to production-grade confidence.”

The experiment demonstrates a highly structured, agent-driven development pipeline with clear guardrails (PRD, architecture review, escalation paths) and multi-model validation (Claude + Codex). The 13k lines of code + tests in 2 weeks is impressive, but production-grade confidence hinges on rigorous validation. Prioritize: (1) **Independent test generation** by a separate agent (e.g., Codex) to catch implementation biases. (2) **Static analysis** (SonarQube, Roslyn analyzers) and **security scans** (OWASP ZAP) to flag anti-patterns. (3) **Chaos testing** (random inputs, edge cases) to stress-test robustness. (4) **Documentation artifacts**: auto-generated API docs (Swagger), decision logs for escalations, and traceability matrices linking PRD → epics → code. (5) **Performance baselines** (load tests, Azure Monitor metrics) to prove scalability. The dual-model review already mitigates single-agent blind spots; expand this to include a third model (e.g., Llama) for critical PRs. Token quotas are a bottleneck - prioritize high-risk modules for multi-model review. The 'copy-paste' path is plausible if the engineer sees: clean separation of concerns, 100% test coverage (including integration tests), and evidence of stress-testing.

Market

meta/llama-3.3-70b-instruct

This agent failed to return a verdict (gave up after 1 attempts: timeout (attempt 1)). The synthesis ran with the remaining agents.

Synthesized by meta/llama-4-maverick-17b-128e-instruct (fallback #1) · 2.4s

← Validate anotherPermalink: www.boss.cc/ideas/bw26Sk11pRU/