Shipped Clean, Broke Later: Pick Your Language, Pick Your Debt

The Headline Everyone Missed

If you only read the headline numbers from the latest arXiv white paper on AI-generated code, you would think the story is simple: AI code looks cleaner than human code. Fewer static analysis alerts, lower defect indicators, everything trending in the right direction.

The paper in question is “A Large-Scale Empirical Study of AI-Generated Code in Real-World Repositories” by Mao et al., published in March 2026.

Look one layer deeper and a different story appears. Those same AI-involved commits need more follow-up fixes, take longer to settle down, and behave very differently depending on which language the AI was asked to write. On the surface, things ship clean. Later, they break.

Fewer Alerts, More Incidents

Mao et al. found 10.04 CodeQL alerts per thousand lines of AI code versus 13.56 for human code in their dataset of real GitHub repositories (paper). That sounds like a win, until you account for the selection effect. This measures code that survived developer review. Developers routinely discard the worst AI output before it ever commits. What the study captured is curated AI code, not raw model output.

Lab studies that skip curation tell a very different story. The FormAI‑v2 line of work generated 331,000 C programs directly from LLMs and found at least 62% were vulnerable, using formal verification to label issues. See “Do Neutral Prompts Produce Insecure Code? FormAI-v2 Dataset” and the FormAI dataset repository. The gap between roughly ten alerts per thousand lines of code and 62% of programs being vulnerable is essentially the value of human review. Remove that step, which is exactly what vibe coding does, and the safety floor collapses.

Google’s DORA 2025 report quantifies the stability cost at the delivery level. According to the Accelerate State of DevOps Report 2025, AI tooling pushed PR throughput up 98%, but incidents per PR increased 242.7%. The probability of a production incident more than tripled per merged change.

Mao et al. also examine commit-level behavior in the same GitHub dataset. AI‑involved commits are more likely to be modified or fixed within 7, 30, and 90 days than human‑only commits, which reinforces the “shipped clean, broke later” pattern (paper). GitClear’s analysis of hundreds of millions of changed lines found code churn rising from 3.3% in 2021 to 5.7% in 2024, reaching 7.1% in a 2026 follow‑up. See “Top 8 Cited AI Developer Productivity & Code Quality Research” and “The Maintainability Gap: AI Code Quality in 2026”. Part of the reason is how AI handles errors. GitClear reported a sharp increase in obfuscation patterns like catch blocks and safe‑navigation operators that suppress failures rather than fix them. The code passes review, then fails in production in ways that are hard to trace.

Pick Your Language, Pick Your Debt

A large‑scale analysis of 7,703 GitHub files explicitly attributed to ChatGPT, Copilot, CodeWhisperer, and Tabnine shows how much the programming language matters. Schreiber and Tippe’s “Security Vulnerabilities in AI-Generated Code: A Large-Scale Analysis of Public GitHub Repositories” used CodeQL to scan these files and reported the following vulnerability rates:

Language	Vulnerability Rate
TypeScript	2.50% to 7.14%
JavaScript	8.66% to 8.99%
Python	16.18% to 18.50%

From their analysis, programming language characteristics drive vulnerability patterns more strongly than the specific AI tool used. Switching models barely moves the needle. Switching languages does.

TypeScript wins because of structure. Static type systems catch bugs at compile time. Gao, Bird and Barr’s study “To Type or Not to Type: Quantifying Detectable Bugs in JavaScript” found that TypeScript and Flow each detect roughly 15% of bugs that had already survived human review, which is a conservative lower bound on the real benefit. For AI‑generated code that may misuse types or pass wrong arguments, a compiler that catches this immediately is a meaningful safety net.

Python is the paradox. It is the most represented language in LLM training data, the most popular in AI‑assisted codebases at roughly 38% of analyzed files in Schreiber & Tippe’s dataset (paper), and the most dangerous to vibe‑code in. The reason is not what the LLM knows. It is what Python’s runtime allows. Dynamic typing means type errors only surface at runtime. Insecure patterns like eval, exec, and subprocess(shell=True) compile cleanly and run without complaint. The top CWEs in Python AI code (SQL injection, command injection, code injection, hard‑coded credentials) are all things the runtime will happily execute without flagging anything.

Python supports type hints, but they are not enforced at runtime. A function annotated def process(x: str) will still accept any type at execution. Unless mypy or a similar checker runs as a hard CI gate, the annotations are documentation, not defense.

Fixing Python: The Two-Layer Stack

Python can get closer to TypeScript‑level safety, but only if you build it intentionally, and AI will not do it for you.

Layer 1: Static checkers (before runtime)

Tool	Best For
mypy (`--strict`)	Broad compatibility, industry standard
Pyright (Microsoft)	Much faster, stricter inference, VS Code
basedpyright	TypeScript‑level strictness
Ruff	Fast linting in CI

The mypy documentation and Pyright / basedpyright comparison all make the same point implicitly: these tools only help if you actually fail the build when they complain. A warning that does not fail the pipeline is decoration.

Layer 2: Runtime enforcers (at execution)

Pydantic validates data at runtime at API and input boundaries. It is effectively mandatory in any AI‑assisted Python web backend and directly blocks the injection‑class CWEs that dominate Python AI code. See the Pydantic documentation.
beartype injects type assertions at import time with near‑zero overhead. You can apply it surgically to high‑risk functions. See the beartype project page.
typeguard and its pytest plugin turn your existing test suite into a runtime type audit during CI, with no production overhead. See the typeguard project.

Used together, they close the gap between what mypy promises and what Python actually enforces at runtime.

The Review Gate Is Closing

Everything above assumes a human reviewed the AI code before it merged. That assumption is breaking down. Multiple industry studies now find roughly one‑third of PRs merging without any human review, and AI‑generated PRs waiting several times longer in queues than human‑written ones. The 2025 DORA report and follow‑up analyses such as “DORA 2025: AI Raised Throughput 98%, Tripled Incidents” highlight that AI amplifies existing team habits. Good review culture gets better. Weak review culture gets dangerous.

The numbers on unreviewed AI code are stark. Cloud Security Alliance’s research note “Vibe Coding Security Crisis: Credential Sprawl and SDLC Debt” notes that AI‑assisted commits expose secrets at roughly double the rate of human‑only commits. By March 2026, at least 74 CVEs had been formally traced to AI coding tools, with researchers and industry analysts estimating the true count is several times higher due to missing commit metadata; see CSA briefings and coverage such as “Vibe Coding Security: Why 62% of AI-Generated Code Ships With Vulnerabilities”.

The Bottom Line

AI code often looks better at the moment you merge it and worse in the weeks that follow. It raises throughput, but it raises incident rates too. It writes more code, but it also leaves more work behind.

If you take nothing else from the data, take this. Language choice is not just a stylistic preference in the age of AI. TypeScript gives you a compiler that pushes back when the model gets things wrong. Python gives you incredible speed and flexibility, along with a runtime that will usually let questionable code through and let you find out later.

If you are going to lean on AI in Python, you should strongly consider a two‑layer stack of static and runtime checks. If you are choosing languages for new AI‑heavy systems, TypeScript’s guardrails are not just a developer experience win; they demonstrably reduce bugs and lower vulnerability rates, so they belong in your security design.

Ship fast if you want to. Just make sure you understand what you are shipping into, and what that code is likely to owe you after it lands.