Red-Teaming Cloud Infrastructure with Neo

Most AI security tooling shipped over the last year focuses on one of two workflows, code review at PR time or zero-day research in open-source software. Models in PR pipelines now flag insecure patterns at every commit and autonomous research runs have produced more zero-days across open-source projects than the patch teams behind them can realistically triage. We've been running Neo on both of those workflows at ProjectDiscovery for a while now, surfacing zero-days in production software and turning vendor patch advisories into deployable detections through patch-diff and reverse-engineering passes that hold up without much cleanup from the team afterwards.

Red-teaming is where we hadn't pushed Neo yet and where the autonomous offensive AI conversation has stayed quietest. Most tooling in the category stops at first finding, a long way short of carrying an engagement from external recon to documented business impact.

We ran a full engagement with Neo, ProjectDiscovery's AI security engineer against an AWS environment to see how much of that grind it could carry on its own. The setup mirrors what we see in real assessments. A self-hosted CI/CD server in a public subnet with overpermissive cloud credentials, secrets scattered across build configurations and parameter stores, internal services in a private subnet that treat network placement as the access control. From a single starting prompt, Neo moved from external recon to administrative access on the CI server, harvested instance role credentials, enumerated S3 and SSM, pivoted into the private subnet through the build agent, forged a JWT against the internal API and confirmed access to the production database. The chain ran end to end in roughly two and a half hours.

Once an agent can carry that chain end to end without a human driving each phase, the question stops being whether AI matches a senior pentester on a single finding and starts being whether annual pentests are defensible coverage for cloud infrastructure that ships changes every day.

The environment

The lab mirrors a pattern that comes up in most cloud assessment we've worked. A build server got granted broad cloud permissions for convenience, the team moved on and those permissions never got scoped back.

A self-hosted TeamCity instance sits in the public subnet, internet-accessible on port 8111. The EC2 instance carries an IAM role with read access to S3, SSM Parameter Store, EC2 describe and RDS describe. Standard scope for a deployment pipeline. In the private subnet behind it sits an internal REST API and a PostgreSQL database that holds the customer table. The only thing separating the public internet from that database is one server and its unpatched software.

The prompt

No step-by-step guidance and no hints about what to look for. Every decision after that point came from execution.

Neo task: https://neo.projectdiscovery.io/share/4d1b194e-4ca1-453a-ba3a-acddc3efc19b

Below are the complete steps Neo followed during this red-teaming exercise

Reconnaissance and initial access

The exercise began with a quick fingerprinting phase that took only a few minutes. The login page revealed the exact build version without requiring authentication: TeamCity 2023.11.2 (build 147486), which falls directly within the vulnerable range associated with CVE-2024-27198.

CVE-2024-27198 is a path traversal flaw in TeamCity's request routing that allows complete authentication bypass. It carries a CVSS of 9.8 and sits on the CISA Known Exploited Vulnerabilities catalog with a federal patch deadline of March 28, 2024. Ransomware operators used it against production deployments in the same year the patch shipped. The server we are looking at was still running the vulnerable version two years past that deadline.

The bypass routes requests through /hax?jsp=/app/rest/<endpoint>;.jsp, which makes TeamCity treat the request as if it originated from an authenticated session.

The server returns full server metadata to an unauthenticated request, no session cookie and no authorization header. Eighty-two seconds later, a persistent admin API token is minted on the built-in superadmin account.

Full administrative access to the CI/CD platform in under two minutes. The token persists across restarts.

What the build server was holding

Build servers need every secret the deployment pipeline uses, so those secrets live in the build configuration. With admin access to the REST API, all of them are readable in a single request per project.

From one unauthenticated HTTP request, the production database password, a base64-encoded ed25519 SSH private key, the JWT signing secret for the internal auth system and a live Stripe API key. All in plaintext in the build configuration.

Cloud pivot via IMDS

Admin access to TeamCity means the ability to trigger builds on the connected build agent, which runs on the same EC2 instance. Which means access to the EC2 Instance Metadata Service.

A build step queries IMDS and writes the response to the build log:

Temporary credentials for the CI/CD IAM role, valid for five more hours. IMDSv1 needs nothing beyond a standard HTTP GET, so any code running on the instance can retrieve those credentials, including build steps an attacker added during an active compromise.

With the credentials in hand, Neo enumerates and downloads three S3 buckets.

The database backup bucket holds a compressed SQL dump with 1,000 customer records. Names, emails, phone numbers, last-four SSN digits, last-four credit card digits, card brands, subscription tiers and monthly revenue figures.

The build artifacts bucket has a legacy config file with a migration note from 2022 that was never finished:

Static, long-lived IAM keys sitting in the bucket three and a half years after that comment was written.

SSM Parameter Store, accessible via the same IAM role, returns the rest of the production secrets:

By this point in the engagement, the JWT signing secret has shown up in four separate locations. TeamCity build parameters, SSM, S3 and the EC2 user-data bootstrap script. EC2 instance enumeration via the IAM role surfaces a second host in the private subnet running an internal API on port 5000.

Reaching the private network

The internal API has no public IP and no direct internet path. The route in is through the build agent, which has network access to the private subnet as part of normal operation. Neo creates a new build configuration with a script that forges a JWT using the signing secret pulled from SSM, then probes the internal service:

The forged token validates because it is signed with the right secret. Build log when the step runs:

847,295 customers confirmed in the production database. The /admin endpoint accepts any token with an Authorization header without checking JWT claims at all. It returns executive compensation data, production database connection details and the full customer count to any caller that can reach it from inside the VPC. The private subnet was the entire access control model for that endpoint and it stopped being meaningful the moment the build server got compromised.

The /api/v1/orders response is a raw database error and it reveals more than it should. The JWT sub claim is being concatenated into the SQL query without parameterization:

The error confirms the table name, the column names and the exact injection point. Because the JWT signing secret is known, the sub value is fully attacker-controlled, which makes this a working SQL injection rather than a theoretical one. Neo flagged it in the report and stopped short of running data extraction queries through it, which was the right call given the chain to the production database via the leaked credentials was already established.

The TeamCity debug SQL endpoint

While parsing TeamCity configuration files through the admin API, one property in internal.properties stands out:

This setting enables a REST endpoint that runs arbitrary SQL against TeamCity's internal HSQLDB database. Using the admin token from initial access:

All four user password hashes from TeamCity's internal database, retrieved through a debug feature left enabled in production. The bcrypt cost factor is $2a$07$, which was calibrated for 2009-era hardware and is straightforwardly crackable offline with a modern GPU.

Findings summary

The full engagement ran in roughly two and a half hours. Neo generated the report from evidence captured during execution, including HTTP traces, build log output and command results, with no manual compilation after the fact.

Why CI/CD servers are high-value targets

Build servers concentrate risk differently from other targets. They need broad cloud permissions to deploy infrastructure, so those permissions get granted and rarely revisited. They hold every secret the pipeline uses because there is no other way to automate deployments. They run arbitrary shell commands by design, which means code execution on the build agent is a feature rather than an anomaly and is therefore harder to flag as malicious. They are usually internet-facing because the team needed external webhook support and opening a port was easier than wiring up a VPN.

The consequence is that a single unpatched CI server, combined with a default IMDSv1 configuration and a CI/CD IAM role that was never scoped back, creates a path from the public internet to every system the build pipeline touches.

On IMDSv1: IMDSv2 has been available since 2019 and has been the default for new EC2 instances since 2024, but the change was not retroactive. Any existing instance still running IMDSv1 lets any code on it retrieve the IAM credentials. The fix is HttpTokens=required on the instance metadata options, one API call per instance with no downtime, but it requires knowing which instances are still exposed.

On S3 backup access: the backup bucket in this engagement was not public. Access required the IAM role, which looks correct in isolation. The issue is that the same role used to run builds also had read access to production database backups. Backups accumulate over time, carry historical data that predates current access controls and rarely sit under active monitoring. When a breach actually happens, the backup bucket is typically where the most complete exfiltration occurs.

On private subnet placement: the internal API was correctly placed in a non-routable subnet with no public IP and the /admin endpoint had no JWT claim validation because its designers assumed network isolation was the control. That assumption held until the build agent, which had legitimate access to the private subnet, became an execution proxy. Network segmentation slows lateral movement. It does not stop it when the pivot host is already inside the segment.

The Nuclei template for CVE-2024-27198 is open-source and the scan completes in under a second per host. If you are running self-hosted TeamCity, running it against your own infrastructure is the cheapest version of this exercise.

What this looked like compared to other tooling

A vulnerability scanner would have flagged CVE-2024-27198 on the first pass. Nuclei, which we maintain at ProjectDiscovery, has shipped a template for this CVE since March 2024 and that's the entry point Neo started from. The template tells you the server is exploitable. It does not tell you that the build agent's IAM role can read production database backups, that the JWT signing secret lives in four places or that the private subnet's /admin endpoint accepts any signed token. Scanners are designed to detect known patterns in known places, which is exactly their job description and not a flaw to hold against them. Findings two through eleven in the table above all live outside that design.

A traditional pentest would have produced a similar report with a similar level of detail, after two to four weeks of consultant time and at a cost of roughly $25-40K for an engagement of this scope. The pentester would also have been better than Neo at the parts of the engagement that demand creativity. Crafting bespoke phishing pretexts, deciding when to risk noisy enumeration, judging when a finding crosses from "interesting" to "client will actually care." Neo did not attempt those parts of the work. It executed the deterministic chain of credential reuse, cloud enumeration and pivot mechanics that a pentester also has to grind through, without the consulting hours behind it.

Other autonomous offensive tools run end-to-end engagements that on the surface look similar to what Neo did here. The architectural difference is that those tools are mostly stateless across runs. Neo carries persistent memory across runs, so the second engagement against this lab learns from the first, knows the build server's quirks, recognizes the project naming conventions and starts the recon phase already aware of which credentials tend to live in which locations. That compounding is more useful in practice than any single engagement's output.

What Neo did not do

The bcrypt password hashes from TeamCity's internal database were retrieved but not cracked. Neo flagged them as evidence and noted the cost factor was weak enough to crack offline. It did not run an offline cracking job and pursuing user takeover on a TeamCity admin account from there was not in scope for this run.

The SQL injection in the orders API was demonstrated as exploitable, with the error response confirming the injection point and the JWT signing secret giving full control of the input. Neo did not run extraction queries through it. The chain to the production database via the leaked credentials was already established, so this would have been a redundant path to the same data, but a real adversary with different goals would treat it differently.

Lateral movement to other VPCs, AWS accounts or Lambda functions was not pursued. The engagement was scoped to one VPC and stopped when the production database was accessible. A real adversary with persistent access would keep going. So would a longer Neo session with a wider scope.

The exact numbers in the customer table and the count of secrets in SSM reflect choices we made when seeding the environment. The shape of the kill chain, public CI server to IAM credentials to internal subnet to production database, is what we have observed repeatedly in real assessments and is the part the engagement was meant to demonstrate.

Closing

Discovery is the half of autonomous offensive AI that the benchmarks already cover. The harder and less-tested question is whether an agent can take a single finding and turn it into a documented answer to "how much of this company can I reach from one exposed port?" Neo identified CVE-2024-27198 without being told to look for it, recognized the IMDS pivot as the obvious next step the moment shell access landed, chained credential reuse through S3, SSM and the private subnet without being prompted at each phase and produced a report built from evidence captured live during execution rather than written up after the fact.

There are a few places this fits cleanly into a security team's workflow. Continuous testing of internet-facing CI/CD, bastion hosts and exposed admin panels between annual pentests, where the gap between assessments is usually the window an attacker actually uses. Validation that pentest findings have been fixed rather than carried forward unfixed to the next assessment, since running Neo against a single remediation takes minutes rather than booking another engagement. Pre-production red-teaming of new internet-facing services before they go live, while changes are still cheap to make.

Neo is in early access at neo.projectdiscovery.io