Periodic Audits Can't Keep Up

At CodeLink, many of the systems we work on are complex and continuously evolving, with new features and integrations released regularly.

That creates a challenge for traditional security reviews. A full manual audit takes time to plan and execute, and by the time one cycle is complete, the system it assessed may already have changed.

For enterprises, this can affect release confidence, increase the risk of issues accumulating between audits, and put more pressure on already limited security capacity.

AWS Security Agent for Continuous Testing

AWS Security Agent (AWS Continuum) is supporting our teams to help close that gap by supporting security assessment throughout the software lifecycle, including design review, threat modeling, code review, and penetration testing.

We wanted to understand how useful that could be in practice, not only whether it could find issues, but whether it could make security review more scalable for a complex system.

So we tested it against one of our own environments.

Stage by Stage: What We Found

At the time of our test, our system was running in Oregon, where AWS Security Agent was not yet available. We had to wait for regional support before starting the assessment.

It was a simple blocker, but an important reminder that adopting new security tooling depends on more than capability alone. Infrastructure compatibility, deployment constraints, and setup effort all affect how quickly value can be realized.

Design Review

AWS Security Agent reviews architecture and design documentation against security requirements before implementation.

Limitation

Getting useful results required far more written context than we expected. Architecture diagrams, Mermaid, and PlantUML alone were not enough; we eventually had to describe the system in detailed plain text.

For a complex environment, assembling that documentation took significant effort and several iterations.

Evaluation

Once given sufficient context, the agent demonstrated a strong understanding of the architecture and identified relevant security concerns.

Where it struggled was organizational judgment. Some recommendations were technically sound but did not account for operational risks or internal policies.

For us, the value was therefore less about replacing design review and more about expanding its coverage while leaving business and risk decisions with the team.

Threat Modeling

AWS Security Agent maps components and trust boundaries to identify potential attack paths.

Limitation

Source code alone did not provide enough context across our multi-repository system. We also needed architecture documentation and supporting references before the threat model became coherent.

Runtime also increased as more repositories were added.

Evaluation

The resulting model was detailed and surfaced attack paths we had not explicitly documented ourselves.

That matters beyond the technical findings. In an enterprise-grade system, overlooked dependencies and trust boundaries can become sources of operational and security risk as the architecture evolves.

Automating more of that analysis can give teams broader visibility without rebuilding the threat model manually after every significant change.

Code Review

AWS Security Agent analyzes source code directly, either on demand or through pull-request reviews.

Limitation

Code does not show every control protecting an application. Infrastructure-level safeguards such as API Gateway or CloudFront policies sit outside the agent's view. Some of what it flagged as missing authentication could, in theory, already be covered one layer up, somewhere the agent couldn't see. So, findings still needed human validation against the wider environment.

Runtime also took around 2 hours per repo, which is fine for a single service but scales fast once several repos are in scope.

Evaluation

Of eight findings from one pass, seven were genuine gaps after we checked them against our infrastructure.

For us, that was one of the clearest demonstrations of value.

Issues that remain undetected can become progressively more expensive to fix as systems grow. Finding them closer to development gives teams a better chance to address them before they become production incidents, audit findings, or larger remediation projects.

Penetration Testing

AWS Security Agent runs attacks against application endpoints to test which weaknesses are actually exploitable.

Limitation

Our authentication setup required additional configuration, and AWS Security Agent's pen testing module is built around basic authentication. Rather than fighting that mismatch through configuration, we described our actual auth flow to the agent in plain language. It worked, but it's a step the product's own framing doesn't call out.

On top of that, every public endpoint had to be verified via DNS or HTTP before the agent would target it.

The full assessment also took approximately 45 hours, so this was not an instant process.

Evaluation

Across the run, AWS Security Agent surfaced 23 findings, including one critical and five high-severity issues, against a system our team had already reviewed and felt confident about.

It also detected a loophole we had deliberately introduced in staging.

Not every finding remained significant after human review, but genuine issues remained that our previous manual process had not identified.

For us, that was the strongest business case for the tool: it increased the depth and frequency of security assurance without requiring the same amount of manual discovery effort each time.

Putting It All Together

Across all four stages, a clear pattern emerged. AWS Security Agent increased the breadth of what we could assess, but it still required meaningful setup and human judgment.

The upfront effort was real: preparing architecture context, validating findings against infrastructure, configuring authentication, verifying endpoints, and allowing for runtime across multiple repositories.

But the alternative also has a cost.

As systems grow, periodic audits create longer windows in which new risks can accumulate. Expanding manual review capacity every time the system becomes more complex is rarely sustainable.

For organizations facing that problem, AWS Security Agent creates a different trade-off: invest more effort in preparing context and evaluating findings, while reducing the amount of manual effort required to repeatedly search the system for issues.

That can make security assurance easier to scale alongside software delivery.

What This Changes for Security Engineers

AWS Security Agent cannot determine whether every finding matters equally to the business.

A technically serious issue may have limited practical exposure. A recommended fix may create operational risk. Another finding may affect a critical workflow and require immediate action. Those decisions still require human context.

What changes is where security teams can spend their time.

Instead of using specialist capacity primarily to search large systems for potential weaknesses, teams can spend more of it on:

• validating material risks;

• prioritizing remediation;

• weighing security against operational impact;

• improving architecture and controls; and

• supporting better risk decisions.

Thinking of Trying This?

What we have covered is close to what we would discuss with a client during an initial scoping conversation, not only what AWS Security Agent can do, but also the preparation, constraints, and human oversight required to use it effectively.

Whether the investment makes sense depends on the system. For a small or slowly changing application, periodic assessment may still be sufficient. For a large system with frequent releases, multiple repositories, and growing security requirements, the economics can look very different.

For full technical requirements and current capabilities, AWS's own requirements are documented here.

As an AWS Select Tier Partner, CodeLink continues to deepen our AWS capabilities through hands-on delivery. Testing tools such as AWS Security Agent against our own systems helps us understand where they can create practical value for clients, and where engineering and security judgment still matter.

Discover some of our AWS delivery work below:

• Live Events Platform: Built a serverless AWS architecture to support real-time experiences and traffic spikes during major events.

• Enterprise Media Platform: Centralized media assets and automated processing with Amazon S3, AWS Lambda, and AWS Elemental MediaConvert.

• Cloud Modernization: Used AWS to support scalable modernization with secure access, resilience, and long-term maintainability.

Nolan

Senior DevOps Engineer

Nolan is an experienced DevOps Engineer with over four years of expertise working across various cloud platforms, including AWS, Azure, and GCP.