Automated security scanners are genuinely useful. If you build software and aren’t using any, you probably should be. They’re fast, consistent, inexpensive and tireless — they’ll check every file on every change without getting bored.
But a clean scan is frequently mistaken for a clean bill of health. That’s a problem, because some of the most serious issues in real applications are exactly the ones automated tools are least able to see. This article explains why, in plain terms, so you can decide what level of assurance you actually need.
What scanners are good at
It helps to start with what automated tools do well, because the point isn’t that they’re bad — it’s that they answer particular questions.
- Known-vulnerable dependencies. Software composition analysis (SCA) tools compare the packages your application uses against public vulnerability databases, such as published CVEs. If you’re running a library version with a known flaw, they’ll usually tell you. This is valuable, because modern applications rely on large numbers of third-party packages.
- Recognizable code patterns. Static analysis (SAST) tools read source code looking for patterns associated with problems — building database queries by gluing strings together, for example, or using weak cryptographic functions.
- Committed secrets. Secret scanners look for API keys, tokens and passwords accidentally committed to a repository.
- Configuration and header checks. External scanners can spot missing security headers, outdated TLS settings, exposed administrative pages, and similar issues visible from the outside.
- Common, well-defined vulnerabilities. Dynamic scanners (DAST) can probe a running application for some classes of injection and misconfiguration.
All of these work by matching something they observe against something they already know to look for. That’s their strength, and their limit.
Why some problems are invisible to them
A scanner can tell you whether code looks like a known mistake. It generally cannot tell you whether the code does what your business needs it to do — because it doesn’t know what your business needs.
Consider a simple rule: a customer should only see their own invoices. Nothing about a line of code like “fetch invoice number 1042” looks dangerous. It’s syntactically fine, uses safe database methods, and contains no known-vulnerable functions. Whether it’s a serious data breach depends entirely on a rule that exists in your head and your contract with customers — not in the code.
That gap explains most of what follows.
1. Authorization and access control
Broken access control sits at the top of the OWASP Top 10, a widely used awareness document describing the most critical web application security risks. It covers situations where users can act outside their intended permissions: viewing other users’ records, changing data they shouldn’t, or reaching administrative functions.
These flaws are hard to detect automatically because “correct” behaviour is specific to each application. A scanner can’t easily know that user A shouldn’t see user B’s statement, or that a “manager” role may approve expenses but a “staff” role may not. Finding these issues usually requires understanding the intended rules, then testing with several real accounts — deliberately trying to cross the boundaries.
This matters especially for AI-assisted development. If a prompt asked for “a page that shows the invoice”, the generated code will very likely show the invoice. Whether it checks ownership depends on whether anyone asked.
2. Business logic
Business logic flaws are mistakes in the rules of an application rather than its syntax. Some examples:
- A discount code that can be applied repeatedly.
- A booking system that allows two people to reserve the same slot if they click at the same moment.
- A refund process that can be triggered for an order that was never paid.
- A free trial that can be restarted indefinitely with a new email address.
- A multi-step form where skipping a step bypasses a required check.
Every line involved may be technically sound. The vulnerability is in the sequence, the assumptions, or a missing rule. Identifying these requires knowing what the application is for — which is human, contextual work.
3. Architecture and design decisions
Scanners examine code and configuration as they are; they don’t evaluate whether the overall design is a good one. Questions like these are beyond their reach:
- Is sensitive data held somewhere it doesn’t need to be?
- Does every request go to an expensive external API that could be cached?
- Will the database design cope with ten times the data, or will key pages slow to a crawl?
- Is the application relying on a third-party service in a way that creates a single point of failure?
- Has the system become more complicated than its job requires?
These decisions often determine whether an application stays cheap and reliable or becomes expensive and fragile. They’re also far easier to change early than late.
4. Context and severity
Automated tools often produce long lists of findings, many of which don’t matter in context. A “critical” vulnerability in a library function your application never calls may be far less urgent than an unflagged authorization flaw in your customer portal.
Turning a scan report into a sensible plan — what matters, what doesn’t, what to fix first — requires judgement about how the application is actually used and what the business consequences would be. Without that, teams either ignore scanner output entirely (because there’s too much of it) or spend time on the wrong things.
5. Real-world behaviour and operations
Many problems only appear when software meets the real world:
- Behaviour that differs between a developer’s laptop and the hosting environment.
- Caching that serves one user’s private page to another.
- Error messages that reveal internal details only under specific failure conditions.
- Rate limits, timeouts and connection limits that appear only under load.
- Backups that exist but can’t actually be restored.
Some of these sit outside the code entirely — in DNS, hosting configuration, deployment pipelines or operational habits. A code scanner never sees them.
6. User experience and accessibility
Automated accessibility checkers are useful, and they reliably catch certain issues such as missing labels or insufficient colour contrast. But many accessibility and usability problems require a person to notice: confusing journeys, forms that are technically labelled but practically unusable with a keyboard, or error messages that don’t explain how to recover. Usability problems aren’t usually security issues — but they can still cost you customers.
Scanners and experienced review work best together
The right conclusion isn’t “don’t use scanners”. It’s to understand what different kinds of assurance provide:
| Approach | Good for | Limited at |
|---|---|---|
| Dependency (SCA) scanning | Known vulnerabilities in third-party packages | Whether the vulnerable code is actually reachable or matters |
| Static analysis (SAST) | Recognizable unsafe coding patterns | Business rules, authorization intent, design |
| Dynamic scanning (DAST) | Some injection and configuration issues in a running app | Multi-user authorization and logic flaws |
| Experienced manual review | Authorization, business logic, architecture, context and prioritization | Exhaustive coverage of every line — it’s slower and must be scoped |
| Formal penetration testing | Structured, adversarial testing under a defined methodology | Cost and time; requires careful scoping and authorization |
A sensible approach for most small and medium-sized applications is to run automated tools continuously — they’re cheap and catch regressions — and to bring in experienced human review at key moments: before launch, before inviting many more users, after major changes, or when the stakes rise.
What this means for AI-built applications
AI coding tools and automated scanners share a characteristic: both are excellent at patterns and weaker at intent. An AI assistant may write code that passes every scan while omitting a rule nobody stated. A scanner will confirm the code looks safe. Neither will notice that one customer can read another’s records.
That isn’t a criticism of the tools. It’s a description of the gap that experienced review fills: understanding what the application is supposed to do, who it’s for, and what would genuinely hurt the business if it went wrong — then checking whether the software actually behaves that way.
If you’d like that kind of second opinion on something you’ve built, an Independent Application Review is designed for exactly this. You can see the format in our sample assessment.
This article is general information, not advice about your specific situation. We review our articles periodically and update them when the facts change.