Security testing of AI applications, chatbots and LLM agents
Does your application use a language model? The testing checks whether it can be talked into things it should not do, and whether it leaks data it must not leak.
Book a free consultationWho it is for
- Customer service chatbots
- SaaS assistants and copilots
- Agents with tools (e-mail, CRM, payments, code)
- RAG on company documents
- No-code integrations (Zapier/Make + LLM)
- Bots from external vendors
What gets tested, mapped to the OWASP Top 10 for LLM Applications
Prompt injection
Direct (the user) and indirect (documents in RAG, web pages, e-mails, files, product names). Whether the bot’s behavior can be changed, its instructions bypassed or actions forced.
Information leakage
The system prompt, other users’ and customers’ data, knowledge base contents, keys and configuration in context, personal data in responses and logs.
Excessive agency
Which tools the agent has, whether it acts without confirmation, whether it can be talked into acting on someone else’s data.
Improper output handling
Model output rendered as HTML or Markdown (XSS), inserted into SQL or shell commands, executed as code.
API abuse and cost
No authentication, no per-user limits, very long prompts, the endpoint used as a free proxy to the model.
RAG
Knowledge base poisoning, access to documents without permission (no per-user filtering), leaks between tenants.
Filters and guardrails
Moderation bypasses, jailbreaks, offensive or off-brand content, promises and “discounts”.
AI Act Art. 50 obligations
Whether users are told they are talking to AI, and whether generated content is labeled where the law requires it.
How the testing works
AI security testing starts with manual scenarios from the perspective of a user and an attacker (AI red teaming). On top of that come automated prompt suites (custom and open-source: promptfoo, garak), architecture analysis (where untrusted text enters, which tools, which data) and a business impact assessment.
An LLM penetration test of a chatbot, a prompt injection assessment, an AI agent security audit or a review of the whole LLM application: we set the scope on a call. Testing stays within the token budget agreed in the scope.
What you get
- A report: executive summary, findings with priority, example prompts and responses, evidence, recommendations (architecture, tool permissions, filtering, limits, guardrails)
- A walkthrough meeting
- A free retest if the fixes are ready within 30 days of the report
- A “Tested by codequality.pl” badge after the retest
- Optionally, a suite of evals for CI (the “AI Evals in CI” package), so report findings become regression tests
Terms
- Written consent from the system owner
- Agreed scope and testing window
- Token budget
- Emergency contact on both sides
- Prompts and test data never go to external tools without consent
Packages
| Package | What it includes | Time |
|---|---|---|
| AI Security Quick Checkrecommended | review of one AI application: prompt injection (direct and indirect), prompt and data leakage, tool permissions, API limits, AI Act transparency obligations; report with priorities, walkthrough, free retest | report in 5 working days |
| AI Security Review | full testing of the AI layer (mapped to the OWASP Top 10 for LLM Applications) + business logic + RAG + agents; report, walkthrough, free retest | approx. 10 working days |
| AI + API Pentest | AI Security Review + pentest of the API and web application around the model (ASVS L1/L2); one report, free retest | approx. 15 working days |
| AI Evals in CI | a suite of prompt regression tests (promptfoo) and chat interface e2e tests (Playwright) wired into GitHub Actions / GitLab CI; handover to the team | approx. 5 working days |
Frequently asked questions
We use the OpenAI / Anthropic / Google API. The model is not ours, so what exactly gets tested?
Everything that is yours: the system prompt, the knowledge base (RAG), the tools the agent can access, permissions, input and output filtering, limits, logging. That is where most vulnerabilities arise, not in the model itself.
We have a chatbot from an external vendor (a widget). Does testing make sense?
Yes. The test checks what the bot can reveal and do in your context (data, integrations) and whether it meets the AI Act transparency obligations; you also get a list of questions/fixes for the vendor. Some risks sit with the vendor, and the report separates them. If the bot runs on the vendor’s infrastructure, the scope takes the vendor’s security testing policy into account and, where needed, its consent.
Will the tests burn our tokens and raise the API bill?
We agree on a token budget in the scope. Testing runs on a test environment or with limits. There are no load tests on a production key without consent.
Does the testing also cover an open-source model we host ourselves?
Yes, together with the inference server configuration and its API.
Can I get the badge after the fixes?
Yes, after the retest.
Contact
A few sentences are enough: what your application does, where it uses AI and what you want tested. The reply comes within 1 working day, with a proposed time for a free call.