使用AgentDojo对代理防御进行压力测试
Run AgentDojo benchmark environments to evaluate prompt-injection attacks and defenses against LLM agents before trusting them with real tools or data.
Prerequisites
Python 3.10+, agentdojo package, model provider credentials for the target pipeline, optional transformers extra for prompt-injection detector workflows
Installation
Use the upstream install or setup path that matches your environment:
- pip install agentdojo
- pip install "agentdojo[transformers]"
Requirements and caveats from upstream:
-
- python -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \
- python -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \
Basic usage or getting-started notes:
- [!IMPORTANT]
- Note that the API of the package is still under development and might change in the future.
- If you want to use the prompt injection detector, you need to install the transformers extra:
- Source: https://github.com/1991513ccie-png/skills
- Extracted from upstream docs: https://raw.githubusercontent.com/ethz-spylab/agentdojo/HEAD/README.md
Documentation
- https://agentdojo.spylab.ai/
