Command reference

Command map

create test-dataset

Generate a dataset from one or more knowledge files or directories.

lladar create test-dataset --knowledge PATH [OPTIONS]

lladar create test-dataset --knowledge policy/ --knowledge faq.md --count 30 --seed 7 --output artifacts/dataset.jsonl

run-agent

Run an Agent through an inspected project or a calibrated question webpage, verify the route, and fill actual_response in a new responses file.

Choosing the target

--project PATH can be enough on its own: LLaDAR learns the public interface and startup procedure from source and documentation. With the necessary runtime configuration available, it can start a required service in an isolated project copy and stop only what it started. Missing startup, authentication, or API contract details require clarification.

Add --service-url URL only to use an already running test service. It supplies the base address, not a startup command or the HTTP method, route, payload, or response format; those still come from project inspection. LLaDAR does not start or stop that existing service. The option is --service-url, not --service-api.

Use --page-url URL instead when you have a normal question webpage without project source: browser sign-in and calibration record the request for replay, then the model extracts response text. It cannot be combined with --project or --service-url. With neither target option, LLaDAR inspects the current directory. See the three-path selection guide.

lladar run-agent DATASET [OPTIONS]

lladar run-agent artifacts/dataset.jsonl --project path/to/agent --no-interactive --output artifacts/responses.jsonl
lladar run-agent artifacts/dataset.jsonl --project path/to/agent --service-url http://127.0.0.1:8000 --output artifacts/responses.jsonl
lladar run-agent artifacts/dataset.jsonl --page-url https://example.test/chat --output artifacts/responses.jsonl

Direct model extraction is the only browser answer path, default gemini:gemini-3.8-flash. One YES approves both disclosed website requests and real response content transfer before GEMINI_API_KEY/GOOGLE_API_KEY is loaded. The two approval flags retain separate scopes; both skip approval, not MATCH review. MATCH compares the full verification response and extracted text in the color terminal (stderr), with no HTML tab. NO_COLOR or TERM=dumb selects plain text. Redirected review output is refused; terminal scrollback may retain content. Browser capture remains required. Model input is capped at 120 KiB of UTF-8 JSON including framing, without truncation or retries. Valid extraction does not guarantee fidelity or answer correctness. See the run guide.

eval

Use an evaluation method to judge completed responses and calculate a summary from the saved judgments.

lladar eval RESPONSES [OPTIONS]

lladar eval artifacts/responses.jsonl --strict --output artifacts/evaluation.json

report

Render an evidence-bounded Markdown report from an evaluation output.

lladar report EVALUATION [OPTIONS]

lladar report artifacts/evaluation.json --output artifacts/report.md