> ## Documentation Index
> Fetch the complete documentation index at: https://docs.argalabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Training tasks and verifiers

> Generate and review training tasks, starting state and executable verifiers from filtered traces

Use the `training` resource in Python SDK v0.2.0+ to turn [SDK traces](https://docs.argalabs.com/sdks/python/tracing.md) or [Braintrust sync evidence](https://docs.argalabs.com/sdks/python/braintrust.md) into reviewable tasks. The same methods are available on `Arga` and `AsyncArga`; await async calls.

Use a full Arga credential for these operations. Upload-only `ARGA_TRACE_KEY` credentials cannot read collections, generate tasks or approve them.

## Generate a draft

Use the collection ID from an upload receipt or the Training URL. The persisted trace ID is returned by evidence upload and is also available on the collection:

```python theme={null}
import os
from arga_sdk import Arga

arga = Arga(api_key=os.environ["ARGA_API_KEY"], timeout=300)
dataset = arga.training.get("DATASET_UUID")
trace_id = dataset.traces[0]["id"]
task = arga.training.generate_task(dataset.id, trace_id=trace_id)
print(task.id, task.status, task.revision)
```

A successful response contains a draft, its starting environment, provenance, assumptions, scoring rules and a compiled Python verifier. Generation can take longer than the SDK's default request timeout; the example gives it 300 seconds.

Generation uses only the stored evidence projection and any explicitly supplied scoped snapshots. Omitted state remains unknown. Missing objectives/state, incomplete ordering and collection gaps block generation while accepted evidence remains inspectable.

The recorded agent is not the definition of success. For example, a $49 duplicate charge with a prior $12 refund needs $37 more refunded. The starting environment retains the prior $12 refund, not the attempt's new refund. A verifier should require \$49 total refunded while protecting the original charge and the owner-review policy.

## Inspect the verifier

The examples below share the open `arga` client, `dataset` and `task`. Inspect and save the compiled artifact:

```python theme={null}
from pathlib import Path

bundle = arga.training.get_grader(dataset.id, task.id)
artifact = bundle["grader"]
Path("grader.py").write_text(artifact["source"])
print(artifact["engine_version"], artifact["source_hash"])
```

The verifier is compiled from the proposed rules, not runnable code generated by the model. Review its rule coverage and limitations before approval.

Its checks support existence, absence, literal equality, substring containment, counts and unchanged state. `unchanged` compares the full selected record set, ignoring record order and recipient/label ordering. It requires consistent normalized fields before and after.

## Test normalized snapshots

Capture the state needed by your rules and save a local `snapshots.json` containing `before` and `after` objects. A collection has this shape:

```json theme={null}
{
  "before": {
    "stripe": {
      "charges": {
        "complete": true,
        "records": [
          {"id": "YOUR_TASK_CHARGE_ID", "amount_refunded": 1200}
        ]
      }
    },
    "gmail": {"messages": {"complete": true, "records": []}}
  },
  "after": {
    "stripe": {
      "charges": {
        "complete": true,
        "records": [
          {"id": "YOUR_TASK_CHARGE_ID", "amount_refunded": 4900}
        ]
      }
    },
    "gmail": {"messages": {"complete": true, "records": []}}
  }
}
```

Replace the ID with the identifier in your task and include every collection, record and field its rules require. The example illustrates the envelope; it is not a complete snapshot for every task.

Supported collections are Gmail `messages`/`drafts` and Stripe `customers`/`payment_intents`/`charges`/`refunds`. Use stable, normalized records and task-consistent identities, not arbitrary provider payloads. Every record needs a unique nonempty string `id`. The downloaded source's `FIELDS` lists accepted fields. Stripe amounts are integer minor units.

Declare an empty collection explicitly. Set `complete: true` only after retrieving the complete relevant scope; omission means unknown. Arga trusts that declaration. The SDK does not automatically capture these before/after snapshots. The `example` returned by `get_grader()` is illustrative, not captured runtime evidence.

Test through the API using your current task revision:

```python theme={null}
import json

snapshots = json.loads(Path("snapshots.json").read_text())
result = arga.training.test_grader(
    dataset.id,
    task.id,
    revision=task.revision,
    before=snapshots["before"],
    after=snapshots["after"],
)
print(result["outcome"])
```

Or run the downloaded standard-library-only verifier locally:

```bash theme={null}
python grader.py snapshots.json
```

| Outcome | Meaning | CLI exit code |
| - | - | - |
| `pass` | All reviewed rules hold | 0 |
| `fail` | A required outcome is missing or incorrect | 1 |
| `unsafe` | A forbidden-state guard is violated | 2 |
| `error` | Evidence is missing, incomplete or invalid | 3 |

Tests accept at most 2 MB of snapshots, 5,000 records per collection and 30 rules. Unsafe takes precedence over error, then fail. Any rule error suppresses the numeric score.

A snapshot verifier does not execute an agent, judge writing quality or detect actions undone between captures. A passing result establishes only that the reviewed predicates hold for the supplied snapshots.

## Edit and rebuild

Submit a full revised specification. The response contains the current revision and a compiled verifier:

```python theme={null}
spec = task.spec.copy()
spec["name"] = "Duplicate-payment correction"
task = arga.training.update_task(
    dataset.id, task.id, revision=task.revision, spec=spec
)
```

Keep the objective, seed, provenance and grading consistent. Editing invalidates earlier approval. Stale revisions return HTTP 409. To rebuild the verifier explicitly:

```python theme={null}
task = arga.training.prepare_grader(
    dataset.id, task.id, revision=task.revision
)
```

Use the returned revision for subsequent testing and review.

## Review and approve

Open the collection in [Training](https://app.argalabs.com/training). Review the task, starting state, scoring rules, Python grader, evidence and assumptions. Approval requires explicit acknowledgement of both the assumptions and the grader.

After that review, you can approve through the SDK:

```python theme={null}
approved = arga.training.review_task(
    dataset.id,
    task.id,
    revision=task.revision,
    decision="approved",
    acknowledge_assumptions=True,
    acknowledge_grader=True,
)
print(approved.scenario_id)
```

Approval saves the starting state as a reusable Scenario. Rejected or unreviewed tasks cannot be used for approved export or environment preparation. Do not automate the acknowledgements without reviewing the actual revision.

## Prepare an approved environment or export

Approve the current revision first. If you edited or rebuilt a task, review its current revision again before preparing an environment.

```python theme={null}
environment = arga.training.create_environment(dataset.id, task.id)
jsonl = arga.training.export_jsonl(dataset.id)
Path("approved-tasks.jsonl").write_text(jsonl)
```

Environment preparation requires approval of the current task and grader and provisions the task's Scenario twins. Export includes approved tasks. These operations prepare data and service environments; they do not run your agent or train a model.

Collections are private to their owner. Selected business facts remain in the existing Training storage lifecycle and are sent to Arga's configured model provider when you request generation. V1 adds no automatic expiry or new derived-artifact retention policy.

Close the shared client when finished:

```python theme={null}
arga.close()
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.