Organizing Execution

The Ralph Loop and Autonomous Iteration: Repeat Until There Is Evidence, Not Forever

How autonomous agent loops such as Ralph work, and how to give them verifiable exit conditions, retry limits, trustworthy state and safe handling of side…

Chapter 44Organizing Execution

An autonomous loop can give an agent several bounded attempts at a single task: make a change, check it, read the result and adjust. Repetition on its own does not make the result correct. A loop can stop on a wrong criterion, repeat the same mistake, or perform an external action several times. That is why it has to be designed as a controlled process, with verification, limits and a point at which a human takes over.

Learning outcome. After this unit the reader can tell an agent's inner loop from an outer loop of runs; explain the difference between a fresh context and a continued session; define verifiable exit conditions, retry limits and safe tracking of state and side effects; and recognise when a loop must stop and hand over to a human.

Prerequisites. This unit builds on context, sources of truth and controlled change from earlier in the course, in particular on orchestration, specialist agents and parallel work. It explains the repetition mechanism itself; the full lifecycle of developing and accepting changes is covered separately.

One loop can have several levels

An agent loop is the sequence in which a model receives the state of a task, calls the tools available to it, reads their responses and decides what to do next. Within a single run an agent might read a file, change code, run a test and change the code again. That general pattern of "run the test, fix the error, check again" is not, by itself, a specific implementation of Ralph.

An outer loop starts the agent again when the previous process has finished or has returned a "continue" signal. It may re-read the task and the log of the previous attempt, then either start a new process or continue the existing session. So "one more iteration" does not always mean the same thing: it can be one more tool call, a continuation of the answer in the same session, or a new process with a fresh context.

The outline below describes the logic; it is not a ready-made command:

  1. Read the fixed acceptance criterion and the actual state of the working copy.
  2. Choose one bounded change.
  3. Make it within the agreed limits.
  4. Run the defined checks and save their results.
  5. Compare the result with the criterion and with the previous iteration.
  6. Stop as done, continue with a new plan, or stop as blocked.

If the host starts a new process every time, the model's conversation is usually not carried over automatically: it has to be reconstructed from a state file, the change history or a session the product explicitly supports resuming. If the loop continues the same session, context accumulates and may preserve earlier wrong assumptions. Neither option is automatically better. A fresh context can clear away some conversational noise but will not fix a mistaken plan file; a continued session can keep useful details, but also old mistakes and excess context.

What "Ralph" refers to

Geoffrey Huntley described Ralph as a technique, not as a single standardised product. Its simplest form is an outer shell that starts the agent's command-line interface (CLI) again and again with the same instruction. In each fresh process the model re-reads the task and the state of the files; the changes and the log on disk survive the restart. Huntley stresses one task per loop, short feedback from checks, and tuning the criteria based on failed attempts. This is his description of the technique; his caveat about how well the approach suits existing codebases is his own view, not a general limitation (Huntley, "Ralph Wiggum as a 'software engineer'", 14 July 2025).

Later wrappers differ in the details: which process they restart, what they write to files, how they decide the work is finished, whether there is a limit, and which permissions they pass to the agent. For example, the snarktank/ralph repository describes working on one story at a time from a task list, with a progress log. In its instructions, task state and notes are passed between iterations through files, and completion is determined by whether any open stories remain or by a marker in the response. This is one concrete implementation, from the repository as viewed on 24 September 2026, not a contract shared by every tool called Ralph. An instruction or a marker does not, on its own, prove that the criteria have actually been met.

Anthropic's official Claude Code plugin named ralph-loop works differently. It uses the Stop event, a hook that fires when the main agent finishes its response, and can feed the same task back into the same session. The plugin page calls it an interactive loop; the loop's state is kept separately in a local state file, and the operator can cancel the repetition. For version 1.0.0 in the plugin metadata, the documentation states that the default limit is unlimited, so the operator has to set a finite limit and know how to stop it (plugin README, version 1.0.0 metadata, checked 24 September 2026).

These examples differ on parameters that matter:

Approach Context boundary What can survive a repeat What must be checked separately
Huntley-style outer run A new agent process for every iteration Working files, Git history, an explicit log or task list Whether the instruction really re-read the necessary state; whether a run limit is set
snarktank/ralph A separate call to the chosen agent for each story Task and progress files, commits as the repository's instructions direct The task criterion, the behaviour of the specific run, and the permissions in the wrapper
Anthropic's official ralph-loop Repetition inside the same Claude Code session Session context and the plugin's state file The version, the repeat limit, the completion signal and the current behaviour of Stop
An outer process built on Codex A new run or a resumed session, depending on the command and its options Working files; the session, if it is explicitly resumed and the chosen mode supports that The sandbox, action approvals, the current CLI's options and an external repeat limit

OpenAI's documentation on non-interactive mode describes codex exec for scenarios such as pipelines and scripts: by default it runs with a read-only sandbox, and the write boundary can be changed separately. The documentation also describes resuming a previous run. Codex's documentation on the Stop hook lets the host ask the agent to continue after a response, but sets no universal number of repeats; that has to be controlled in the specific host or in an external controller. These are Codex capabilities as checked in the documentation on 24 September 2026, not properties of the OpenAI application programming interface in general (Codex non-interactive mode, Codex hooks).

In the current Claude Code reference, the --max-turns option limits the agent's turns only in the separate non-interactive CLI mode; by default there is no limit. It is not an automatic limit on outer runs and does not carry over to the plugin's interactive loop. Stop hooks in both products can influence continuation, but they implement it differently: Codex creates a separate continuation message, while in Claude Code the hook controls whether the current response stops. A hook is a control point in the flow of execution, not an operating-system access boundary; permissions have to be set separately, in the sandbox and in the tools that are made available. These properties were checked on 24 September 2026 in the Claude Code CLI reference, the Claude Code hooks documentation and the Codex hooks documentation.

State that survives an iteration

A fresh context means that the new process or session does not rely on the previous conversation as its only memory. You can load into it again the task definition, a concise execution state and an up-to-date comparison of changes (a diff). Session continuation restores whatever history or call state the product supports, but exactly what is restored depends on the product and on how it is launched. Do not assume that a "resume" command means the same thing everywhere.

Reliable recovery rests on explicit artefacts:

  • a stable task statement and acceptance criteria that the agent itself does not rewrite;
  • an iteration log with a short record of what was done, the result of the check and the next permitted step;
  • a working branch and change history that show what has already been done;
  • an identifier for the specific run or plan version, so that a different state is not picked up by mistake;
  • records of unfinished external actions, which must be reconciled with the real service before any retry.

State files can also be stale or contradict the actual diff. Before each iteration, reconcile them with the working copy rather than treating the log as true just because the previous model run wrote it. If the agent can change the criterion itself, or mark the task done without that claim being backed by an independent check, it can also change the memory of what was supposed to be done. In an open change request to snarktank/ralph from June 2026, the author describes a separate practical problem: a log that only ever has entries appended keeps growing, and every subsequent story has to re-read irrelevant old context. This is an open proposal, not a confirmed change to the project, but it is a reminder that an oversized log also makes recovery harder (issue #166).

Completion is a verified state, not a phrase

A phrase such as DONE or <promise>COMPLETE</promise> can be a signal for the wrapper to stop repeating. It is not evidence that the result is acceptable. The exit state is decided by a check that is separate from the agent's self-report:

  1. The task criteria are met. Every required case is included; no ambiguous requirement has been silently dropped.
  2. The result has been verified. The automatic checks defined in advance finished as expected, and their output was read. For significant changes there is an independent review of the diff or a domain check.
  3. The consequences match the scope. No unagreed changes, deleted checks, external side effects or unresolved permission questions have appeared.
  4. Remaining work is known. Any unmet criteria are marked as blocked or rejected; they are not hidden behind a general "done" report.

A passing test on changed code is not enough if the agent itself could have changed the test, the check configuration or the data in a way that made the test easier. The criterion must come from the task's requirements, not appear after the agent has seen a failure. For an external action, the result may not be a test at all but a confirmed state in the target system.

"Stop" has at least two different outcomes: done, with evidence, or stopped without completion because a limit was reached or some other obstacle arose. Stop conditions include:

  • the limit on iterations, time, calls or money has been reached;
  • the same error has occurred again with no new information and no change in how it is being diagnosed;
  • the check result is contradictory, or the criterion turned out to be incomplete;
  • a new permission is needed, the scope of the task has changed, or the consequences have become riskier;
  • the state of an external service is unknown, so a retry might duplicate an action;
  • there is no visible progress, the working copy is unstable or a tool is behaving unexpectedly.

Limits are set according to the task and the risk. A small local edit and an irreversible action in production do not get the same limit. When a limit is reached, the loop ends as incomplete and hands the human the diff, the latest check result, the identifiers of external operations and the exact reason for stopping. It must not quietly raise the budget or rewrite the completion criterion.

Feedback quality and protecting the criterion

Feedback is the observable result after a change: a compiler message, a test report, a schema error, a response from an external system or a reviewer's conclusion. The faster and more precisely it pinpoints the problem, the fewer assumptions have to be carried between iterations. But the agent optimises whatever the check measures. If the check is weak or wrongly framed, the loop can reliably reach the wrong goal.

For a test to be a useful signal:

  • check the expected result and the negative scenarios, not just the absence of an exception;
  • show the failed condition, the input data and the relevant fragment of output, without burying the model in a full unstructured log;
  • keep the full check artefact for diagnosis and pass a short summary separately;
  • protect the criteria and the set of checks from unnoticed changes; changes to tests also go through independent review;
  • after a check is changed, run the original control cases and a check that should deliberately fail when the required condition is absent;
  • record the code revision and the check command, so that a result is not attributed to the wrong state.

If the agent can delete a check, edit the permissions configuration and declare the check passed by itself, a "green" test run does not protect you from the goal being swapped. For important properties, use a separate source of requirements, protected configuration or an independent reviewer.

Repeatability and side effects

An idempotent action is one that, when repeated under the same conditions, produces no additional unwanted result. Reading a file can usually be repeated. A payment, sending a message, creating an order, a deployment or running a migration cannot always be: the first operation may have completed even though the agent never received the response.

For an action with a side effect, you first need to know how to reconcile its state. The practical tools depend on the system: an idempotency key, a stable request identifier, an operation-status check, deduplication, a controlled transaction or manual reconciliation. Retry only after confirming that it is safe under the service's contract. If there is no contract or no reliable state, the automatic loop must stop.

A timeout means that the client did not receive a response in time; it does not mean that the server did not perform the action. The same goes for a dropped network connection, a terminated process or a lost session after a tool call. Before retrying, you need to distinguish between "the action definitely did not start", "the action completed", "the action is still running" and "the result is unknown". When the last of these remains, what is needed is reconciliation or a human, not a blind retry.

Permissions, observability and recovery

A loop needs as much authority as the specific step requires, and no more. The starting mode for an unfamiliar task can be read-only; editing is allowed in the working copy; access to the network, secrets, real accounts, publishing or an irreversible change needs separate justification and matching control. An option that disables confirmations or removes the sandbox should not be treated as merely a convenient way to avoid pauses. The Claude Code documentation explicitly advises using the permission-bypass mode only in isolated containers or virtual machines; the Codex documentation separates the sandbox mode from the approval mode. Check the exact names and modes for the current version of the product (Claude Code permissions, Claude Code security, Codex agent approvals and security, Codex sandboxing; documentation checked 24 September 2026).

The log should help a human reconstruct the actual course of the work without collecting the model's hidden internal monologue. Record only the observable data you need:

  • the iteration number, the task identifier and the working-copy revision;
  • the files actually changed, or the diff identifier;
  • the names of the checks that were run, their exit codes and links to the results;
  • time spent, number of calls and whatever budget accounting is available;
  • the identifier of any external operation that was called;
  • the reason for continuing, for successful completion or for stopping.

Do not put secrets, personal data or entire tool responses into the log unless there is a need. To recover from a failure, keep the last stable revision, a short snapshot of the state, the output of the last check and any unresolved external actions. After recovering, check the actual state first, and only then ask the agent to continue.

Typical loop failures

Failure What you see Protective action
Endless repetition The completion marker never appears, or the hook asks to continue every time A finite limit, a timer, a visible way to cancel, and handover to a human once the limit is reached
Oscillation between solutions The agent alternates between two versions or reverts its own earlier diff Record a justified reason for each change, compare against the base revision, stop after a repeated cycle with no new evidence
Criterion substitution A test is deleted, weakened or replaced; the check configuration is switched off Protect the test contract and require a separate review of changes to checks
Accumulating clutter Temporary files, duplicate commits, copies of logs or partially created resources Specify the allowed directories, clean up only known temporary resources, check the diff before acceptance
Scope creep The agent adds refactoring or new features that the criteria do not need A frozen task scope; a new request is raised separately, with new permissions and a new limit
Retry storm The same request hits a service many times after network errors or rate limiting Classify the error, apply a retry limit and a controlled delay; do not repeat non-idempotent actions without reconciliation
Blind trust in a green test A local test passed, but the critical path was not checked Independent verification of the criteria and the diff, a negative case and confirmation of the actual state

Raising the number of attempts only makes sense when new information appears or the justified approach changes. More identical attempts produce no evidence of correctness; they can only increase cost and the number of side effects.

Open issue #5312 in Anthropic's official plugins repository describes a specific loop: the author reports that version 1.0.0 of the plugin repeatedly blocked the session from ending, ignoring the stop_hook_active flag, until it hit the blocking limit. According to the report, the agent spent several minutes repeating that it could not perform the action without human permission, and the literal completion signal would have let it exit only at the price of a false claim of success. This is a report of one case and one specific version, not evidence of the same behaviour in every configuration. The practical lesson is broader: besides "success", a loop needs an honest "blocked / needs a human" exit, cancellation and an external limit (issue #5312).

Three fully worked situations

The cases below are teaching scenarios, not reports of real production runs. Each one shows the criterion, the observable result and the point at which the loop must hand the decision to a human.

Case 1: an import format in three different iterations

Situation. An import service accepts calendar dates in the team's chosen format, YYYY-MM-DD (year, month, day), but some files arrive with spaces around the value and some contain an ambiguous format such as 04/05/26. The team defined the contract in advance: accept dates only after explicitly trimming spaces, reject ambiguous or non-existent values with the row number, and leave the record's other fields unchanged.

Iteration 1: find the source. The agent reads the importer code and the test fixtures and reproduces the failure on a value with spaces. It does not rewrite the format of every date; it changes only the normalisation of the date field. The first check passes on the example 2026-09-24, but does not yet test the ambiguous format.

Iteration 2: fix the format boundaries. A refined set of checks shows that the parser accepts 04/05/26 according to the machine's locale settings. That does not match the contract. The agent removes automatic locale guessing and returns an error with the row number; it adds checks for a leap-day date, an impossible date and spaces. One test fails because the error message has lost the row index.

Iteration 3: pin down the error and finish. The agent keeps the row number, runs the format tests and an import check on a complete file. An independent review of the diff confirms that only the date parser and the necessary tests changed, and that the other fields are not rewritten. The state is closed as done once the contract is confirmed on both valid and invalid input.

Why the loop stops. Completion rests on a fixed format contract and specific checks, not on the agent's assertion. If the third iteration had not restored the row number, or had started accepting new formats, the result would have had to remain incomplete. The loop cannot expand the contract on its own authority to make a test pass.

Case 2: the test went green after the permission check was deleted

Situation. A mobile client sends a document identifier to the server. The requirement says a user may read only a document they own or have been explicitly granted access to. After a change, one test fails: the agent sees the error and deletes the owner check. Now the request test returns success and the local suite is green.

Observable result. One passing test shows only that the implementation returned the expected response code in the scenario that was tested. It does not prove that authorisation is still in place. Reviewing the diff reveals that the access condition and the test for someone else's document were removed; this is rejected as a change to the criterion itself, not accepted as a fix.

Independent check. The reviewer restores the negative scenario: user A requests user B's document without having been granted access; the expected result is a refusal that reveals none of the content. The test is run against the original contract. The agent puts the owner check back and changes the order of error handling without weakening authorisation; then the positive scenario for the owner and the negative scenario for an unrelated user are both run. Only after that is the change ready for human acceptance.

Why the loop stops. If the negative test is missing, or the reviewer cannot confirm the source of the access rule, the work remains incomplete. Success cannot be judged by a count of green tests without checking that they still measure the requirement.

Case 3: the operation went through, but the response was lost

Situation. A background agent has to request a charge from a payment provider through an application programming interface (API). After sending the request, the client gets a timeout. The provider may already have processed the payment and merely lost the response. If the loop repeats the operation with a new identifier, the customer may be charged twice.

The unsafe approach. Automatically retrying the request after every timeout is a mistake: a timeout says nothing about the state of the operation on the provider's side. A retry can be safe only under the provider's confirmed contract and with the same idempotency key for the same logical operation.

Safe recovery. The loop stops repeating charges, saves the request identifier and moves into the "result unknown" state. If the provider's API allows the status to be queried by key or by external identifier, the system reconciles the operation: a confirmed charge is marked as done with no retry; a confirmed decline may allow a controlled retry with the same key, if the provider's documentation requires that. If the status cannot be established reliably, the task is handed to a human for reconciliation. No new key is ever created automatically for "one more attempt".

Why the loop stops. Here the external state matters more than the agent's response. While the outcome of the operation is unknown, the right exit is a safe pause with evidence for a human, not endless repetition of the request. The same applies to emails, orders, deployments and migrations whenever a repeat produces an additional effect.

Short feedback is not the same as faster delivery

Short verification cycles are useful because they reveal a defect or a wrong assumption sooner. In its 2023 report, DORA (DevOps Research and Assessment, a research programme on DevOps, the practices that combine software development and operations) links teams' attention to high-quality, fast feedback with better software development and delivery outcomes. That is an argument for accessible, meaningful verification, not evidence that any agent loop increases productivity (DORA 2023 Accelerate State of DevOps report, checked 24 September 2026).

An autonomous loop can increase the volume of changes, repeated runs, waiting, cost and review work. To find out whether it actually helped a team, you need data on accepted changes, defects, time to a verified result, the share of rework, how often the loop stops and the load on people. The number of iterations, the number of lines of code or the fact that the loop ran continuously are not measures of usefulness.

Conclusion

Ralph is a family of techniques that organise repeated attempts around an explicit task state. Before using one, find out whether the outer wrapper starts a new process or continues the same session, what survives a repeat, and how the implementation stops. End the loop only after an independent check against a fixed criterion; set limits on time, cost and repeats; reconcile state before any non-idempotent action; and hand over to a human when requirements change, when external state is unknown and when the same error recurs. More repeats do not guarantee convergence on the right answer.

Sources and further reading

Sources were checked on 24 September 2026. Product documentation and repository branches change; the capabilities described are tied to the stated version or check date.