Open the receipt-checking diagram at full size.
A RAG ingestion worker submits three chunks to Elasticsearch. The HTTP response is 200, so the worker records three indexed chunks and advances its source checkpoint. Later, retrieval cannot find the policy paragraph a user needs.
The response body would have explained why. One chunk indexed, another was rejected under load, and the third failed its mapping. The worker counted the request envelope, so the document outcomes went uncounted.
I would block a corpus release on unresolved item receipts. Tuning retrieval against an incompletely loaded corpus sends the investigation towards embeddings and ranking before anyone has established what the search engine accepted.
This is a synthetic application example and describes no production incident. The downloadable Python fixture checks constructed response objects. It makes no Elasticsearch requests, calls no models and does not measure retrieval quality. The API behaviour comes from Elastic’s documentation; the checkpoint and retry rules below are application design choices.
HTTP success and document success are separate checks
The Elasticsearch bulk API reference documents a 200 response containing an errors Boolean and an items array. errors: true means one or more operations did not complete successfully. Each item contains its own status, and results appear in submission order.
That contract makes this checkpoint rule unsafe:
if response.status_code == 200:
mark_all_submitted_documents_indexed()
Checking errors is a necessary improvement, but a single failed-batch flag still loses useful information. You need to know which documents succeeded and which require action. Retrying everything can repeat successful work; skipping the batch can leave missing chunks behind a checkpoint that says ingestion is complete.
For a small, explicit-ID index batch, the receipt might contain these outcomes:
HTTP 200, errors=true
chunk-a -> 201: acknowledged
chunk-b -> 429: retry candidate
chunk-c -> 400: blocked for repair
The 429 is a candidate for a later attempt, subject to the retry policy. The mapping failure needs investigation. Repeating the same invalid document against the same mapping does not fix it.
A missing chunk can affect more than answer completeness. If the corpus already holds an older version of that chunk, failure to index its replacement may leave stale text available. If a permission update was part of the rejected document, stale access metadata may remain too. Those are consequences of the application’s update design, and a successful HTTP envelope does not rule them out. The earlier RAG cache revocation article examines a related problem after data has reached the serving path.
Run the receipt checker
Download and inspect the standard-library Python fixture. Run it with Python 3.9 or later:
python3 rag-bulk-receipt.py
python3 rag-bulk-receipt.py --test
python3 -O rag-bulk-receipt.py --test
The default run prints:
weak_acknowledged=3
{"acknowledged": ["chunk-a"], "blocked": ["chunk-c"], "retry_candidates": ["chunk-b"]}
weak_acknowledged=3 demonstrates the envelope-only count. The second line classifies the same submitted batch using its item receipts. Only chunk-a belongs in the acknowledged set.
The fixture passed 14 tests on Python 3.9.6, both normally and with optimisation enabled. Tests cover mixed outcomes, a complete successful batch, missing or reordered receipts, duplicate submitted IDs, a wrong index, contradictory summary flags and malformed status fields. Other tests reject an error attached to a successful item and a failure without error detail. A conflict or server error stays blocked for investigation and never enters the fixture’s automatic retry-candidate class.
This checker deliberately supports a narrow contract: index actions with explicit, unique document IDs in one concrete index, rag-v7, and a full response body. It does not support mixed operations, aliases, data streams or response filtering. Elasticsearch can return a backing index name for a data stream; comparing that blindly with the submitted stream name would reject a legitimate receipt. Resolve and test your actual target contract before adapting this example.
Nor is the fixture a complete Elasticsearch response-schema validator. It validates the fields needed for this classification and leaves other response fields untouched. Its output acknowledges the item status only, and leaves replica health, search visibility and corpus readiness uncertified.
Validate the whole receipt before advancing state
The checker first requires a usable envelope and the expected number of items. It then walks submitted IDs alongside their receipts:
for doc_id, item in zip(expected, items):
if type(item) is not dict or set(item) != {'index'}:
raise ValueError('expected an index receipt')
row = item['index']
if (type(row) is not dict
or row.get('_id') != doc_id
or row.get('_index') != 'rag-v7'):
raise ValueError('receipt identity mismatch')
The full download checks cardinality before this loop. Without that check, zip would stop at the shorter list and silently ignore an unmatched submission. It also rejects duplicate submitted IDs, because this fixture uses document identity to distinguish outstanding work. A production batch that intentionally writes the same document more than once needs operation-level identity and ordering rules instead.
The checker returns its classifications only after validating every item and reconciling the errors flag with the observed outcomes. It performs no checkpoint writes inside the loop. If the last receipt is malformed, the function raises an error and returns no partially trusted result.
That refusal does not mean the server performed no writes. A lost response, malformed receipt or transport failure can leave an unknown outcome. Retain the submission manifest and reconcile the destination before deciding what to replay. A parser exception cannot roll back a document that the server already accepted.
For durable application state, persist item outcomes together with the batch identity and the submitted content versions. Then advance a source checkpoint according to an explicit completion rule. If the checkpoint moves past unfinished items, those items need their own durable recovery records. Otherwise a worker restart can turn a temporary rejection into a permanent omission.
For an initial corpus build, my default would be to keep the new corpus out of service until every required item has an accepted outcome and the serving checks pass. A deliberate partial release is possible, but it needs a declared missing-document set and a product decision about the affected questions. A green ingestion job should not hide that decision.
Retry only after checking what another attempt would do
Elastic’s indexing-speed guidance recommends pausing and using randomised exponential backoff when indexing receives 429 responses. The fixture therefore places 429 items in retry_candidates. It does not send them again.
A retry worker still needs an attempt limit, a deadline and a record of the exact payload it would submit. Keep the successful chunk-a out of that retry batch. Repair chunk-c separately, and record the mapping or input correction while keeping the failed attempt marked as failed.
Stable document IDs help identify the intended destination, but they do not make every replay safe. The bulk reference states that an index action adds or replaces a document. An old retry can overwrite newer content under the same ID. Replaying an update script can have different consequences again, which is one reason this example excludes update operations.
Mutable documents need a concurrency rule. The bulk API supports if_seq_no and if_primary_term on individual index and delete actions. An application can use those preconditions to prevent a write based on an obsolete document state. A conflict then requires reconciliation; treating every 409 as transient and retrying without the precondition would discard the protection.
Another design for a versioned corpus is to write into an isolated generation and keep its payloads immutable. That narrows the overwrite problem, but still requires durable recovery state and a controlled promotion step. The fixture proves neither design. It gives you a small receipt boundary to test before connecting a retry queue to real writes.
Indexed is not yet searchable
Even a batch with successful item receipts needs a serving check. The bulk API’s refresh setting controls when changes become visible to search. Elastic documents that refresh=wait_for waits for the affected shards to refresh. That leaves readiness for unrelated shards unproven, and it leaves open whether your RAG query will retrieve the intended chunk.
Test the actual retrieval path after the chosen visibility boundary. Use a known chunk and the same tenant identity, filters and index selection the application will use. A direct document lookup is useful evidence that a document exists, but it does not exercise the ranking and authorisation path.
Keep the acceptance questions separate in your job record: did each required write receive a valid acknowledgement, are the expected versions visible to search, and does the serving query enforce the intended access policy? If a document is acknowledged but not yet visible, inspect refresh and query behaviour. If it has a 400 item receipt, start with the indexing error and leave the embedding model alone.
Before connecting this checker to production, add one adapter test against your deployed Elasticsearch version: submit a valid document and a deliberately invalid document to a disposable index. Confirm that the client exposes both item outcomes, the checkpoint stays open for the invalid item, and the retry path excludes the successful document. Do not run that test against a live customer corpus.
For broader production tool controls and review practices, see my Claude Code book.