LLMs in a PHP application: queues, idempotency and validation instead of a Python rewrite
A hosted language model (Anthropic, OpenAI, Mistral) is an HTTPS endpoint that takes JSON and returns JSON. It doesn't care what language the caller is written in. Python does have a real advantage for training, fine-tuning and running models locally. For adding LLM features to an existing PHP system, the hard parts are elsewhere: long and unpredictable latency, output that differs between calls, validation of that output, and cost. Those are queue, database and contract problems. A PHP application already has tools for all of them.
This article walks through those problems in a Laravel application. The examples use the Anthropic Messages API over Laravel's HTTP client, so every field in the code exists in the provider's documentation.
Library or plain HTTP
The usual options:
- Plain HTTP through Guzzle or Laravel's
Httpfacade. One endpoint, a few headers, no extra dependency. Enough when you call one provider for a handful of features. - A client library:
openai-php/client(with a Laravel wrapper), Prism for Laravel, or LLPhant. Depending on the library, they add provider abstraction, streaming helpers or ready-made retrieval components. - Symfony's AI components (the
symfony/aiproject), if the application is built on Symfony and you want integration with its container and configuration.
Whichever you choose, check before production that you can set connect and read timeouts, you can decide which errors are retried, you get token usage from every response, and you can add your own logging around the call. Some libraries hide one of these behind their own defaults, and then you spend time working around them.
A thin client
use Illuminate\Http\Client\ConnectionException;
use Illuminate\Http\Client\RequestException;
use Illuminate\Support\Facades\Http;
final class AnthropicClient
{
public function __construct(
private readonly string $apiKey,
private readonly string $model,
) {}
/** @return array<string, mixed> */
public function messages(array $payload): array
{
return Http::baseUrl('https://api.anthropic.com/v1')
->withHeaders([
'x-api-key' => $this->apiKey,
'anthropic-version' => '2023-06-01',
])
->connectTimeout(5)
->timeout(30)
->retry(2, 2000, fn (\Throwable $e): bool =>
$e instanceof ConnectionException
|| ($e instanceof RequestException
&& in_array($e->response->status(), [429, 500, 529], true))
)
->post('/messages', ['model' => $this->model, ...$payload])
->json();
}
}
The model name lives in configuration. Providers retire model versions on their own schedule. Retries cover only connection errors, rate limiting (429), internal server errors (500) and overload (529). A 400 means the request itself is wrong, so the client doesn't retry it. The exception still leaves messages(), though, and a job with $tries = 3 will try again unless it calls fail() on a 4xx itself.
The read timeout must match the largest response you ask for. Time grows with the length of the response, and max_tokens sets its upper bound. A classification limited to 256 tokens finishes in seconds (on models where thinking can't be turned off, thinking tokens count toward that limit too, so it has to go up); a long summary may not fit in 30 seconds and needs either a higher timeout or streaming.
With local models the same thing gets much more visible. On my home server with Ollama, a small 4B model on CPU generated 12 tokens per second, and a full answer on a log of about 3,000 tokens took 146 seconds, with reading the log (prefill) running at 28 tok/s. With a graphics card it was 66 tok/s, and prefill ran at 1,900 to 3,400 tok/s. The first request after the model was loaded still ran at 3 tok/s, so I set the timeouts for that cold start and not for the warm average.
Model calls in queue jobs
Most LLM work belongs in a queue. The user shouldn't hold an HTTP request open for several seconds, and a worker can retry. Trouble starts with how the timeouts relate to each other.
In Laravel, a job that runs longer than the connection's retry_after (in config/queue.php) is handed to another worker while the first one is still running it. With a slow API this produces two parallel executions of the same job, two paid calls and possibly two different results. The Laravel documentation requires the job's timeout to be shorter than retry_after, by at least several seconds for the worker's --timeout. In addition, the job's timeout must cover the entire client call including its retries. With the client above that is at most 2 × 30 s plus a 2 s pause, so a job timeout of 90 s and retry_after of 120 s are consistent.
Then there's non-determinism. The same input can produce a different classification on the next call, so a retry isn't really a repetition of the same operation. If the first attempt saved a result and something downstream already acted on it, a retry may overwrite it with a different answer. The fix is to make the write conditional, so only one attempt can commit:
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Queue\InteractsWithQueue;
final class TriageTicket implements ShouldQueue, ShouldBeUnique
{
use Dispatchable, InteractsWithQueue, Queueable;
public int $timeout = 90;
public int $tries = 3;
public array $backoff = [10, 60];
public int $uniqueFor = 600;
public function __construct(public readonly int $ticketId) {}
public function uniqueId(): string
{
return (string) $this->ticketId;
}
public function handle(TicketClassifier $classifier): void
{
$ticket = Ticket::findOrFail($this->ticketId);
if ($ticket->triaged_at !== null) {
return;
}
$result = $classifier->classify($ticket->body);
$updated = Ticket::whereKey($ticket->id)
->whereNull('triaged_at')
->update([
'department' => $result->department->value,
'priority' => $result->priority->value,
'triaged_at' => now(),
]);
if ($updated === 1) {
TicketTriaged::dispatch($ticket->id);
}
}
}
ShouldBeUnique prevents a second copy of the job from being queued until the first one finishes processing or fails all its attempts (or uniqueFor expires). The whereNull('triaged_at') condition decides which attempt wins if two run anyway. The follow-up event fires only for the attempt that actually wrote the row. A retry may still pay for a second API call, but the row and the event happen only once.
Structured output and validation
Free text is a poor interface between a model and code. For classification and extraction, use structured outputs: pass a JSON Schema in output_config.format and the model returns a text block containing JSON that matches it. Forcing a tool call with tool_choice is the older workaround for getting structured data, and the newest models (e.g. Claude Opus 5.5) reject a forced tool_choice with a 400. The documentation lists cases where the output does not match the schema: a refusal (stop_reason: refusal), a response cut off by max_tokens, and enum values returned with different capitalization, which is why the code validates the result.
enum Department: string
{
case Billing = 'billing';
case Technical = 'technical';
case Sales = 'sales';
}
enum Priority: string
{
case Low = 'low';
case Normal = 'normal';
case Urgent = 'urgent';
}
final class TicketClassifier
{
public function __construct(private readonly AnthropicClient $client) {}
public function classify(string $body): TicketClassification
{
$values = fn (array $cases): array => array_map(fn ($c) => $c->value, $cases);
$response = $this->client->messages([
'max_tokens' => 256,
'system' => 'Classify the customer support ticket by department and priority.',
'messages' => [['role' => 'user', 'content' => $body]],
'output_config' => [
'format' => [
'type' => 'json_schema',
'schema' => [
'type' => 'object',
'properties' => [
'department' => ['type' => 'string', 'enum' => $values(Department::cases())],
'priority' => ['type' => 'string', 'enum' => $values(Priority::cases())],
],
'required' => ['department', 'priority'],
'additionalProperties' => false,
],
],
],
]);
$text = collect($response['content'])->firstWhere('type', 'text')['text'] ?? '';
$output = (array) json_decode($text, true);
$department = Department::tryFrom(strtolower((string) ($output['department'] ?? '')));
$priority = Priority::tryFrom(strtolower((string) ($output['priority'] ?? '')));
if ($department === null || $priority === null) {
throw new InvalidModelOutput('ticket_classification', $response);
}
return new TicketClassification($department, $priority, $response['usage']);
}
}
The enums are the single source of allowed values: they generate the schema and they validate the answer. Values are lowercased before tryFrom(), because capitalization of enum values isn't guaranteed. An invalid answer raises an exception, the job retries, and after the last attempt the ticket stays unclassified and the job lands in failed_jobs. Someone will see an unclassified ticket there, while a guessed value would go unnoticed.
I saw how confident a wrong answer can look when I tested small local models on PHP tasks. Both 4B models wrote an incorrect NIP (Polish tax ID) validation and presented it without any hesitation. A larger 26B model got it right, 6 out of 6 test cases, at 33 tok/s. One of the models also ignored the setting that turns "thinking" off and used up the whole 400-token limit on reasoning, so part of the answers came back empty with a normal HTTP status. The API reports no error then, so the code has to catch that case itself. The same goes for the tool loop below, which can return empty text when stop_reason is max_tokens or refusal.
Tool calls run with the user's permissions
When the model may call application functions (order status, refund calculation), PHP has an advantage: the tools are existing services with existing authorization. The loop below follows the Messages API protocol. The model responds with stop_reason: tool_use, the application runs the tools and sends back tool_result blocks, and this repeats until the model returns text.
public function answer(User $user, string $question): string
{
$messages = [['role' => 'user', 'content' => $question]];
for ($step = 0; $step < 5; $step++) {
$response = $this->client->messages([
'max_tokens' => 1024,
'tools' => $this->tools,
'messages' => $messages,
]);
if ($response['stop_reason'] !== 'tool_use') {
return collect($response['content'])
->where('type', 'text')
->pluck('text')
->implode("\n");
}
$messages[] = ['role' => 'assistant', 'content' => $response['content']];
$results = [];
foreach ($response['content'] as $block) {
if ($block['type'] !== 'tool_use') {
continue;
}
$results[] = [
'type' => 'tool_result',
'tool_use_id' => $block['id'],
'content' => json_encode($this->runTool($user, $block['name'], $block['input']), JSON_THROW_ON_ERROR),
];
}
$messages[] = ['role' => 'user', 'content' => $results];
}
throw new ToolLoopLimitExceeded($step);
}
private function runTool(User $user, string $name, array $input): array
{
return match ($name) {
'get_order_status' => $this->orders->statusFor($user, (string) ($input['order_id'] ?? '')),
default => ['error' => "Unknown tool: {$name}"],
};
}
The loop needs a step limit. Without it, a model that keeps requesting tools loops until the timeout and every iteration is billed.
Arguments from the model are untrusted input, like a form field. statusFor($user, ...) scopes the query to the user's own orders, so this tool won't return another customer's order, even if someone in the conversation asks the model to look.
Tools that change state (refunds, cancellations) shouldn't execute straight from the model's request. Let the tool prepare a proposal and require a confirmation outside the model: a button, a second request, a person.
Retrieval in Postgres with pgvector
Retrieval-augmented generation does not need a separate vector database if you already run Postgres. The expensive part, computing embeddings, happens when documents are indexed. At query time it's one embedding call and one SQL query.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE document_chunks (
id bigserial PRIMARY KEY,
document_id bigint NOT NULL REFERENCES documents (id) ON DELETE CASCADE,
content text NOT NULL,
embedding vector(1536) NOT NULL
);
CREATE INDEX document_chunks_embedding_idx
ON document_chunks USING hnsw (embedding vector_cosine_ops);
$embedding = Http::withToken(config('services.openai.key'))
->timeout(15)
->post('https://api.openai.com/v1/embeddings', [
'model' => 'text-embedding-3-small', // 1536 dimensions, matches the column
'input' => $question,
])
->throw()
->json('data.0.embedding');
$vector = '[' . implode(',', $embedding) . ']';
$chunks = DB::select(
'SELECT id, content, embedding <=> ?::vector AS distance
FROM document_chunks
ORDER BY embedding <=> ?::vector
LIMIT 5',
[$vector, $vector],
);
$relevant = array_filter($chunks, fn (object $c): bool => $c->distance <= $maxDistance);
<=> is cosine distance, and the hnsw index with vector_cosine_ops serves exactly this ordering. I filter the cutoff in PHP after LIMIT to see the distances of rejected chunks. A SQL WHERE on the distance, with the same ORDER BY … LIMIT, would use the index too.
$maxDistance has no universal default. It depends on the embedding model, the language and the length of the chunks. You have to measure it. Prepare a few dozen questions with known relevant fragments, run them at several cutoffs and record how many relevant fragments come back and how many irrelevant ones come with them. Repeat the measurement when you change the embedding model, because distances from different models are not comparable. Changing the model also means re-indexing all documents.
When nothing passes the cutoff, tell the user. Calling the model with an empty context invites an answer from its general knowledge, which is exactly what retrieval was supposed to prevent.
What to measure
Record every call in a table or a metrics system with these fields: feature name, model, usage.input_tokens, usage.output_tokens, latency, and outcome (success, invalid output, API error). That covers the questions people actually ask:
- Cost per feature. Cost scales with tokens. A long system prompt sent with every call often dominates the bill, and grouping by feature shows where.
- Latency percentiles (p50, p95). The average hides the slow calls, and those are the ones hitting timeouts.
- Invalid output rate. Alert when it moves away from its own baseline. A jump usually means a prompt change, a different input distribution or a new model version.
- Queue wait time for LLM jobs. It shows that workers are falling behind before users notice.
If prompts contain personal or financial data, log identifiers and counts. Full prompt text stays out of the logs.
When a separate Python service makes sense
A separate service is justified when you run your own models (fine-tuning, local inference), when you need libraries that exist only in Python, or when a team maintains ML pipelines independently of the product. Even then, keep it narrow: a defined input, a defined output, no business logic. The domain rules (who may see which order, what a refund is) should stay where they already are. Moving them to a new service so that a model can call them adds a network boundary, a second deployment and a second copy of the rules.
Before the first production deploy
- Job
$timeoutcovers the whole client call with retries and stays belowretry_after. - Results are written conditionally, and only the attempt that wrote triggers side effects.
- The tool loop has a step limit, and state-changing tools wait for confirmation.
- The retrieval cutoff comes from a measurement on your own data.