OmniHai grows a backbone

OmniHai 1.5 is out! After 1.1 gave the library ears, 1.2 a voice, 1.3 the ability to step outside and browse the web, and 1.4 taught it to count the cost, 1.5 gives it a backbone.

Talking to a remote AI provider means talking to something that occasionally says no. Rate limits, a provider that is briefly down, a connection that drops halfway. None of that is your fault, and none of it should reach your users as a stack trace. Until now you had to manually wrap your own retry loop around every call. OmniHai 1.5 ships two ready-to-use resilience decorators so you never have to write that loop anymore.

Retry

Wrap any service in RetryingAIService and transient failures are retried for you. It triggers on an HTTP 429 rate limit, an HTTP 503 unavailable, and transient I/O, with exponential backoff and full jitter between attempts.

AIService resilient = new RetryingAIService(service); // 3 attempts, sensible defaults

That is the whole change. The wrapped service keeps its exact API, so every caller downstream stays the same. Need other numbers? Reach for the builder.

AIService tuned = RetryingAIService.newBuilder(service)
    .maxAttempts(5)
    .initialBackoff(Duration.ofSeconds(1))
    .maxBackoff(Duration.ofSeconds(20))
    .maxDuration(Duration.ofMinutes(1))
    .build();

It never retries a deterministic error. A bad request or an authentication failure would fail the same way on the second attempt, so retrying it only wastes time and tokens. You can override the condition with retryOn(...) when your case is different.

Failover

Retrying the same provider helps when the provider is merely busy. It does not help when the provider is down. For that there is FailoverAIService, which tries a primary service and then falls back to alternates in order, on those same transient failures.

@Inject @AI(apiKey = "#{keys.openai}")
private AIService gpt;

@Inject @AI(provider = ANTHROPIC, apiKey = "#{keys.anthropic}")
private AIService claude;

AIService resilient = new FailoverAIService(gpt, claude);
String response = resilient.chat("Explain the Jakarta EE security model.");

When OpenAI is rate limiting you, the call quietly lands on Anthropic instead. Your code asked one question and got one answer; which provider answered it is an operational detail, not an application concern.

Composing them

Both are pure decorators built on InterceptingAIServiceWrapper, so they wrap the entire service surface: chat, image, audio, moderation, synchronous and asynchronous alike. And because they are decorators, they compose. Retry each provider a few times before giving up on it and failing over to the next.

AIService resilient = new FailoverAIService(
    new RetryingAIService(gpt),
    new RetryingAIService(claude));

No CDI magic, no configuration file, no framework to buy into. Just constructors that take an AIService and return an AIService. Stack them in whatever order your situation asks for.

Streaming and partial results

Retrying a plain chat call is easy; you just call it again. Retrying a streaming call is not, because the first attempt may already have handed a dozen tokens to your consumer. A blind second attempt would replay the stream from the start and leave the consumer with a duplicated prefix. Rather than corrupt your output in silence, the decorator throws an AIStreamAbortedException, with the original failure as its cause. This exception is terminal and is never retried nor failed over.

If you do want a partially consumed stream re-attempted, hand in a ResettableConsumer as your token consumer. It carries a second handler next to the token handler, invoked right before each new attempt, so it can discard what it accumulated and let the retry start from a clean slate.

var response = new StringBuilder();
AIService resilient = new RetryingAIService(service);

resilient.chatStream("Explain the Jakarta EE security model.", ResettableConsumer.of(
    token -> response.append(token),           // append every token as it streams in
    (cause, attempt) -> response.setLength(0)  // a retry is starting; drop the partial output
));

The token handler is your normal streaming consumer. The reset handler receives the failure that triggered the re-attempt and the number of the attempt about to start, and clears the buffer so the fresh stream does not stack on top of the old one.

Refreshed default models

As always the per-provider default models have moved forward to the current generation.

Provider	Default model
OpenAI	gpt-5.6-terra
Anthropic	claude-sonnet-5
Google AI	gemini-3.5-flash
xAI	grok-4.5
Mistral AI	mistral-medium-3-5
Meta AI	muse-spark-1.1
Azure OpenAI	gpt-5.5

Meta moved the most. It retired the old Llama endpoint and now serves Muse Spark through the OpenAI-compatible Meta Model API at https://api.meta.ai/v1. OmniHai follows: the default is now muse-spark-1.1, and the provider rides the shared OpenAI text handler instead of its own. If you pinned the old Llama-4-Maverick-17B-128E-Instruct-FP8 model or the api.llama.com endpoint, update those.

This generation also changed the rules. The newest Claude and Fable models (Opus 4.7 and up, Sonnet 5 and up, Fable 5 and up) dropped the classic sampling knobs; send them a temperature or a legacy thinking budget and they answer with an HTTP 400. OmniHai now knows this through supportsSamplingParameters() and simply omits those fields for such models, steering them with the ReasoningEffort from 1.4 instead. Older models keep the legacy behavior. You set your options the same way you always did; OmniHai sends whatever the target model still accepts.

Getting 1.5

Non-Maven users: download the OmniHai 1.5 JAR and drop it in /WEB-INF/lib the usual way, replacing the older version if any. Maven users: update the version.

<dependency>
    <groupId>org.omnifaces</groupId>
    <artifactId>omnihai</artifactId>
    <version>1.5</version>
</dependency>

OmniHai still needs only Java 17 and Jakarta EE 10 or MicroProfile 7, with JSON-P required and CDI, EL and MP Config optional. No new dependencies; the resilience decorators are plain Java.

Give it a try

Wrap your existing service in a RetryingAIService, or chain a couple of providers behind a FailoverAIService, and watch the transient failures stop reaching your users. As always, feedback and contributions are welcome on GitHub. If you run into anything, open an issue. Pull requests are welcome too. :)