How Does CompletionPrism™ Failover Work When a Model Goes Down?

Error rates and response times are tracked per provider and model. A provider that fails or slows is removed from routing in seconds, then returned in steps.

Nothing Your Customers Can See

CompletionPrism™ watches error rates, timeouts and response times for every approved provider and model, all the time.

When Something Goes Wrong

• Outage detection: a provider that stops answering is taken out of routing for every tenant within seconds
• Degradation detection: rising errors and slow responses are caught before they become an outage users feel
• Automatic failover: every request the provider would have taken goes to the next best approved model on cost to finish
• Safe return: a recovered provider is brought back in steps, only after it has stayed healthy

For Employees and Products Alike

Employees keep working in the same place. Inside a vendor's product, customers keep getting answers. An optional availability and SLA tier supports vendors who want contractual failover commitments. See the Embedded License.