Routing
Every request to Pulze will flow through our Engine, which depending on the settings configured, will impact the model used, the stored data, etc.
Examples of this are:
- Selected models for an App
- Weights (optimize for cost, for speed…)
- Policies (modify the privacy, use a Prompt, retry on failure…)
- …
For more information on how to perform requests, please see our API Reference
Built on top of that, however, there are two different modes of operation for any given app:
SMART
SMART stands for Semantic Model Allocation Routing Tool. In short, it’s a system developed by Pulze which, based on the prompt (i.e. the text you send) and the different parameters, it allocates a model. Semantic, in this context, means “related to the contents of the request”.
SMART is enabled by default for all new apps, and we encourage you to try it, in combination with the weights.
When you request a model such as pulze, pulze-v0, … (i.e. one of our routing models) the request will be redirected to the best selected model, and if it
fails it will try the second best, then the third best… This is configurable by this policy.
When you request a specific model such as claude-2 from Anthropic, or gpt-4 from OpenAI, in case of failure the request will end. That is, no other models will be tried.
To fall back to other models on failure, use the manual mode.
How to use it
In your App, access the “Models” tab and change the mode
Switch between SMART and Manual modes
After that, select the models you would like the router to consider
Select the models to be used
Manual
The manual mode is designed to give you full control over which models are used, and what to do when a specific model fails.
💡 In Manual mode, you can request any model, as long as you have access to it. It is not necessary for the model to be part of your failover chain.
You might be aware that there is a policy which, when enabled, will jump to a different model when a model fails. This is only true, however, when you request one of our Synthetic Models (see the example code here.)
How to use it
In your App, access the “Models” tab and change the mode
Switch between SMART and Manual modes
After that, choose the models you’d like the system to consider.
Switch between SMART and Manual modes
If any request fails, or if the the prompt doesn’t fit in the model’s context window, the next model will be tried.
Feedback
Was this page helpful?