The KNN Router is an open-sourced intent-tuned LLM router designed to efficiently select the best language model (LLM) for a user’s query. This documentation provides an overview of how to deploy and utilize the KNN Router, as well as insights into its underlying technology.
The KNN Router uses k-nearest neighbors (KNN) to find semantically similar queries, allowing for precise model selection based on user input.
Weighted Scoring:
Each LLM associated with the nearest neighbors is scored based on a weighted average of distances, ensuring the most appropriate model is selected for each query.
Minimal Latency:
Built in Go, the KNN Router is optimized for low-latency responses, making it suitable for real-time applications.
Integration Flexibility:
The router can be integrated with various systems, including information retrieval systems, agents, and other LLMs.
Open Source:
Being open-sourced, the KNN Router is available for customization and enhancement by the community.
The KNN Router leverages the pulze-intent-v0.1 model, which is trained to select the most appropriate LLM based on user queries. The model and dataset can be accessed via the following links:
We encourage developers and enthusiasts to contribute to the KNN Router project. Your feedback and contributions help us improve the tool for everyone.For more information and to stay updated, visit our GitHub repository: KNN Router on GitHub.