​

Documentation on Labeling at Pulze.ai

​

Understanding Labels

Labels are a crucial tool to manage and track specific requests made to machine learning models. They offer numerous benefits, including the ability to filter logs, track feature usage, and identify potential issues.

​

Compatibility of Labels with Different Models

Labels are versatile and can be attached to any request, irrespective of the type of request or model being used, be it open-source or proprietary models such as Anthropic, AlephAlpha, or OpenAI.

​

Extensive Label Filtering on Pulze.ai Platform

Pulze.ai enhances the utility of labels by offering extensive filtering options on its platform. On the Pulze.ai platform under the ‘Logs’ section, you can search for specific prompts and responses or filter based on labels.

​

Detailed Log Analysis with Label and App Filters

Pulze.ai enables detailed log analysis by returning only requests that contain all specified labels. Moreover, it allows filtering by specific apps and their corresponding API Keys, thus offering even more control over your data.

​

How to set Custom Labels

To set custom labels, please visit the Pulze-labels docs here

​

How to use Labels with Examples

​

Creating a Chat Response

Below is a simple example of creating a chat response using the Pulze.ai API:

chat_response = openai.chat.completions.create(
  model="pulze-v0",
  messages=[{
    "role": "user",
    "content": "Say Hello World!"
  }],
)

​

Adding Custom Labels

Adding custom labels to your requests is straightforward. These labels are added as headers and are converted to a JSON string. These labels will then appear in your logs, making it easier for you to filter and locate specific requests.

Below is an example of adding custom labels to a request:

import json

labels = {"mode": "internal", "type": "testing", "foo": "bar"}

response = openai.chat.completions.create(
  model="pulze-v0",
  messages=[{"role": "user", "content": "Say Hello World!"}],
  headers={"Pulze-Labels": json.dumps(labels)},
)

In this example, the labels mode, type, and foo are added with the values internal, testing, and bar respectively. These labels can then be used to filter and analyze your requests.

Just like you can add user IDs in OpenAI’s API, you can add similar identifiers using custom labels in Pulze.ai. This allows for effective monitoring and tracking of user activity, helping to identify any potential issues or abuse. With custom labels, Pulze.ai provides a flexible and powerful way to manage and understand your requests.

​

Retrieving and Understanding Labels on Pulze.ai Platform

The process of retrieving labels and the corresponding information is streamlined on the Pulze.ai platform. To access this feature, simply log into the Pulze.ai platform and navigate to the ‘Logs’ section from the left-hand menu.

The logs provide crucial details about each request such as:

• Timestamp
• Model used
• Latency
• Cost
• Score
• Prompt
• Response

To reveal more in-depth information about a specific request and its response, click on the log row. The request details may contain parameters and values like:

{
  "n": 1,
  "echo": false,
  "stop": "",
  "user": "",
  "model": "j2-ultra",
  "top_p": 1,
  "labels": {
    "env": "dev",
    "mode": "internal",
    "type": "modelmonitor"
  },
  "prompt": "0",
  "stream": false,
  "suffix": "",
  "best_of": 1,
  "headers": { /* various header details */ },
  "weights": {
    "cost": 0.4,
    "latency": 0.4,
    "quality": 0.4
  },
  "logprobs": null,
  "messages": [
    {
      "role": "user",
      "content": "0"
    }
  ],
  "provider": "ai21labs",
  "logit_bias": null,
  "max_tokens": 1,
  "temperature": 1,
  "presence_penalty": 0,
  "frequency_penalty": 0
}

And the response data might look like:

{
  "choices": [
    {
      "text": ".",
      "index": 0,
      "finish_reason": "length"
    }
  ],
  "created": 1689671271,
  "metadata": {
    "id": "a6373c30-5760-4c73-9b82-0bac7e7417e7",
    "costs": {
      "total_tokens": 0.000045,
      "prompt_tokens": 0.00003,
      "completion_tokens": 0.000015
    },
    "error": null,
    "model": "j2-ultra",
    "usage": {
      "total_tokens": 3,
      "prompt_tokens": 2,
      "completion_tokens": 1
    },
    "labels": {
      "env": "dev",
      "mode": "internal",
      "type": "modelmonitor"
    },
    "latency": 0.3625,
    "category": "Science & Research",
    "provider": "ai21labs",
    "max_tokens": 1,
    "status_code": 200,
    "temperature": 1,
    "cost_savings": {
      "total_tokens": 0.000135,
      "prompt_tokens": 0.00009,
      "completion_tokens": 0.000045
    }
  }
}

All these metadata points from both the request and response are easily filterable in the Pulze.ai platform, allowing you to effortlessly sift through your logs. Moreover, if a particular data point is a metric, you can find more details about it in the Monitoring section of the Pulze.ai documentation.