Migrating from OpenAI to ZC Technologies' Inference Exchange can reduce your LLM costs by up to 70% without altering your application's behavior. This guide will walk you through the process of replacing OpenAI's API with ZC Technologies' OpenAI-compatible endpoint, focusing on the /v1/chat/completions endpoint. By changing a single URL, you can seamlessly switch to a more cost-effective solution, maintaining the same functionality and performance.
To illustrate the process, let's start with a typical integration of the OpenAI API. The following code demonstrates a basic request to the /v1/chat/completions endpoint:
import requests
headers = {
'Authorization': 'Bearer YOUR_OPENAI_API_KEY',
'Content-Type': 'application/json',
}
payload = {
'model': 'gpt-3.5-turbo',
'messages': [{'role': 'user', 'content': 'Hello, how are you?'}],
}
response = requests.post('https://api.openai.com/v1/chat/completions', headers=headers, json=payload)
print(response.json())
To switch to ZC Technologies' Inference Exchange, you only need to update the URL and your API key. Here's the modified code:
import requests
headers = {
'Authorization': 'Bearer YOUR_ZC_TECHNOLOGIES_API_KEY',
'Content-Type': 'application/json',
}
payload = {
'model': 'qwen2.5:32b',
'messages': [{'role': 'user', 'content': 'Hello, how are you?'}],
}
response = requests.post('https://zcx.zctechnologies.org/v1/chat/completions', headers=headers, json=payload)
print(response.json())
The cost savings are significant. For instance, at the Pro tier, ZC Technologies' Inference Exchange offers 12.0M tokens for $499 per month, which translates to $42 per 1M tokens. This is a substantial reduction compared to the pricing of other providers, offering a cost-effective solution for your LLM needs.
Switching from OpenAI to ZC Technologies' Inference Exchange is a straightforward process that can lead to significant cost savings. By changing the API endpoint and your API key, you can continue to leverage the power of LLMs while reducing your expenses. For more information on our plans and pricing, visit our website.