After reading this post, check out this one where I show you how to integrate Cloudflare AI Workers, which gives you 10 000 free neurons every day, with LiteLLM and OpenWebUI:
Also also check out this one where I show how to get a Google Gemini API key with free daily API calls and integrate it with LiteLLM and OpenWebUI:
Explaning Azure AI Foundry and AI Hubs
From URL: https://learn.microsoft.com/en-us/azure/ai-foundry/what-is-azure-ai-foundry
Check out this video to learn more: https://www.youtube.com/watch?v=AH1d4kYI7Ac
Azure AI Foundry supports two types of projects: a hub-based project and a Foundry project. In most cases, you want to use a Foundry project.
- A Foundry project is built on an Azure AI Foundry resource. This project type allows for a simple setup, access to agents, and industry leading models from OpenAI, Mistral, Meta, and more.
- You will end up with 2 resources when you create a Foundry project: The Foundry Resource and a Project within the Foundry resource (see further down below). You can create more projects within the Foundry resource.
- A hub-based project is hosted by an Azure AI Foundry hub. If your company has an administrative team that created a hub for you, you can create a project from that hub. If you’re working on your own, you can create a project and a default hub is automatically created for you.
We want to use Foundry Projects going forward:
Create an AI Foundry Resource
You can do it from https://portal.azure.com or https://ai.azure.com/ I prefer using https://portal.azure.com but you can use any one of the 2.
Using Azure Portal
When creating a new AI Foundry, you will be asked for an AI Foundry name and a Project name, so you will end up with 2 resources.
Resource 1 is your AI Foundry and when you click on the AI Foundry name, you will see the Project you created when creating the AI Foundry.
Resource 2 is your Project that lives inside that Foundry resource. You can create more projects if you want to.
Using Azure AI Portal
When creating a Foundry Resource from https://ai.azure.com/ just click on “Create New” in the top left corner.
You will see 2 resources when in https://ai.azure.com just like you would when in the Azure Portal. And if you created your AI Foundry in https://portal.azure.com it will show up here (just give it a few minutes).
The green arrow is your Project, the orange arrow is your Foundry.
You want to click on your Project, the green arrow, in order to create any resource such as OpenAI resources. When you click on your Project, just click as in the below screenshot to deploy any LLM model.
I will deploy a ChatGPT 5 mini model so long because we need the endpoints and API key.
Take Note!
Not all models can be deployed into your Foundry resource. At time of writing, GPT 5-mini does not have any quoata available in EastUS, only in EastUS 2, so Azure will automatically create a new Foundry resource with a default Project in cases like this. So you will see 2 Foundries now, the one you created and the one Azure automatically created. This can be quite confusing.
When you choose a model that can be deployed into a region where your Foundry resource is located, you will see a page like this where you can select your already existing Foundry resource.
On your Models and Endpoint page within https://ai.azure.com you can see under which project your model was delployed. If you are in the project in which your model was deployed, it will show “This resource” instead of the project name.
Usage Metrics
When you click on your model name above you will be able to see usage metrics. It can take an hour or so for the metrics to appear.
You will also be able to see your costs within your Azure Subscription. Remember to set up Budget Alerts.
Using Azure OpenAI with OpenWebUI
This post will cover how to use Azure OpenAI with OpenWebUI and LiteLLM (without Ollama) in order to connect to an OpenAI model hosted in Azure Foundry.
OpenWebUI Docker Compose
version: '3.8'
services:
openwebui:
image: ghcr.io/open-webui/open-webui:main
container_name: lc-open-webui
restart: unless-stopped
ports:
- "8090:8080"
env_file: stack.env
volumes:
- open-webui:/app/backend/data
networks:
- lc_network
litellm-proxy:
image: ghcr.io/berriai/litellm:main-latest
container_name: lc-litellm-proxy
restart: unless-stopped
ports:
- "4000:4000"
env_file: stack.env
depends_on:
- litellmproxy_db
environment:
DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@litellmproxy_db:5432/${POSTGRES_USER}
command: ["--config", "/app/config.yaml", "--detailed_debug", "--num_workers", "4"]
volumes:
- /opt/litellmproxy/litellm_config.yaml:/app/config.yaml
networks:
- lc_network
litellmproxy_db:
image: postgres:17.2-alpine3.21
container_name: lc-postgresql
restart: unless-stopped
env_file: stack.env
shm_size: 96mb
volumes:
- postgresdatalitellmproxy:/var/lib/postgresql/data
networks:
- lc_network
volumes:
postgresdatalitellmproxy:
open-webui:
networks:
lc_network:
driver: bridge
You will also need the following environment variables as we are running in Docker (I am using Portainer):
AZURE_API_BASE=<SEE BELOW SCREENSHOT>
AZURE_API_KEY=<API KEY. SEE BELOW SCREENSHOT>
AZURE_API_VERSION=<FROM URL. SEE BELOW SCREENSHOT>
AZURE_MODEL=<SEE BELOW>
POSTGRES_PASSWORD=<PASSWORD OF YOUR CHOOSING>
POSTGRES_USER=litellm
LITELLM_MASTER_KEY=<ANY RANDOM VALUE STARTING WITH sk->
LITELLM_SALT_KEY=<CAN BE ANYTHING BUT NEVER CHANGE IT ONCE SET>
WEBUI_SECRET_KEY=<PASSWORD OF YOUR CHOOSING>Here is what the important environment variables above mean:
- AZURE_API_BASE: Your Azure OpenAI deployment URL as in the image below.
- AZURE_API_KEY: This is just the API key as in the image below.
- AZURE_API_VERSION: If you look at the image below, you will see the URL contains an API version. Use that API version.
- AZURE_MODEL: LiteLLM requires the AZURE_MODEL variable to be in the format “azure/DEPLOYMENT_NAME” when using Azure OpenAI deployments. So in our case this value will be “azure/gpt-5-mini”.
- LITELLM_MASTER_KEY: Must start with “sk-” and is the key you will use when logging in as admin and for the REST APIs.
So using the above screenshot, here is what my environment variables look like. I also have deepseek and normal OpenAI in here, but just ignore them if you don’t need them. (Take note the API keys and such have been redacted or replaced)
- AZURE_API_BASE=https://xxxxxx-xxxxx-eastus2.cognitiveservices.azure.com/
- AZURE_API_KEY=CSPxxxxxx
- AZURE_API_VERSION=2025-04-01-preview
- AZURE_MODEL=azure/gpt-5-mini
- POSTGRES_PASSWORD=xxxxxx
- POSTGRES_USER=litellm
- LITELLM_MASTER_KEY=sk-xxxxxxxxxxx
- LITELLM_SALT_KEY=never_change_this_key_123
- WEBUI_SECRET_KEY=xxxxxx
- OPENAI_MODEL=gpt-4o
- OPENAI_API_KEY=sk-proj-xxxxxx
- DEEPSEEK_API_KEY=sk-xxxxxxxx
- DEEPSEEK_MODEL_REASONER=deepseek/deepseek-reasoner
- DEEPSEEK_MODEL_CHAT=deepseek/deepseek-chat
- DEEPSEEK_MODEL_CODER=deepseek/deepseek-coder
LiteLLM Config
In the docker compose file you will see this line: /opt/litellmproxy/litellm_config.yaml:/app/config.yaml
This means that LiteLLM is looking for the litellm_config.yaml file on the host on which you have docker installed. This is what that file must look like:
model_list:
- model_name: azure-gpt-5-mini
litellm_params:
model: os.environ/AZURE_MODEL
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY
api_version: os.environ/AZURE_API_VERSION
- model_name: openai-gpt-4o
litellm_params:
model: os.environ/OPENAI_MODEL
api_key: os.environ/OPENAI_API_KEY
- model_name: deepseek-reasoner
litellm_params:
model: os.environ/DEEPSEEK_MODEL_REASONER
api_key: os.environ/DEEPSEEK_API_KEY
- model_name: deepseek-chat
litellm_params:
model: os.environ/DEEPSEEK_MODEL_CHAT
api_key: os.environ/DEEPSEEK_API_KEY
- model_name: deepseek-coder
litellm_params:
model: os.environ/DEEPSEEK_MODEL_CODER
api_key: os.environ/DEEPSEEK_API_KEY
This file is used to define all your models.
The parameters pretty much just read the environment variables we defined as part of the docker-compose file or in Portainer.
Endpoints
Several endpoints will be accessible after deploying this compose file (replace the IP and port with your host):
| LiteLLM Swagger Docs | http://192.168.1.109:4000 |
| LiteLLM User Interface | http://192.168.1.109:4000/ui (log in with the LITELLM_MASTER_KEY) |
| LiteLLM Model Cost Map (not local but important) | https://models.litellm.ai/ |
| OpenWeb-UI User Interface | http://192.168.1.109:8090/ |
Model Pricing
In LiteLLM under models you will see the gpt-4o model with pricing. This is the default pricing and is a bit off from the actual pricing.
Track spend per Model using LiteLLM
Here is the Azure pricing for the GPT models: https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/
As of writing this, there is no pricing for the GPT 5 models on this page yet. But the pricing will correspond with the “Model Version” in the Azure Foundry screenshot above. I’m using 4o as an example.
I did manage to find pricing for GPT 5 on Azure on this page, but it will change soon:
https://azure.microsoft.com/en-us/blog/gpt-5-in-azure-ai-foundry-the-future-of-ai-apps-and-agents-starts-here/
Link OpenWebUI with LiteLLM
We need to tell OpenWeb-UI how to use the LiteLLM models we have defined in our yaml file:
Click on Admin Panel
Click on Settings, Connections, then the plus Icon to add a new connection
The URL will be the litellm service name as defined in the docker compose file. In our case it is “litellm-proxy” and the internal port is 4000.
The Key is the “sk-” value you defined in the “LITELLM_MASTER_KEY” environment variable. Include the entire key, including “sk-“
Then click the refresh button, and you will see the green message up top.
Open a new chat, and you will see all the models listed in the dropdown. That little icon tells us it is an external model.
Platform API Key URLs
- Deepseek API Key: https://platform.deepseek.com/
- OpenAI: https://platform.openai.com/
