How to Send Images to an LLM API: A Practical Vision Guide for Developers
Modern LLM APIs can do far more than process text. Vision-capable models can inspect screenshots, read receipts, analyze charts, compare product photos and extract information from scanned documents. For developers, the important part is knowing how to package the image correctly and choose a model that supports image input.
With an OpenAI-compatible API format, the basic pattern is straightforward: send a normal chat request, but replace the user message's plain text content with an array containing both text and image data. CostRouter provides an OpenAI-compatible gateway at https://costrouter.ai/v1, allowing supported GPT, Claude, Gemini and other models to be accessed through one API key and one billing account.
This guide focuses on image understanding rather than image generation. The goal is to send an existing image to an LLM and ask the model to understand, classify, extract or reason about what it sees.
How Image Input Works in an LLM API Request
A normal text-only Chat Completions request may contain a user message like this:
{
"role": "user",
"content": "What is shown in this image?"
}When an image is added, the content field becomes an array of typed content blocks. For OpenAI-compatible vision routes, the request commonly follows this structure:
{
"role": "user",
"content": [
{
"type": "text",
"text": "Read this receipt and tell me the total."
},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/receipt.jpg"
}
}
]
}The text explains what the model should do, while the image block provides the visual input. A vision-capable model then converts the image into an internal representation that can be processed together with the text prompt.
Not every LLM accepts images, so model capability should always be checked before deployment. Current OpenAI models such as GPT-5.6 Sol support image input, while Google's Gemini 3.7 Flash supports text, image, video, audio and PDF input.
Sending an Image Through the CostRouter API
CostRouter uses an OpenAI-compatible Base URL:
https://costrouter.ai/v1A basic vision request can therefore be sent to the Chat Completions endpoint with a CostRouter API key.
cURL Example With a Hosted Image
curl -X POST "https://costrouter.ai/v1/chat/completions" \
-H "Authorization: Bearer YOUR_COSTROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-5.6-sol",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe this image and identify any important details."
},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/image.jpg"
}
}
]
}
]
}'The same overall integration can be reused when switching between compatible models. In practice, you should verify the selected model's endpoint and image-input capability on the live CostRouter Models page before production deployment. CostRouter currently exposes supported models through one API key and provides request-level usage and cost visibility.
Hosted Image URL or Base64: Which Should You Use?
There are two common ways to provide an image to a vision API: send a URL pointing to the image or place the image bytes directly inside the request using Base64.
The correct choice depends primarily on where the image is stored and whether it can safely be accessed through a URL.
Use an Image URL for Files That Are Already Hosted
If an image already exists on a CDN, object-storage service or web server, sending its URL is usually the simplest solution. The JSON request remains relatively small because it contains only the URL rather than the full image data.
This works well for product catalogs, publicly accessible screenshots, website images and files stored behind temporary signed URLs.
However, the model provider must be able to retrieve the image. Expired links, authentication requirements, network restrictions or incorrect permissions can cause the request to fail.
Use Base64 for Local or Private Images
Base64 is more appropriate when the image originates from a local file, private upload or internal application and should not be exposed through a public URL.
A Base64 image can be converted into a data URL and passed directly in the image_url field:
data:image/jpeg;base64,/9j/4AAQSkZJRgABAQ...A simple Python example looks like this:
import base64
import requests
import os
def image_to_data_url(path):
with open(path, "rb") as f:
encoded = base64.b64encode(f.read()).decode("utf-8")
return f"data:image/jpeg;base64,{encoded}"
response = requests.post(
"https://costrouter.ai/v1/chat/completions",
headers={
"Authorization": f"Bearer {os.environ['COSTROUTER_API_KEY']}",
"Content-Type": "application/json"
},
json={
"model": "gpt-5.6-sol",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Extract the main information from this receipt."
},
{
"type": "image_url",
"image_url": {
"url": image_to_data_url("receipt.jpg")
}
}
]
}
]
}
)
print(response.json())Base64 increases request size because the entire file is transferred inside the API payload. For relatively small private images this is usually acceptable, but large image collections are better handled through hosted files, provider file APIs or a dedicated storage workflow.
Choosing the Right Vision Model
The request format may be similar, but vision models can behave very differently on the same image.
Some models are stronger at reading dense documents, while others are better at visual reasoning, screenshots, charts or high-volume image processing. Latency and price can also differ substantially, so selecting the most expensive model for every image is rarely the most efficient strategy.
As of August 2026, several relevant models are available through CostRouter:
| Model | CostRouter Input Price | Typical Role |
|---|---|---|
| Gemini 3.7 Flash | $0.15 / 1M tokens | High-volume multimodal tasks and fast visual analysis |
| GPT-5.6 Sol | $1.00 / 1M tokens | Difficult visual reasoning and professional workflows |
| Claude Opus 5 | $1.50 / 1M tokens | Detailed documents, charts and careful analysis |
The prices above reflect currently listed Value routes and can change as routing and provider pricing change. CostRouter's live Models page should be treated as the source of truth before estimating production costs.
This pricing difference also shows why a multi-model API can be useful. A simple screenshot classification task may not require a flagship model, while a complicated financial chart or technical diagram may justify stronger reasoning. Instead of rebuilding the API integration, developers can select a different compatible model according to the workload.
Image Resolution Directly Affects Token Cost
Images are not free input.
Vision models typically divide an image into regions or patches and convert visual information into representations that can be processed by the language model. A larger or more detailed image usually requires more visual tokens or greater image-processing cost than a smaller image.
The exact calculation depends on the model provider. This means that a 4,000-pixel phone photo may cost considerably more to process than a resized version, even when both contain the same information.
For production applications, resize images to the lowest resolution that still preserves the information the model needs. If the model only needs to read a total from the bottom of a receipt, crop that region instead of sending the entire photograph.
Cropping can improve both cost and accuracy because irrelevant visual content is removed before inference.
Sending Multiple Images in One Request
Vision APIs are also useful when the task requires comparison rather than interpretation of a single image.
For example, you might ask an LLM to compare two dashboards, inspect before-and-after product photos or determine which of several charts shows stronger growth. The request simply includes several image blocks.
{
"role": "user",
"content": [
{
"type": "text",
"text": "Compare these two dashboards and summarize the most important differences."
},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/dashboard-a.png"
}
},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/dashboard-b.png"
}
}
]
}The practical image limit depends on the selected provider and model rather than the API pattern itself. Each additional image also adds processing cost, so sending dozens of full-resolution images in a single request is rarely ideal.
When processing large collections, batch images intelligently or retrieve only the images relevant to the current question.
Turn Image Understanding Into Structured Data
One of the most useful production applications of vision models is not image description but structured extraction.
Suppose an application receives thousands of receipts. Instead of asking the model to write a paragraph describing each receipt, the API can be instructed to return fields such as merchant, date, subtotal, tax and total.
A desired response might look like this:
{
"merchant": "Example Store",
"date": "2026-08-22",
"subtotal": 38.20,
"tax": 3.06,
"total": 41.26
}Vision input can be combined with structured outputs or tool calling when supported by the selected model. This pattern is useful for invoices, receipts, forms, product labels, screenshots and other semi-structured documents. OpenAI's current GPT-5.6 models support both image input and structured outputs, making this type of workflow practical without adding a separate extraction model.
Building Multimodal RAG With Images
Traditional RAG systems usually retrieve text chunks from a vector database. The problem is that many real documents contain information that does not exist as plain text.
Annual reports include charts. Research papers contain diagrams. Manuals include screenshots. Scanned documents may contain pages that OCR cannot reliably interpret.
A multimodal RAG pipeline treats those visual elements as retrievable knowledge rather than ignoring them.
Convert Visual Content Into Searchable Information
One straightforward approach is to analyze each image during indexing and create a text description of the chart, screenshot or scanned page. That description can then be embedded using the same vector database as the rest of the document.
When a user asks a question, the system retrieves the matching description and follows the reference back to the original image. The image is then sent to the vision model together with the user's question.
A more advanced architecture uses multimodal embeddings, placing text and images inside a shared embedding space. This allows a text query to retrieve an image directly without first reducing it to a text summary.
The first approach is easier to integrate with existing RAG systems. Native multimodal retrieval can preserve more visual information but requires an embedding model and vector infrastructure that support it.
Practical Ways to Reduce Vision API Costs
Vision workloads can become expensive when applications process large numbers of screenshots, document pages or product images. The easiest optimization is therefore to reduce unnecessary visual input before trying to reduce model quality.
Resize oversized images, crop irrelevant regions and avoid sending identical images repeatedly. When the workload is simple, use a lower-cost vision model; reserve more capable models for cases where the cheaper model fails.
CostRouter can make this testing process easier because multiple supported model families use one account and one API key. Its models catalog shows current token pricing, while Usage Logs provide request-level usage and cost visibility. The platform also states that discounted routes maintain the requested model rather than silently substituting a weaker model.
This makes it possible to evaluate the same receipt, screenshot or chart across several models and compare the real cost and output quality before choosing a production route.
Where General-Purpose Vision Models Are Not the Best Choice
Vision LLMs are flexible, but they should not automatically replace every specialized image-processing system.
For extremely dense OCR where every character must be exact, a dedicated OCR pipeline may still be more reliable. Real-time video processing also requires different infrastructure because sending every video frame through a chat endpoint creates unnecessary latency and cost.
Image generation is another separate task. An image-understanding request asks the model to interpret an existing image, while image-generation or image-editing endpoints create or modify visual content. The two workflows should not be treated as interchangeable.
Frequently Asked Questions
Can I Send an Image to an LLM API?
Yes. Vision-capable models accept image input together with text. In an OpenAI-compatible Chat Completions request, this is commonly done by adding an image_url block to the user message's content array.
Should I Use Base64 or an Image URL?
Use a hosted URL when the file already exists online and can be reliably accessed by the provider. Use Base64 for local or private files that you do not want to expose through a separate public URL.
Do Images Consume AI Tokens?
Yes. Visual inputs contribute to the cost of the request. The exact image-token calculation differs between providers, but higher resolution and additional images generally increase processing cost.
Which LLM Is Best for Image Understanding?
There is no single best model for every image task. Gemini models can be attractive for high-volume multimodal processing, while GPT and Claude models are often useful when images require more complex reasoning. Testing the same production images across several models is more reliable than choosing based only on benchmarks.
Can I Use One API Key for GPT, Claude and Gemini Vision Models?
CostRouter provides one API key and one balance for supported models across multiple providers. Developers can copy the exact model ID from the live catalog and use the compatible endpoint supported by that model.
Conclusion
Sending an image to an LLM is technically simple: combine the user's instruction with an image input and send the request to a vision-capable model. The more important production decisions are choosing the right model, controlling image resolution and understanding how visual input affects cost.
For applications that use several model providers, CostRouter offers a practical way to test and route supported vision models through one API key while keeping token usage and request-level costs visible. This makes it easier to choose the right balance of vision quality, latency and API price for each workload.