Groq
AIRun ultra-fast inference on open models like Llama, Qwen, and Whisper through the Groq API. Chat, vision, audio transcription, batches, and agentic Compound systems in one connector.
What This Integration Enables
Groq runs open models at very high speed, and covers chat, vision, audio transcription, batches, and its Compound agentic systems. The differentiator is latency: when a model call returns in a fraction of the usual time, it can sit inside an interactive flow without making a person wait. That speed moves AI from the batch tier into the live tier. A classification that was too slow to run during a chat can now run on every message. But a fast wrong answer is still wrong, so the fast steps need a layer that routes on the output and hands off to a person when a conversation needs one. An orchestration layer owns that handoff, and FlowRunner is built for it.
Without FlowRunner
With FlowRunner
Use Case Scenarios
Live Chat Triage
A customer sends a message in a live chat. The agent classifies intent and urgency with a fast Groq model and pulls the details the receiving team will need, all quickly enough that the customer notices no delay. Routine intents get a self-serve answer; everything else is routed to the right queue with context attached. Live conversations are triaged in the moment instead of sorted by whoever grabs them.
High-Volume Transcription
A steady stream of short audio clips needs transcription without a backlog. The agent transcribes each clip with Whisper on Groq and moves it into the flow fast enough to keep up in real time. Transcription stops being the bottleneck it is on slower services.
Inline Extraction
An interactive process needs structured fields pulled from user input as it happens. The agent extracts the fields with a fast Groq model and validates them inline. Clean input continues immediately; anything ambiguous is surfaced to a person without stalling the flow for everyone else.
Human-in-Loop Highlight
Speed makes it tempting to let a model just handle everything in real time, which is exactly when a human gate matters most. When a live conversation is complex, high-value, or something the model classifies with low confidence, FlowRunner routes it through a [human-in-the-loop](/concepts/human-in-the-loop/) step: the agent pauses the automated path, hands the conversation and its context to a live agent, and steps back. Fast inference triages the volume in the moment; a person takes the conversations that need one.
Agent Capabilities
17 actionsChat
2- Chat Completion Generates a text response for a single prompt using a Groq-hosted model (Llama, GPT-OSS, Qwen, Compound and more) with industry-leading inference speed.
- Chat Completion (Advanced) Sends a fully custom chat completion request to Groq with a complete messages array (multi-turn conversations, multimodal content parts), tool/function calling passthrough, structured outputs via a response format object (json_object or json_schema), reasoning controls, penalties and sampling parameters.
Vision
1- Analyze Image Analyzes up to 5 images with a Groq multimodal vision model (e.g. Llama 4 Scout) and answers a prompt about them, describe content, extract text, compare images or answer visual questions.
Audio
3- Transcribe Audio Transcribes an audio file into text in its original language using Groq's ultra-fast Whisper models.
- Translate Audio Translates speech from an audio file in any supported language into English text using Groq's Whisper models.
- Text to Speech Converts text into natural-sounding speech audio using Groq's Orpheus text-to-speech models (English and Arabic voices).
Files
5- Upload File Uploads a JSONL file to Groq for batch processing. Provide either the URL of an existing JSONL file (FlowRunner file URL or any public URL) or raw JSONL content as text.
- List Files Lists all files uploaded to Groq, including batch input files and generated batch output/error files, with their IDs, names, sizes and purposes.
- Get File Retrieves the metadata of a file stored in Groq by its ID, including filename, size, purpose and creation time.
- Delete File Permanently deletes a file stored in Groq by its ID. Returns a deletion confirmation object.
- Download File Content Downloads the content of a file stored in Groq (typically JSONL batch results from a batch's output_file_id or error_file_id), saves it to FlowRunner file storage and returns its URL.
Batches
4- Create Batch Creates an asynchronous batch job that processes a previously uploaded JSONL file of API requests at a 50% cost discount.
- Get Batch Retrieves the current state of a batch job by its ID, including status (validating, in_progress, finalizing, completed, failed, expired, cancelled), request counts, and the output_file_id / error_file_id for downloading results once finished.
- List Batches Lists batch jobs in the Groq account with their statuses, endpoints and request counts. Supports pagination via the limit and after parameters.
- Cancel Batch Cancels an in-progress batch job by its ID. The batch moves to 'cancelling' and then 'cancelled'; any completed requests are still available in the output file.
Models
2- List Models Lists all currently active models available through the Groq API, including chat, vision, reasoning, speech-to-text, text-to-speech, guard and agentic models, with their owners and context window sizes.
- Get Model Retrieves the details of a specific Groq model by its ID, including owner, active status and context window size.
Start building with Groq
$100 in credits. No card required. Connect in minutes.