Module 05 — Audio-Visual

⏱️ 5 minutes

The Reality Check

The big AI platforms promise everything in one place — images, video, audio, all inside the chat. Your Agent works differently. It is text-based at heart and orchestrates specialized tools instead of doing everything itself.

🎬 The Orchestration Model

Your Agent does NOT:

Generate images directly
Edit photos or videos
Create video content
Watch YouTube links or analyze video natively

Your Agent DOES:

Connect to specialized platforms via API
Analyze images you upload
Write detailed prompts based on your project knowledge
Direct the workflow — you approve, platforms execute

Why this matters: Your Agent knows your project's context, history, and goals. It crafts specific, informed prompts that generic AI cannot generate.

Images: What Actually Works

✅ Your Agent CAN Do This With Images

Analyze content: "What's in this product photo?" → detailed description
Extract text (OCR): Pull text from screenshots, documents, signs
Answer questions: "Does this image contain a price tag?"
Retrieve from URLs: Scan images hosted online
Write prompts: Create detailed descriptions for external image generators

❌ Your Agent CANNOT Do This

Create or modify images (no built-in DALL-E)
Create or edit animations or video

Image Generation Platforms

Your Agent writes prompts; these platforms generate the visuals:

DALL-E 3 (OpenAI) — Best for precise, literal interpretations
Midjourney — Best for artistic, stylized images
Leonardo — Good balance of control and artistry

Cost note: These typically use a credits system per generation. Prices vary. Ask your agent to research options and talk you through setup

Audio: Input, Output, and Production

Audio adds a powerful new layer to your agent. It can be used in two primary ways: real-time interaction (talking directly with your agent) and production output (creating polished audio for content like audiobooks, voiceovers, and training materials).

Two Core Audio Use Cases

Conversational Audio: Speak naturally with your agent using voice input and receive spoken responses. This creates a more intuitive, hands-free workflow.
Production Audio: Generate high-quality voice content for external use — including audiobooks, narration, marketing voiceovers, and social content.

Voice Input (Speaking to Your Agent):
The Nerve interface currently has a voice option for speaking to your agent. WhatsApp currently is not supported. It supports native voice messages that are automatically transcribed into text for your agent.

For Discord, native speech-to-text (microphone-to-text) support is in development. Until that is released, you could try running Heyron on your iphone and using the built in microphone if you want to speak directly to your agent. ElevenLabs is not as good for this use.

Audio Capabilities (via API Integrations)

Text-to-Speech (TTS): Convert written content into natural-sounding voice output using a variety of voices and styles
Speech-to-Text (STT): Transcribe spoken audio into text for processing, automation, or storage
Voice Customization: Adjust tone, pacing, and style — or use cloned/custom voices depending on provider
Audio Generation: Some platforms also support music or background audio creation for richer content

Multiple providers support these features (e.g., ElevenLabs, Play.ht, Murf, AWS Polly). Your agent can guide you through selecting and connecting the best option for your use case. We know the ElevenLabs integration with Heyron works.

Pricing: Most platforms offer free tiers for testing, with paid plans for higher volume or production use.

Getting Started: Ask your agent something like: "Help me set up ElevenLabs" or "Connect a text-to-speech provider," and it will walk you through the process step-by-step.

Video: The Studio Workflow Option

Your Agent cannot watch YouTube links or analyze video content natively. This is a common misconception. Video requires a different workflow. There are other outside apps for this.

ElevenLabs Studio 3.0

For video creation and editing, ElevenLabs offers Studio 3.0 — an AI-powered video editor that integrates with their audio models.

What Studio 3.0 does:

Upload MP4/MOV files for AI enhancement
Add AI voiceovers synchronized to video
Generate captions,edit, collaborate and more

Plus: Access to video generation models (Veo, Sora, Kling) for creating footage from prompts. It does not generate video natively.

Typical Video Workflow:

Your Agent writes script → ElevenLabs Studio generates voiceover + adds to timeline → You upload footage or generate video → Your Agent reviews and suggests edits → Platform renders final output

The Cost Reality

What Sometimes Costs Money

Image generators: Usually credits per generation (varies by platform, free generators exist, or are included in programs you already pay for, like Canva)
ElevenLabs: Free tier for testing; paid for scale (characters/month)
Video platforms: Generally most expensive — compute-intensive

Money-saving tip: Your Agent can optimize prompts to reduce waste. Better prompts = fewer attempts = lower cost. Ask: "Make this prompt more specific so we get it right the first time."

Bottom Line

Your Agent is the strategic partner who knows your project. External platforms are the production tools that execute. Your Agent bridges that gap — turning your goals into specific, actionable prompts.

Remember: Your Agent brings context and knowledge; specialized tools bring production capability. Together, they get you better results than either alone.