AI Operations Specialist – LLM & Conversation Systems

Role Overview

We are looking for a hands-on AI Operations Specialist to own and improve our production AI conversation systems. The role involves onboarding new clients, configuring AI conversation systems, monitoring live conversations, diagnosing issues, improving prompts and system behavior, and measuring whether changes improve conversion.

What You Will Do

You’ll onboard new clients, configure AI conversation systems, monitor live conversations, diagnose issues, improve prompts and system behavior, and measure whether your changes actually improve conversion.

Why It Might Be a Fit

You’ll work directly with the founder on daily AI operations and improvements, communicate issues and findings clearly, and take ownership of problems from identification through resolution.

Requirements

• Proven experience running LLM or AI systems in production
• Hands-on experience monitoring and improving production AI outputs
• Experience with prompt engineering and system prompt optimization
• Strong understanding of LLM behavior, failure modes, and iterative testing
• Experience analyzing AI-generated conversations or customer-facing outputs
• Ability to work with webhooks, APIs, knowledge bases, or similar integrations
• Strong analytical and problem-solving skills
• Excellent written English
• Strong attention to detail
• Ability to work independently and take ownership of production systems