Skip to content
Podcast

53. Why Your Vibe Code Needs Tests, Ditching SaaS & AI Agents That Work While You Sleep

Create With0

Summary

James and Kieran discuss the practical realities of building AI-powered software, emphasizing the importance of testing in vibe-coded applications, exploring alternatives to traditional SaaS tools, and implementing autonomous AI agents for business operations. The episode covers technical topics like Convex databases, app store bottlenecks from AI-generated submissions, and using OpenClaw as a marketing automation agent.

What You'll Learn

The Evolution from Vibe Coder to AI Engineer

The landscape of AI-powered software development is shifting rapidly, and with it comes a crucial realization: vibe coding without testing is no longer viable for production applications. As more solo founders build sophisticated software using AI tools, the gap between quick prototypes and reliable products becomes increasingly apparent.

Why Testing Matters for Vibe-Coded Applications

Traditional developers have long understood the importance of automated testing, but it's been largely absent from the vibe coding conversation. This gap became painfully clear when a HubSpot integration broke core functionality in VidPop, a video testimonial SaaS built entirely with Claude Code.

The incident revealed a fundamental truth: as vibe-coded applications grow in complexity—handling video recording, uploads, storage, and processing—the surface area for bugs expands exponentially. A single feature addition can cascade through the codebase, breaking functionality in unexpected ways.

The Traditional Testing Barrier

Historically, testing presented a significant barrier for non-traditional developers. Ruby on Rails tutorials would introduce test-driven development (TDD) midway through, requiring developers to write tests before features—a conceptually challenging shift that confused beginners and discouraged adoption.

The abstract, mathematical nature of writing tests for non-existent features felt disconnected from the tangible work of building. For companies building mission-critical software (like code running in Tesla vehicles), 100% test coverage is mandatory, but this standard seemed unattainable for vibe coders.

AI-Powered Testing: The Game Changer

Modern AI coding assistants have eliminated the testing barrier entirely. Claude can generate comprehensive test suites in minutes—tasks that would traditionally take developers a week. These tests run automatically before deployment, checking hundreds of scenarios to ensure code changes haven't introduced regressions.

The process is straightforward: instruct your AI assistant to write tests for new features in your project configuration file (like claude.md), and it handles the implementation. Before each deployment, the test suite runs, providing confidence that existing functionality remains intact.

Beyond Traditional Testing

AI tools now support multiple testing approaches:

Programmatic Testing: Traditional unit and integration tests that verify code behavior at the function and module level.

Browser Automation Testing: Tools like Playwright enable AI to interact with applications through actual browser instances, testing user workflows end-to-end. Chrome's new built-in automation features promise to simplify this further, potentially eliminating the need for third-party automation tools entirely.

Documentation Synchronization: Automated tests serve double duty—they verify functionality while keeping documentation current. When tests pass, the documented behavior matches reality.

The Database Paradigm Shift: Convex vs. Supabase

Another significant development in the AI development ecosystem is the rise of code-defined databases like Convex. Unlike traditional database platforms that provide graphical interfaces for schema management, Convex defines everything in code.

This approach offers several advantages for AI-assisted development:

  • Schema changes live alongside application code, making them easier to version control and review
  • AI assistants can reason about database structure without navigating external interfaces
  • The entire application stack—frontend, backend, and database—exists in a unified codebase

While tools like Supabase offer powerful database UIs, the code-first approach of Convex aligns more naturally with AI-driven development workflows.

The App Store Bottleneck

As AI democratizes software creation, traditional distribution channels are struggling. Both Apple's App Store and Shopify's app marketplace are experiencing significant submission backlogs, overwhelmed by the surge in AI-generated applications.

This bottleneck raises important questions about the future of software distribution. When individuals can build custom applications in minutes, do centralized app stores remain relevant? The shift toward personal software—tools built for specific workflows rather than mass markets—challenges the entire app store model.

Replacing SaaS with Custom-Built Alternatives

One practical application of accessible AI development is replacing expensive SaaS subscriptions with custom-built alternatives. The barriers that once made building your own CMS, email platform, or project management tool impractical have largely disappeared.

Examples of successful replacements include:

Website Platforms: Moving from WordPress or paid website builders to custom Vercel deployments that take 10 minutes to build

Blogging Platforms: Ditching Ghost and Framer for custom solutions tailored to specific content needs

Project Management Tools: Building lightweight alternatives to complex platforms when your workflow doesn't match their assumptions

The key consideration is whether the SaaS tool you're replacing needs to be "AI-proof"—capable of integrating with AI assistants through APIs or Model Context Protocol (MCP) servers. If not, building your own might provide better long-term value.

Autonomous AI Agents: OpenClaw as Marketing Manager

The frontier of AI-assisted business operations is autonomous agents that handle entire business functions. OpenClaw, an open-source marketing agent, demonstrates this potential by operating independently within Slack.

Unlike traditional automation that follows rigid scripts, OpenClaw makes contextual decisions about marketing activities:

  • Monitoring community discussions and identifying engagement opportunities
  • Creating and scheduling social media content
  • Analyzing campaign performance and adjusting strategies
  • Responding to customer inquiries with appropriate context

The agent runs continuously, handling tasks that would otherwise consume significant founder attention. Setup and management happen through Claude Code, which can configure OpenClaw's behavior, adjust its decision-making parameters, and troubleshoot issues.

The Rise of Scheduled AI Tasks

Anthropic's introduction of scheduled tasks for Claude represents another step toward autonomous AI operations. Rather than triggering agents manually or through webhook events, developers can schedule regular agent activities—daily reports, weekly analyses, or continuous monitoring loops.

This capability, combined with tools like OpenClaw, enables truly autonomous business functions that operate without constant human oversight.

The Solo Founder Renaissance

These technological shifts are enabling a dramatic rise in solo founders. A16Z data shows solo entrepreneurship accelerating as AI tools multiply individual capability. What once required a team of 20 can now be accomplished by a single person with the right AI toolkit:

  • AI assistants handling coding and technical implementation
  • Autonomous agents managing marketing and customer engagement
  • Automated testing ensuring product reliability
  • Code-defined infrastructure simplifying deployment and scaling

This isn't just about building faster—it's about maintaining the operational complexity of a full business without the overhead of a large team.

Best Practices for Modern Vibe Coding

To build reliable, production-ready applications using AI tools:

1. Implement automated testing from the start: Add testing requirements to your AI assistant's project configuration

2. Choose tools designed for AI workflows: Code-first platforms like Convex integrate more naturally with AI development patterns

3. Deploy autonomous agents strategically: Start with well-defined business functions like marketing or customer support

4. Document as you build: Let your tests inform your documentation, keeping them synchronized automatically

5. Plan for the AI-proof future: Build or choose tools that expose APIs and support AI integration

The divide between "vibe coders" and "serious AI engineers" is closing rapidly. The tools and practices that separate hobbyist projects from production software are now accessible to anyone willing to learn them. Testing, infrastructure as code, and autonomous agents are no longer advanced topics—they're becoming fundamental requirements for anyone building software with AI.

Key Learnings

1AI-generated test suites eliminate the testing barrier for vibe coders

Claude and similar AI assistants can write comprehensive test suites in minutes that would traditionally take developers a week, including unit tests, integration tests, and end-to-end browser automation tests. These tests run automatically before deployment to catch breaking changes.

Learn more about Claude

2Code-defined databases like Convex align better with AI development workflows

Unlike UI-based database platforms, code-first databases keep schema definitions alongside application code, making it easier for AI assistants to reason about and modify database structure without navigating external interfaces or losing context.

3Chrome's built-in automation may replace third-party testing tools

New features in Chrome enable AI agents to control the browser directly without requiring tools like Playwright, simplifying the setup and maintenance of automated browser testing for vibe-coded applications.

4App stores are experiencing bottlenecks from AI-generated submissions

Both Apple's App Store and Shopify's app marketplace are overwhelmed by the surge in AI-built applications, creating significant review delays and raising questions about the future viability of centralized app distribution as software creation becomes democratized.

5OpenClaw enables autonomous marketing operations within Slack

OpenClaw functions as a marketing agent that makes contextual decisions about content creation, community engagement, and campaign management, operating continuously within Slack and managed through Claude Code for configuration and troubleshooting.

Learn more about OpenClaw

6Automated testing enables automatic documentation synchronization

When tests verify application behavior, they can simultaneously update documentation to reflect actual functionality, ensuring docs remain accurate as the application evolves without manual maintenance overhead.

Learn more about Claude

7Replacing expensive SaaS tools with custom builds is increasingly viable

The time investment required to build custom alternatives to platforms like Ghost, Framer, or WordPress has dropped dramatically—a fully functional website can be built and deployed to Vercel in approximately 10 minutes using AI coding assistants.

Learn more about Vercel

Resources Mentioned

Claude AI Assistant

Anthropic's AI assistant capable of generating comprehensive test suites, managing autonomous agents, and building full-stack applications through conversation.

Visit

Convex Database Platform

Code-first database platform that defines schemas in code rather than through UI, designed for seamless integration with AI development workflows.

Visit

OpenClaw GitHub Repository

Open-source autonomous marketing agent that operates within Slack, handling content creation, community engagement, and campaign management independently.

Visit

Create With Community

UK community for builders working with AI agents, vibe coding, and no-code tools. Hosts the annual Create With Conference featuring talks and workshops on AI development.

Visit

Supabase Documentation

Open-source Firebase alternative with PostgreSQL database, authentication, and real-time subscriptions—a popular choice for vibe-coded applications before code-first alternatives.

Visit

Playwright Browser Automation

Microsoft's browser automation framework for end-to-end testing, though potentially being superseded by Chrome's native AI automation capabilities.

Visit

Frequently Asked Questions

Why do vibe-coded applications need automated testing?

As vibe-coded applications grow in complexity, manual testing becomes impractical and unreliable. Automated tests catch breaking changes when new features are added, ensure existing functionality continues working after updates, and provide confidence for deployment. AI tools like Claude can generate comprehensive test suites in minutes, eliminating the traditional barrier that made testing inaccessible to non-traditional developers.

How is Convex different from Supabase for AI development?

Convex is a code-defined database where schemas live in your codebase alongside application logic, while Supabase provides a graphical UI for database management. For AI-assisted development, Convex's approach lets AI assistants reason about database structure without navigating external interfaces, keeping everything in a unified codebase. This makes schema changes easier to version control, review, and manage through AI conversation.

What is OpenClaw and how does it work as a marketing agent?

OpenClaw is an open-source autonomous agent that operates within Slack to handle marketing tasks like content creation, community engagement, and campaign management. Unlike traditional automation that follows rigid scripts, OpenClaw makes contextual decisions based on ongoing conversations and business goals. It's configured and managed through Claude Code, enabling founders to delegate entire marketing functions to AI while focusing on product development.

Why are app stores experiencing bottlenecks with AI-generated apps?

Apple's App Store and Shopify's app marketplace are overwhelmed by a surge in submissions as AI tools democratize software creation. More individuals can now build complete applications quickly, but the centralized review processes haven't scaled accordingly. This raises questions about whether traditional app stores remain relevant as software shifts toward personal, custom-built tools rather than mass-market applications.

Can Chrome really replace Playwright for automated testing?

Chrome's new built-in automation features allow AI agents to control the browser directly without third-party tools, potentially eliminating the need for Playwright in many testing scenarios. This simplifies setup and maintenance by removing an external dependency. However, Playwright still offers cross-browser testing across Chrome, Firefox, and Safari, which may remain valuable for applications requiring broad browser compatibility verification.

More content like this

Never miss a video

Get weekly updates on the best AI tutorials, conference recordings, and community content.

Keep Watching

Related Videos

View all videos
55. We're Back! Conference Recap, Running Events with AI & Self-Healing Apps
Podcast35:53

55. We're Back! Conference Recap, Running Events with AI & Self-Healing Apps

James and Kieran return to the Create With Podcast after their summer break to recap the 2026 Create With Conference in Brighton. They discuss how 550 attendees experienced workshops, a charity hackathon where functional apps were built in 30 minutes, and an innovative conference operating system built entirely with AI. The episode covers the latest developments in AI development tools including Anthropic's Fable model, scheduled tasks in Codex, and self-healing applications using PostHog.

The Biggest AI Opportunities for Non-Technical Builders in 2026
Conference38:09

The Biggest AI Opportunities for Non-Technical Builders in 2026

A closing panel discussion from Create With featuring leaders from Glide, Softr, Bubble, Momen, and TechFoundHer examining how AI and no-code platforms are democratizing software development. The panelists explore what non-technical builders can now create, how AI is transforming service businesses, and where expertise still matters when prototypes become production applications.

Everyone Can Build with AI. What Should Companies Do Next?
Conference29:07

Everyone Can Build with AI. What Should Companies Do Next?

A leadership panel from Create With conference exploring how companies should approach AI adoption when building tools becomes accessible to everyone. Four experts debate the gap between technical capability and strategic judgment, discussing governance frameworks, safe experimentation, product sense development, and organizational learning pathways for AI transformation.

How to Get Your Product Recommended by Google and ChatGPT
Conference30:12

How to Get Your Product Recommended by Google and ChatGPT

Tanya Van Gastel presents the APEX framework for making products discoverable in AI-powered search engines like ChatGPT and Google's AI Overviews. She explains how AI is fundamentally changing product discovery and buyer behavior, then breaks down five critical components: Authority (being cited as a solution), Presence (visibility across search platforms), Expertise (creating high-quality money pages), Endorsements (off-site validation), and Extractability (making content easy for AI to process). The talk emphasizes quality over quantity, with data showing 50% of SaaS SEO revenue comes from fewer than 10 high-performing pages.

Inside a 60-Skill Claude Code System Running AI Agents in Parallel
Conference35:21

Inside a 60-Skill Claude Code System Running AI Agents in Parallel

Stephany Oliveros, CEO of Lyra AI, presents a production-ready Claude Code architecture for running dozens of AI agents in parallel. She demonstrates a 60-skill system built on progressive disclosure, layered memory, and reusable components that enable coordinated multi-agent workflows without burning through tokens. The talk covers practical folder structures, guardrails, and token economics for building scalable agentic systems.

The AI-Native Solopreneur: Build Without a Technical Co-Founder
Conference34:21

The AI-Native Solopreneur: Build Without a Technical Co-Founder

Effie Bersoux presents a paradigm shift in entrepreneurship where AI enables solo founders to build, validate, and scale businesses without technical co-founders or significant capital. Drawing from Growth Girls' experience building 40+ companies in one year and going from a £4m exit (7 years) to a £40m acquisition target (1 year), she demonstrates how AI breaks the traditional capital-talent-traction loop that disadvantages undercapitalized founders. The talk provides concrete frameworks for AI-native operating models, automation decision-making, and rapid idea-to-market validation.