# robots.txt for VidMind.ai (Marketing Website) # Domain: https://vidmind.ai # Generated: 2026-01-13 # Strategy: Allow all bots for maximum visibility (AI + Search) # ============================================================================ # CONTENT SIGNAL (Allow AI Training) # ============================================================================ # We allow AI bots to train on our content to increase visibility in # AI-powered search results (ChatGPT, Claude, Gemini, etc.) User-Agent: * Content-signal: search=yes,ai-train=yes Allow: / # ============================================================================ # AI TRAINING BOTS (ALLOWED for AI Search Traffic) # ============================================================================ # These bots will help our content appear in AI-generated answers # OpenAI (ChatGPT, GPT models) User-agent: GPTBot Allow: / # Anthropic (Claude) User-agent: ClaudeBot Allow: / # Google AI (Bard, Gemini) User-agent: Google-Extended Allow: / # Meta AI User-agent: meta-externalagent Allow: / # Common Crawl (used by many AI systems) User-agent: CCBot Allow: / # ByteDance (TikTok AI) User-agent: Bytespider Allow: / # Amazon (Alexa AI) User-agent: Amazonbot Allow: / # Apple (Siri, Apple Intelligence) User-agent: Applebot-Extended Allow: / # ============================================================================ # SEARCH ENGINE SPECIFIC RULES (Traditional SEO) # ============================================================================ # Google Search User-agent: Googlebot Allow: / # Google Image Search User-agent: Googlebot-Image Allow: / # Bing Search User-agent: Bingbot Allow: / # Baidu Search (Chinese market) User-agent: Baiduspider Allow: / # Yandex Search (Russian market) User-agent: Yandex Allow: / # ============================================================================ # SITEMAP # ============================================================================ Sitemap: https://vidmind.ai/sitemap.xml # ============================================================================ # NOTES # ============================================================================ # This robots.txt is for the MARKETING WEBSITE only (vidmind.ai) # # Product application (app.vidmind.ai) has its own robots.txt # Staging environment (staging.vidmind.ai) has its own robots.txt # # Strategy: Allow all bots for maximum visibility # - Marketing content should be indexed by search engines # - Marketing content should be available to AI training # - This helps with both traditional SEO and AI search visibility