@silentsilas/vite-plugin-ai-robots

Vite plugin that automatically generates and updates a robots.txt file blocking AI agents using the Dark Visitors API.

Installation

Create a free account at Dark Visitors and obtain your access token.

npm install @silentsilas/vite-plugin-ai-robots --save-dev

Add the token to your environment, and pass it to the plugin's config. You should now see a robots.txt file in your output directory during builds.

Usage

Basic Configuration

// vite.config.ts
import { defineConfig } from "vite";
import { aiRobots } from "@silentsilas/vite-plugin-ai-robots";

export default defineConfig({
  plugins: [
    aiRobots({
      accessToken: process.env.DARK_VISITORS_TOKEN,
    }),
  ],
});

Configuration Options

Option	Type	Default	Description
accessToken	string	Required	Dark Visitors API token
agentTypes	string[]	["AI Data Scraper", "Undocumented AI Agent"]	Agent types to block
disallow	string	"/"	Paths to disallow
cacheHours	number	24	Cache duration in hours
outputDir	string	"static"	Output directory
debug	boolean	false	Enable debug logging

Troubleshooting

Common Issues:

• 401 Unauthorized: Check your access token
• Empty robots.txt: Enable debug: true
• Cache not updating: Delete .ai-robots-cache.json
  • You may want to add this file to your .gitignore