Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.supaproxy.cloud/llms.txt

Use this file to discover all available pages before exploring further.

Knowledge source plugins define how external content is ingested into workspace context. Each source type handles fetching, chunking, and indexing content so the AI can reference it during queries.

​
Installation

npm install @supaproxy/knowledge-sources

​
KnowledgeSourcePlugin interface

interface KnowledgeSourcePlugin<TConfig = unknown> {
  /** Unique type identifier, e.g. "url", "confluence" */
  type: string;

  /** Human-readable name */
  name: string;

  /** Short description */
  description: string;

  /** Zod schema for configuration fields */
  configSchema: ZodSchema<TConfig>;

  /** Fetch and return raw content from the source */
  fetch(config: TConfig): Promise<RawContent[]>;

  /** Optional: custom chunking logic. Falls back to shared chunker. */
  chunk?(content: RawContent): Promise<Chunk[]>;
}

interface RawContent {
  text: string;
  metadata: Record<string, string>;
  sourceUrl?: string;
}

interface Chunk {
  text: string;
  metadata: Record<string, string>;
  index: number;
}

​
Shared chunker

All knowledge source plugins have access to a shared chunker utility. If a plugin does not implement its own chunk() method, the shared chunker is used automatically. 
import { chunker } from "@supaproxy/knowledge-sources";

const chunks = chunker.chunk(rawContent, {
  maxChunkSize: 1000,
  overlap: 100,
});
The shared chunker splits text by paragraphs, respects sentence boundaries, and maintains configurable overlap between chunks for context continuity.

​
Available plugins

Fetches and indexes content from a web URL. Supports HTML pages (content is extracted and cleaned).
Config fieldTypeDescription
urlstringURL to fetch
selectorstringOptional CSS selector to extract specific content
refreshIntervalnumberOptional re-fetch interval in hours
import { urlSource } from "@supaproxy/knowledge-sources";

​
Adding a custom source

To add a new knowledge source (e.g. Google Drive):

​
1. Implement the interface

import { z } from "zod";
import type { KnowledgeSourcePlugin, RawContent } from "@supaproxy/knowledge-sources";

const configSchema = z.object({
  folderId: z.string().min(1, "Folder ID is required"),
  serviceAccountKey: z.string().min(1, "Service account key is required"),
  fileTypes: z.array(z.string()).default(["document", "spreadsheet"]),
});

type GoogleDriveConfig = z.infer<typeof configSchema>;

export const googleDriveSource: KnowledgeSourcePlugin<GoogleDriveConfig> = {
  type: "google-drive",
  name: "Google Drive",
  description: "Index documents from a Google Drive folder.",
  configSchema,

  async fetch(config): Promise<RawContent[]> {
    // 1. Authenticate with service account
    // 2. List files in folder matching fileTypes
    // 3. Export each file as plain text
    // 4. Return as RawContent[]
  },

  // Optional: use custom chunking for spreadsheets
  async chunk(content) {
    if (content.metadata.type === "spreadsheet") {
      // Custom chunking: one chunk per sheet
    }
    // Fall back to shared chunker for documents
    return undefined;
  },
};

​
2. Register the plugin

import { knowledgeSourceRegistry } from "@supaproxy/knowledge-sources";
import { googleDriveSource } from "./google-drive";

knowledgeSourceRegistry.register(googleDriveSource);
The dashboard will render the Google Drive config form automatically. No frontend changes needed.