diff --git a/ARCHITECTURE_OVERVIEW.md b/ARCHITECTURE_OVERVIEW.md deleted file mode 100644 index 5d7df2405..000000000 --- a/ARCHITECTURE_OVERVIEW.md +++ /dev/null @@ -1,1098 +0,0 @@ -# Quartz v5 Architecture Overview - -This document provides a bird's-eye view of the Quartz v5 rework for maintainer review. It covers the plugin-based architecture, build pipeline, page type system, layout engine, and the full catalog of community plugins. - -## Table of Contents - -- [High-Level Architecture](#high-level-architecture) -- [Build Pipeline](#build-pipeline) -- [Plugin System](#plugin-system) - - [Plugin Categories](#plugin-categories) - - [Plugin Lifecycle](#plugin-lifecycle) - - [Plugin Manifest](#plugin-manifest) - - [Plugin Template Structure](#plugin-template-structure) -- [PageType System](#pagetype-system) - - [Matchers](#matchers) - - [Virtual Pages](#virtual-pages) - - [PageTypeDispatcher](#pagetypedispatcher) -- [Layout System](#layout-system) - - [Layout Positions](#layout-positions) - - [Priority and Ordering](#priority-and-ordering) - - [Groups (Flex Containers)](#groups-flex-containers) - - [Display Modifiers](#display-modifiers) - - [Conditions](#conditions) - - [Per-Page-Type Overrides](#per-page-type-overrides) -- [Component Architecture](#component-architecture) - - [QuartzComponent](#quartzcomponent) - - [Component Registry](#component-registry) - - [Page Rendering](#page-rendering) - - [Page Frames](#page-frames) - - [Transclusion](#transclusion) -- [Configuration](#configuration) - - [quartz.config.yaml](#quartzconfigyaml) - - [TypeScript Override](#typescript-override) -- [Plugin Catalog](#plugin-catalog) - - [Transformers](#transformers) - - [Filters](#filters) - - [Emitters](#emitters) - - [PageType Plugins](#pagetype-plugins) - - [Component Plugins](#component-plugins) - ---- - -## High-Level Architecture - -Quartz v5 is a fully plugin-based static site generator for digital gardens. Every functional unit — content transformation, filtering, page generation, and UI rendering — is implemented as an external community plugin loaded from Git repositories. - -``` -quartz.config.yaml ← Single source of truth for all configuration - │ - ▼ - ┌─────────────┐ - │ Config Loader│ Reads YAML, installs plugins from Git, validates - │ (loader/) │ dependencies, categorizes, sorts, instantiates - └──────┬──────┘ - │ - ▼ - ┌─────────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────┐ - │ Transformers│────▶│ Filters │────▶│ Emitters │────▶│ Output (HTML)│ - │ (parse) │ │ (filter) │ │ (emit) │ │ CSS, JS, etc│ - └─────────────┘ └──────────┘ └──────────┘ └──────────────┘ - │ - ┌──────┴──────┐ - │ PageType │ - │ Dispatcher │ - │ (phase 1) │ - └─────────────┘ -``` - -**Key architectural decisions:** - -1. **Everything is a plugin.** There is no monolithic core. The framework provides the pipeline, loader, and component infrastructure; all domain logic lives in plugins. -2. **Git-native plugin distribution.** Plugins are cloned from GitHub repositories into `.quartz/plugins/` via `isomorphic-git`. No npm registry required. -3. **Declarative configuration.** A single `quartz.config.yaml` declares all plugins, their options, ordering, and layout positions. TypeScript overrides are optional. -4. **PageType system.** A new abstraction that unifies content rendering, virtual page generation, and layout binding into a single plugin type. -5. **Two-phase emit.** The PageTypeDispatcher runs first (generating virtual pages), then all other emitters receive content + virtual pages together. - ---- - -## Build Pipeline - -The build pipeline lives in `quartz/build.ts` and the `quartz/processors/` directory. It follows a linear flow: - -### 1. File Discovery - -``` -glob("**/*.*", directory, ignorePatterns) - → allFiles (all file paths) - → allSlugs (slugified paths) - → Extension-stripped slug aliases for PageType extensions (.canvas, .base) -``` - -PageType plugins register file extensions (e.g., `.canvas`, `.base`). The build adds extension-stripped slug aliases so wikilinks like `![[file.canvas]]` resolve to the virtual page slug `file`. - -### 2. Parse (`quartz/processors/parse.ts`) - -Only `.md` files are parsed. The parsing pipeline: - -``` -Markdown source - → textTransform() Transformer plugins modify raw text - → remark-parse Markdown → mdast (Markdown AST) - → markdownPlugins() Transformer plugins add remark plugins - → remark-rehype mdast → hast (HTML AST) - → htmlPlugins() Transformer plugins add rehype plugins - → ProcessedContent [hast tree, VFile with data] -``` - -Each step runs every enabled transformer plugin's corresponding hook in order. - -### 3. Filter (`quartz/processors/filter.ts`) - -``` -ProcessedContent[] - → filter.shouldPublish(ctx, content) For each filter plugin - → FilteredContent[] -``` - -Filter plugins decide which files to publish. Examples: `remove-draft` (excludes `draft: true` frontmatter), `explicit-publish` (requires `publish: true`). - -### 4. Emit (`quartz/processors/emit.ts`) - -Emission is split into two phases to support virtual pages: - -``` -Phase 1: PageTypeDispatcher - ├── Generate virtual pages (tag pages, folder pages, bases pages, etc.) - ├── Populate ctx.virtualPages - ├── Render Body components → populate htmlAst (for transclusion) - └── Emit all pages as HTML - -Phase 2: All other emitters - ├── Receive content + virtualPages merged - ├── ContentIndex (sitemap, RSS, search index) - ├── AliasRedirects, CNAME, Favicon, OGImage - └── ComponentResources (CSS, JS bundles), Assets, Static files -``` - -This two-phase approach ensures that emitters like ContentIndex include virtual pages in their output (sitemap, RSS, explorer data). - -### Incremental Rebuild - -During `--watch` mode, the same two-phase approach applies. `partialEmit` is used when available to only re-emit changed files. The content map tracks add/change/delete events, and the trie is rebuilt on each partial emit. - ---- - -## Plugin System - -### Plugin Categories - -There are **4 processing categories** and **1 UI category**: - -| Category | Interface | Key Methods | Purpose | -| --------------- | ----------------------------------- | ---------------------------------------------------------------------- | --------------------------------------- | -| **Transformer** | `QuartzTransformerPluginInstance` | `textTransform`, `markdownPlugins`, `htmlPlugins`, `externalResources` | Transform content during parsing | -| **Filter** | `QuartzFilterPluginInstance` | `shouldPublish` | Decide which files to include | -| **Emitter** | `QuartzEmitterPluginInstance` | `emit`, `partialEmit`, `getQuartzComponents`, `externalResources` | Produce output files | -| **PageType** | `QuartzPageTypePluginInstance` | `match`, `generate`, `body`, `layout`, `fileExtensions`, `priority` | Define page rendering and virtual pages | -| **Component** | _(registered in ComponentRegistry)_ | QuartzComponent function + static CSS/JS | UI components placed via layout config | - -A plugin can belong to multiple categories. For example, `content-page` is both `pageType` and `component` — it defines how content pages are matched/rendered AND provides the Body component. - -### Plugin Lifecycle - -``` -quartz.config.yaml - │ - ▼ -1. Read YAML config (config-loader.ts) - │ - ▼ -2. For each enabled plugin entry: - │ a. Parse source string (e.g., "github:quartz-community/explorer") - │ b. Clone/update from GitHub via isomorphic-git (gitLoader.ts) - │ c. Read manifest from package.json "quartz" field - │ d. Load components into ComponentRegistry (componentLoader.ts) - │ e. Load frames into FrameRegistry (frameLoader.ts) - │ - ▼ -3. Validate dependencies - │ a. Check all dependencies are present and enabled - │ b. Verify execution order (dependency must run before dependent) - │ c. Detect circular dependencies - │ - ▼ -4. Categorize plugins - │ a. Read category from manifest - │ b. For dual-category (e.g., ["pageType", "component"]), resolve processing category - │ c. Component-only plugins are loaded into registry but not into processing pipeline - │ d. If category unknown, detect by instantiating and inspecting exported methods - │ - ▼ -5. Sort by order within each category (entry.order ?? manifest.defaultOrder ?? 50) - │ - ▼ -6. Instantiate plugins - │ a. Import module from dist/index.js - │ b. Find factory function (default export, or "plugin" export, or first matching export) - │ c. Merge options: { ...manifest.defaultOptions, ...entry.options } - │ d. Call factory(mergedOptions) → plugin instance - │ - ▼ -7. Build layout (loadQuartzLayout) - │ a. Resolve component references from ComponentRegistry - │ b. Apply display wrappers (MobileOnly/DesktopOnly) - │ c. Apply condition wrappers (ConditionalRender) - │ d. Resolve flex groups - │ e. Build per-page-type layout overrides - │ - ▼ -8. Add built-in plugins - │ a. ComponentResources, Assets, Static (emitters) - │ b. NotFoundPageType (built-in 404 page) - │ c. PageTypeDispatcher (wired with resolved layout) - │ - ▼ -9. Return QuartzConfig { configuration, plugins } -``` - -### Plugin Manifest - -Each plugin declares metadata in its `package.json` under the `"quartz"` field: - -```json -{ - "quartz": { - "name": "explorer", - "displayName": "Explorer", - "category": "component", - "version": "1.0.0", - "quartzVersion": ">=5.0.0", - "dependencies": [], - "defaultOrder": 50, - "defaultEnabled": true, - "defaultOptions": {}, - "components": { - "Explorer": { - "displayName": "Explorer", - "defaultPosition": "left", - "defaultPriority": 50 - } - } - } -} -``` - -| Field | Type | Description | -| ---------------- | ---------------------------------------- | --------------------------------------------------------------------------- | -| `name` | `string` | Plugin identifier (kebab-case) | -| `displayName` | `string` | Human-readable name | -| `category` | `string \| string[]` | One or more of: `transformer`, `filter`, `emitter`, `pageType`, `component` | -| `version` | `string` | Plugin version | -| `quartzVersion` | `string` | Compatible Quartz version range | -| `dependencies` | `string[]` | Required plugin sources (e.g., `"github:quartz-community/crawl-links"`) | -| `defaultOrder` | `number` | Default execution order (0-100, lower = runs first) | -| `defaultEnabled` | `boolean` | Whether enabled by default on install | -| `defaultOptions` | `object` | Default options merged with user config | -| `configSchema` | `object` | JSON Schema for options validation | -| `components` | `Record` | UI components provided by this plugin | -| `frames` | `Record` | Page frames provided by this plugin (see [Page Frames](#page-frames)) | - -### Plugin Template Structure - -Every community plugin follows this structure: - -``` -plugin-name/ -├── src/ -│ ├── index.ts # Main exports (plugin factory + manifest) -│ ├── [plugin-type].ts # Plugin implementation -│ ├── types.ts # Type definitions -│ └── components/ # UI components (if any) -│ ├── index.ts # Component exports -│ ├── Component.tsx # Preact component -│ └── Component.scss # Styles -├── dist/ # Compiled output (tsup) -│ ├── index.js / index.d.ts -│ └── components/ -├── package.json # With "quartz" metadata field -├── tsconfig.json # TypeScript config -├── tsup.config.ts # Build config (sass.compile for SCSS) -├── vitest.config.ts # Test config -├── .eslintrc.json -├── .prettierrc -├── README.md -├── LICENSE -├── CHANGELOG.md -└── .github/ # CI workflows -``` - -**Build configuration (`tsup.config.ts`):** - -All plugins use `tsup` for building. Plugins with SCSS styles use a custom `esbuildPlugins` entry that compiles SCSS via `sass.compile()` and injects the CSS as a string export. - ---- - -## PageType System - -The PageType system is the central innovation of v5. It replaces the previous monolithic page rendering approach with a pluggable, extensible system where each page "type" (content, folder, tag, canvas, bases) is a separate plugin. - -### Core Interface - -```typescript -interface QuartzPageTypePluginInstance { - name: string - priority?: number // Higher = matched first (descending sort) - fileExtensions?: string[] // e.g., [".canvas"] — registers for slug aliasing - match: PageMatcher // Determines if this page type handles a given page - generate?: PageGenerator // Produces virtual pages (tag listings, folder pages, etc.) - layout: string // Layout key (used for per-page-type overrides) - frame?: string // PageFrame name (e.g. "full-width", "minimal") — defaults to "default" - body: QuartzComponentConstructor // The Body component for rendering page content -} -``` - -### Matchers - -The `matchers.ts` module provides composable matcher factories: - -```typescript -import { match } from "quartz/plugins/pageTypes/matchers" - -match.ext(".canvas") // Matches files with .canvas extension -match.slugPrefix("tags/") // Matches slugs starting with "tags/" -match.frontmatter( - "layout", // Matches based on frontmatter values - (v) => v === "custom", -) -match.and(m1, m2) // Logical AND -match.or(m1, m2) // Logical OR -match.not(m1) // Logical NOT -match.all() // Always matches -match.none() // Never matches -``` - -### Virtual Pages - -PageType plugins with a `generate` method produce virtual pages — pages that don't exist as files on disk but are rendered as if they do. Examples: - -- **Tag pages**: One virtual page per tag (e.g., `/tags/javascript`) -- **Folder pages**: One virtual page per folder (e.g., `/posts/`) -- **Bases pages**: View-based aggregate pages from `.base` files - -Virtual pages are represented as: - -```typescript -interface VirtualPage { - slug: string - title: string - data: Partial & Record -} -``` - -They are converted to `ProcessedContent` (with a synthetic hast tree and VFile) by the PageTypeDispatcher. - -### PageTypeDispatcher - -The PageTypeDispatcher (`quartz/plugins/pageTypes/dispatcher.ts`) is a special emitter plugin that orchestrates all PageType plugins. It runs as Phase 1 of emission and has a 3-phase internal flow: - -``` -Phase 1: Generate Virtual Pages - For each PageType plugin with a generate() method: - → Call generate({ content, cfg, ctx }) - → Create ProcessedContent entries (defaultProcessedContent) - → Push to ctx.virtualPages (except 404) - → Collect in virtualEntries[] - -Phase 2: Populate htmlAst - For each virtual entry: - → Render BodyComponent via preact-render-to-string - → Parse HTML string to hast via hast-util-from-html - → Set tree.children and vfile.data.htmlAst - (This enables transclusion of virtual pages, e.g., ![[file.canvas]]) - -Phase 3: Emit Pages - a. Emit regular pages: - For each file in content: - → Find first matching PageType (sorted by priority, descending) - → Resolve layout (defaults + per-page-type overrides) - → renderPage() → write HTML - b. Emit virtual pages: - For each virtual entry: - → renderPage() → write HTML -``` - -**Priority:** PageType plugins are sorted by `priority` (descending, higher = matched first). The first match wins. This allows specialized types (canvas, bases) to take precedence over the generic content type. - ---- - -## Layout System - -The layout system determines which UI components appear on each page and where they are positioned. It is fully declarative via `quartz.config.yaml`. - -### Layout Positions - -Each component is placed in one of four positions: - -| Position | Location | Rendered As | -| ------------ | ------------------ | ----------------------------------- | -| `left` | Left sidebar | `