Content Storage

Last updated Jul 2, 2026

Relevant source files

The following files were used as context for generating this wiki page:

Content Storage provides the foundational abstraction layer that reconciles heterogeneous content sources—such as local Markdown/MDX files, Obsidian vaults, and OpenAPI or AsyncAPI specifications—into a normalized virtual file system. By decoupling the source of truth from the consumption layer (the documentation UI), it enables seamless ingestion of content regardless of whether it originates from the local disk, dynamic remote schemas, or content collection build pipelines.

The architecture centers on the FileSystem primitive, which serves as an in-memory repository for virtualized files. Content loaders (such as local-md, mdx, or protocol-specific plugins like openapi) map raw source files into uniform ContentStoragePageFile or ContentStorageMetaFile structures. This normalization is essential for the loader utility, which acts as the primary orchestrator, indexing these files and generating a queryable API (e.g., for page tree construction, routing, and metadata retrieval).

By standardizing access via a shared ContentStorage interface, the system supports sophisticated features like internationalization (i18n), workspace scoping, and dynamic revalidation. This unified model ensures that components relying on the content tree do not need to be aware of the underlying file origin, effectively treating all content sources as a singular, consistent, and indexable collection.

The Virtual File System Architecture

The FileSystem class defines the structural core of the content storage system. It maintains an in-memory representation of files and folders, enabling efficient lookups without constant disk I/O. The system uses a virtual file path system where nested folders are tracked, allowing the content loader to walk the tree recursively during page tree generation.

Sources: packages/core/src/source/storage/file-system.ts:6-88, packages/core/src/source/storage/content.ts:6-14

Content Storage Builders

The createContentStorageBuilder utility orchestrates the ingestion of raw StaticSource input into a FileSystem instance. It handles path normalization and locale-specific partitioning based on i18n configuration. This builder performs a crucial transformation: it strips locale prefixes from file paths and places them into partitioned virtual filesystems, ensuring that the remainder of the system treats multilingual content as unified sets.

The process of populating the storage involves iterating over source files, normalizing their paths, and applying ContentStorage transformations. The parser function inside the builder detects the locale—either via directory structure (dir) or file naming conventions—and groups files accordingly before instantiating the FileSystem.

Sources: packages/core/src/source/storage/content.ts:50-87

Indexing Mechanism

Once the FileSystem is populated, the createPageIndexer utility builds an index for performant content discovery. This indexer maintains multiple internal maps that cross-reference files by path and slug. It is the core service that enables path resolution, locale switching, and page tree node lookups (e.g., retrieving a page's metadata from a node tree).

Page Indexing: Uses a combined key of [lang].[slug] to provide O(1) lookups for specific documents.
Path Resolution: Maps [lang].[path] for both metadata (pathToMeta) and page contents (pathToPage), allowing the system to resolve relative paths during content rendering.

Sources: packages/core/src/source/loader.ts:178-246

Plugin Integration

Plugins extend the storage lifecycle at two distinct points. The transformStorage hook allows plugins to modify or filter the file set after it has been loaded into a FileSystem instance, while transformPageTree handles structural adjustments at the node generation phase.

Plugin Hook	Responsibility
`config`	Modifies loader settings before initialization
`transformStorage`	Manipulates the filesystem state (e.g., slug normalization)
`transformPageTree`	Modifies the final hierarchy node attributes

Sources: packages/core/src/source/loader.ts:501-525

Tip

The priorityMap used by buildPlugins ensures that pre plugins execute before standard ones, while post plugins execute last, enabling deterministic control over how storage is transformed. Specifically, priorityMap values (pre: 1, default: 0, post: -1) determine the sort order.

Sources: packages/core/src/source/loader.ts:532-551

Dynamic Source Invalidation

For non-static sources (like remote OpenAPI schemas or local dev servers), the system employs a cache-invalidation mechanism. The DynamicLoader keeps a sourceCache to track remote inputs. When a change event triggers invalidate or revalidate, the system clears the cache, forcing the loader to re-fetch and re-index the content.

Warning

In local-md integrations, devServer connections use an event subscription model (conn.subscribe) to call invalidateFile. This ensures that cache consistency is maintained across HMR cycles, but it requires that files are resolved via path.resolve to avoid path mismatch errors.

Sources: packages/core/src/source/dynamic.ts:81-114, packages/local-md/src/index.ts:168-195

Lifecycle Walkthrough: Initializing the Loader

When loader() is called, it enters a multi-step sequence to construct the operational state:

Config Resolution: resolveConfig merges options and initializes plugin chains.
Storage Building: createContentStorageBuilder is invoked. If i18n is enabled, it returns a record of multiple ContentStorage instances (one per language); otherwise, it returns a single() instance.
Indexing: The loader iterates through the storage instances, passing them to indexer.scan.
Tree Building: The getPageTrees function is invoked on-demand (lazy) to compute the hierarchical tree using the loaded FileSystem and configured PageTreeTransformers.