load.py
The markata.plugins.load
plugin is responsible for loading and parsing markdown files
with frontmatter into Post objects. It provides parallel loading capabilities and
handles both modern Pydantic-based and legacy frontmatter validation.
Installation
This plugin is built-in and enabled by default through the 'default' plugin. If you want to be explicit, you can add it to your list of plugins:
hooks = [ "markata.plugins.load", ]
Uninstallation
Since this plugin is included in the default plugin set, to disable it you must explicitly add it to the disabled_hooks list if you are using the 'default' plugin:
disabled_hooks = [ "markata.plugins.load", ]
Note: Disabling this plugin will prevent Markata from loading any markdown files. This will effectively disable most of Markata's functionality.
Configuration
Configure loading behavior in your markata.toml
:
[markata] # Directories containing markdown content content_directories = [ "content", "posts" ] # Optional: Set to true to use legacy frontmatter validation legacy_frontmatter = false # Optional: Number of worker processes for parallel loading load_workers = 4
Functionality
File Loading
The plugin:
- Discovers markdown files in content directories
- Loads file content and frontmatter
- Validates frontmatter against Post model
- Creates Post objects for further processing
Parallel Processing
Loading is parallelized using:
- Process pool for file reading
- Configurable number of workers
- Chunked file processing
Validation Modes
Supports two validation approaches:
-
Modern Pydantic-based validation (default)
- Strict type checking
- Automatic type coercion
- Detailed validation errors
-
Legacy frontmatter validation
- Looser type checking
- Compatible with older content
- Less strict validation
Error Handling
The plugin provides:
- Detailed validation error messages
- Per-file error reporting
- Graceful fallback to legacy mode
- Optional strict validation
Registered Attributes
The plugin adds:
articles
: List of loaded Post objectscontent_directories
: List of content source directories
Dependencies
This plugin depends on:
- python-frontmatter for YAML parsing
- pydantic for validation
- multiprocessing for parallel loading
Function
load_file_content function
Load file content without validation.
load_file_content source
def load_file_content(path: Path) -> tuple[Path, dict]: """Load file content without validation.""" try: with open(path, "r") as f: raw_content = f.read() try: content = frontmatter.loads(raw_content) except Exception: content = None content["raw"] = raw_content return path, content except Exception: return path, None