diff --git a/src/content/docs/en/guides/content-collections.mdx b/src/content/docs/en/guides/content-collections.mdx index 546df81b74d70..063c0d149b653 100644 --- a/src/content/docs/en/guides/content-collections.mdx +++ b/src/content/docs/en/guides/content-collections.mdx @@ -83,11 +83,11 @@ The Content Layer API allows you to fetch your content (whether stored locally i #### Built-in loaders -Astro provides two built-in loader functions (`glob()` and `file()`) for fetching your local content, as well as access to the API to construct your own loader and fetch remote data. +Astro provides [two built-in loader functions](/en/reference/content-loader-reference/#built-in-loaders) (`glob()` and `file()`) for fetching your local content, as well as access to the API to construct your own loader and fetch remote data. -The `glob()` loader creates entries from directories of Markdown, MDX, Markdoc, or JSON files from anywhere on the filesystem. It accepts a `pattern` of entry files to match using glob patterns supported by [micromatch](https://github.com/micromatch/micromatch#matching-features), and a base file path of where your files are located. Each entry's `id` will be automatically generated from its file name. Use this loader when you have one file per entry. +The [`glob()` loader](/en/reference/content-loader-reference/#glob-loader) creates entries from directories of Markdown, MDX, Markdoc, JSON, or YAML files from anywhere on the filesystem. It accepts a `pattern` of entry files to match using glob patterns supported by [micromatch](https://github.com/micromatch/micromatch#matching-features), and a base file path of where your files are located. Each entry's `id` will be automatically generated from its file name. Use this loader when you have one file per entry. -The `file()` loader creates multiple entries from a single local file. Each entry in the file must have a unique `id` key property. It accepts a `base` file path to your file and optionally a [`parser` function](#parser-function) for data files it cannot parse automatically. Use this loader when your data file can be parsed as an array of objects. +The [`file()` loader](/en/reference/content-loader-reference/#file-loader) creates multiple entries from a single local file. Each entry in the file must have a unique `id` key property. It accepts a `base` file path to your file and optionally a [`parser` function](#parser-function) for data files it cannot parse automatically. Use this loader when your data file can be parsed as an array of objects. ```ts title="src/content.config.ts" {5,9} import { defineCollection, z } from 'astro:content'; diff --git a/src/content/docs/en/reference/content-loader-reference.mdx b/src/content/docs/en/reference/content-loader-reference.mdx index e125ddd050495..59e6e75852e31 100644 --- a/src/content/docs/en/reference/content-loader-reference.mdx +++ b/src/content/docs/en/reference/content-loader-reference.mdx @@ -14,6 +14,140 @@ Astro loaders allow you to load data into [content collections](/en/guides/conte Each collection needs [a loader defined in its schema](/en/guides/content-collections/#defining-the-collection-loader). You can define a loader inline in your project's `src/content.config.ts` file, share one loader between multiple collections, or even [publish your loader to NPM as a package](/en/reference/publish-to-npm/) to share with others and be included in our integrations library. +## Built-in loaders + +Astro provides two built-in loaders to help you fetch your collections. Both offer options to suit a wide range of use cases. + +### `glob()` loader + +

+ +**Type:** (options: GlobOptions) => Loader
+ +

+ +The `glob()` loader creates entries from directories of files from anywhere on the filesystem. The supported file types are Markdown, MDX, Markdoc, JSON, and YAML files. + +This loader accepts an object with the following properties: `pattern`, `base` (optional), and `generateId` (optional). + +```ts title="src/content.config.ts" {2,6,11,17-21} +import { defineCollection } from 'astro:content'; +import { glob } from 'astro/loaders'; + +const pages = defineCollection({ + /* Retrieve all Markdown files in your pages directory. */ + loader: glob({ pattern: "**/*.md", base: "./src/data/pages" }), + schema: /* ... */ +}); +const blog = defineCollection({ + /* Retrieve all Markdown and MDX files in your blog directory. */ + loader: glob({ pattern: "**/*.(md|mdx)", base: "./src/data/blog" }), + schema: /* ... */ +}); +const authors = defineCollection({ + /* Retrieve all JSON files in your authors directory while retaining + * uppercase letters in the ID. */ + loader: glob({ + pattern: '**/*.json', + base: "./src/data/authors", + generateId: ({ entry }) => entry.replace(/\.json$/, ''), + }), + schema: /* ... */ +}); +``` + +#### `pattern` + +

+ +**Type:** `string | string[]` +

+ +The `pattern` property accepts a string or an array of strings using glob matching (e.g. wildcards, globstars). The patterns must be relative to the base directory of entry files to match. + +You can learn more about the syntax to use in the [micromatch documentation](https://github.com/micromatch/micromatch#matching-features). You can also verify the validity of your pattern using an online tool like the [DigitalOcean Glob Tool](https://www.digitalocean.com/community/tools/glob). + +#### `base` + +

+ +**Type:** `string | URL`
+**Default:** `"."` +

+ +A relative path or [URL](https://developer.mozilla.org/en-US/docs/Web/API/URL) to the directory from which to resolve the `pattern`. + +#### `generateId()` + +

+ +**Type:** `(options: GenerateIdOptions) => string` +

+ +A callback function that returns a unique string per entry in a collection. It accepts an object as parameter with the following properties: +* `entry` - the path to the entry file, relative to the base directory +* `base` - the base directory [URL](https://developer.mozilla.org/en-US/docs/Web/API/URL) +* `data` - the parsed, unvalidated data of the entry + +By default it uses [`github-slugger`](https://github.com/Flet/github-slugger) to generate a slug with [kebab-cased](https://developer.mozilla.org/en-US/docs/Glossary/Kebab_case) words. + +### `file()` loader + +

+ +**Type:** (fileName: string, options?: FileOptions) => Loader
+ +

+ +The `file()` loader creates entries from a single file that contains an array of objects with a unique `id` field, or an object with string keys. It supports JSON or YAML, and you can provide a custom `parser` for data files it cannot parse by default. + +This loader accepts a `fileName` property and an optional object as second argument: + +```ts title="src/content.config.ts" {2,6,11-13} +import { defineCollection } from 'astro:content'; +import { file } from 'astro/loaders'; + +const authors = defineCollection({ + /* Retrieve all entries from a JSON file. */ + loader: file("src/data/authors.json"), + schema: /* ... */ +}); +const products = defineCollection({ + /* Retrieve all entries from a CSV file using a custom parser. */ + loader: file("src/data/products.csv", { + parser: (fileContent) => { /* your parser logic */ }, + }), + schema: /* ... */ +}); +``` + +#### `fileName` + +

+ +**Type:** `string` +

+ +Sets the path to the file to load, relative to the root directory. + +#### Options + +

+ +**Type:** `FileOptions` +

+ +An optional object with the following properties: + +##### `parser()` + +

+ +**Type:** `(text: string) => Record> | Array>` +

+ +A callback function to create a collection from a file’s contents. Use it when you need to process file not supported by default (e.g. `.csv`) or when using [nested `.json` documents](/en/guides/content-collections/#nested-json-documents). + ## Loader types Loaders can be defined either as a simple function that returns an array of entries or with the more powerful object Content Loader API for more control over the loading process. @@ -88,7 +222,7 @@ const blog = defineCollection({ }); ``` -## Loader API +## Object loader API The API for [inline loaders](#inline-loaders) is very simple, and is shown above. This section shows the API for defining an object loader. diff --git a/src/content/docs/en/reference/modules/astro-content.mdx b/src/content/docs/en/reference/modules/astro-content.mdx index 8d5c55190355f..a77165d602984 100644 --- a/src/content/docs/en/reference/modules/astro-content.mdx +++ b/src/content/docs/en/reference/modules/astro-content.mdx @@ -59,7 +59,7 @@ This function accepts the following properties:

-**Type:** () => Promise<Array<\{ id: string, [key: string]: any }> | Record<string, Record<string, any>>> | Loader +**Type:** () => Promise<Array<\{ id: string, [key: string]: any }> | Record<string, Record<string, any>>> | Loader

diff --git a/src/content/docs/zh-cn/reference/modules/astro-content.mdx b/src/content/docs/zh-cn/reference/modules/astro-content.mdx index 18fb8982e246f..b9a53062b6e4e 100644 --- a/src/content/docs/zh-cn/reference/modules/astro-content.mdx +++ b/src/content/docs/zh-cn/reference/modules/astro-content.mdx @@ -60,7 +60,7 @@ export const collections = { blog };

-**Type:** () => Promise<Array<\{ id: string, [key: string]: any }> | Record<string, Record<string, any>>> | Loader +**Type:** () => Promise<Array<\{ id: string, [key: string]: any }> | Record<string, Record<string, any>>> | Loader