Skip to content

Commit

Permalink
chore: release [email protected]
Browse files Browse the repository at this point in the history
  • Loading branch information
lin-stephanie committed Dec 13, 2024
1 parent cc0eff4 commit f3c237d
Show file tree
Hide file tree
Showing 8 changed files with 39 additions and 37 deletions.
5 changes: 5 additions & 0 deletions .changeset/two-snails-drum.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"astro-loader-tweets": patch
---

Refine logger output and update docs
12 changes: 3 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,12 @@ This monorepo hosts [Astro loaders](https://docs.astro.build/en/reference/conten

Currently available loaders (see individual READMEs for details):

- [astro-loader-github-releases](https://github.com/lin-stephanie/astro-loaders/tree/main/packages/astro-loader-github-releases): Loads GitHub releases for user-related or specific repositories.
- [astro-loader-github-prs](https://github.com/lin-stephanie/astro-loaders/tree/main/packages/astro-loader-github-prs): Loads GitHub pull reuqests from a given search string.
- [astro-loader-tweets](https://github.com/lin-stephanie/astro-loaders/tree/main/packages/astro-loader-tweets): Loads tweets from multiple tweet IDs.
- [astro-loader-github-releases](https://github.com/lin-stephanie/astro-loaders/tree/main/packages/astro-loader-github-releases): Loads GitHub releases from commits or repositories.
- [astro-loader-github-prs](https://github.com/lin-stephanie/astro-loaders/tree/main/packages/astro-loader-github-prs): Loads GitHub pull requests with a search query.
- [astro-loader-tweets](https://github.com/lin-stephanie/astro-loaders/tree/main/packages/astro-loader-tweets): Loads tweets by ID.

## Resources

Learn more about Astro loaders:

- [Blog - Experimental: Content Layer API](https://astro.build/blog/astro-4140/#experimental-content-layer-api)
- [Blog - Content Layer: A Deep Dive](https://astro.build/blog/content-layer-deep-dive/)
- [Blog - Community Loaders for Astro Content Layer](https://astro.build/blog/community-loaders/)
Expand All @@ -24,10 +22,6 @@ Learn more about Astro loaders:
- [ascorbic/astro-loaders](https://github.com/ascorbic/astro-loaders)
- [HiDeoo/astro-loader-npm-packages](https://github.com/HiDeoo/astro-loader-npm-packages)

## Contribution

If you see any errors or room for improvement on these Astro loaders, feel free to open an [issues](https://github.com/lin-stephanie/astro-loaders/issues) or [pull request](https://github.com/lin-stephanie/astro-loaders/pulls) . Thank you in advance for contributing!

## License

[MIT](https://github.com/lin-stephanie/astro-loaders/blob/main/LICENSE) © 2024-PRESENT [Stephanie Lin](https://github.com/lin-stephanie)
28 changes: 18 additions & 10 deletions packages/astro-loader-tweets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[![jsDocs.io][jsdocs-src]][jsdocs-href]
[![npm downloads][npm-downloads-src]][npm-downloads-href]

This package provides a tweets loader for Astro, fetching tweets from X (formerly Twitter) by their IDs for use in Astro projects. It supports customizable HTML/Markdown generation and saving tweets to a custom JSON file.
This package provides a tweets loader for Astro, fetching tweets from X (formerly Twitter) by ID for use in Astro projects. It supports customizable HTML/Markdown generation and saving tweets to a custom JSON file.

## Installation

Expand Down Expand Up @@ -32,7 +32,7 @@ import { tweetsLoader } from "astro-loader-tweets"

const tweets = defineCollection({
loader: tweetsLoader({
tweetIds: ['1865968097976586582'],
ids: ['1865968097976586582'],
// Check the configuration below
}),
})
Expand Down Expand Up @@ -79,11 +79,11 @@ This loader retrieves tweets via the X API V2 [`GET /2/tweets`](https://develope
| `newlineHandling` | `'none' \| 'break' \| 'paragraph'` (default: `'none'`) | The way for processing `\n` in `text_html` generation:<br>`'none'`: Keep as is.<br>`'break'`: Replace `\n` with `<br>`.<br>`'paragraph'`: Wrap paragraphs with `<p>` while removing standalone `\n`. |
| `authToken` | `string` (Defaults to the `X_TOKEN` environment variable) | The X app-only Bearer Token for authentication. **If configured here, keep confidential and avoid public exposure.** See [how to create one](https://developer.x.com/en/docs/authentication/oauth-2-0/bearer-tokens) and [configure env vars in an Astro project](https://docs.astro.build/en/guides/environment-variables/#setting-environment-variables). |

### About the `storage` Configuration
### About `storage` Configuration

**Why not use the Astro default store?**

- Each time the `content.config.ts` file is modified, Astro clears the [store](https://docs.astro.build/en/reference/content-loader-reference/#datastore) (i.e., `.astro/data-store.json` file).
- Each time the `content.config.ts` is modified, Astro clears the [store](https://docs.astro.build/en/reference/content-loader-reference/#datastore) (i.e., `.astro/data-store.json`).
- Under the X API V2 free plan, the endpoint requested are limited: only **1 request per 15 minutes** is allowed, with a maximum of **100 tweets retrievable per month**.

**Benefits of custom JSON file storage**
Expand Down Expand Up @@ -132,15 +132,23 @@ This is my current solution. If you have a more scalable approach, please share

## Schema

Refer to the [source code](https://github.com/lin-stephanie/astro-loaders/blob/main/packages/astro-loader-tweets/src/schema.ts#L269) for the Zod schema used for loaded collection entries. Astro automatically applies this schema to generate TypeScript interfaces, enabling autocompletion and type-checking for collection queries.
See the [source code](https://github.com/lin-stephanie/astro-loaders/blob/main/packages/astro-loader-tweets/src/schema.ts#L265) for the Zod schema of loaded entries and refer to the [Twitter API v2 data dictionary](https://developer.x.com/en/docs/x-api/data-dictionary/introduction) for field details. Astro automatically applies this schema to generate TypeScript interfaces, enabling autocompletion and type-checking for collection queries.

In addition to API-fetched fields, the loader extends fields defined in the [`TweetV2ExtendedSchema`](https://github.com/lin-stephanie/astro-loaders/blob/main/packages/astro-loader-tweets/src/schema.ts#L124), simplifying the control over tweet content display.
In addition to API-fetched fields, the loader extends fields defined in the [`TweetV2ExtendedSchema`](https://github.com/lin-stephanie/astro-loaders/blob/main/packages/astro-loader-tweets/src/schema.ts#L123), simplifying the control over tweet content display.

To [customize the schema](https://docs.astro.build/en/guides/content-collections/#defining-the-collection-schema), ensure compatibility with the loader's built-in Zod schema to prevent errors. For additional fields, consider [opening an issue](https://github.com/lin-stephanie/astro-loaders/issues).
To [customize the schema](https://docs.astro.build/en/guides/content-collections/#defining-the-collection-schema), ensure compatibility with the loader's built-in Zod schema to prevent errors. For additional fields, consider opening an issue.

[version-badge]: https://img.shields.io/npm/v/astro-loader-tweets?label=release&style=flat&colorA=080f12&colorB=ef7575
## Changelog

See [CHANGELOG.md](CHANGELOG.md) for the change history of this loader.

## Contribution

If you see any errors or room for improvement, feel free to open an [issues](https://github.com/lin-stephanie/astro-loaders/issues) or [pull request](https://github.com/lin-stephanie/astro-loaders/pulls) . Thank you in advance for contributing! ❤️

[version-badge]: https://img.shields.io/npm/v/astro-loader-tweets?label=release&style=flat&colorA=080f12&colorB=f87171
[version-link]: https://www.npmjs.com/package/astro-loader-tweets
[jsdocs-src]: https://img.shields.io/badge/jsdocs-reference-080f12?style=flat&colorA=080f12&colorB=ef7575
[jsdocs-src]: https://img.shields.io/badge/jsdocs-reference-080f12?style=flat&colorA=080f12&colorB=f87171
[jsdocs-href]: https://www.jsdocs.io/package/astro-loader-tweets
[npm-downloads-src]: https://img.shields.io/npm/dm/astro-loader-tweets?style=flat&colorA=080f12&colorB=ef7575
[npm-downloads-src]: https://img.shields.io/npm/dm/astro-loader-tweets?style=flat&colorA=080f12&colorB=f87171
[npm-downloads-href]: https://npmjs.com/package/astro-loader-tweets
2 changes: 1 addition & 1 deletion packages/astro-loader-tweets/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "astro-loader-tweets",
"version": "1.2.1",
"description": "Aatro loader for loading tweets from multiple tweet IDs.",
"description": "Astro loader for loading tweets by ID.",
"author": "Stephanie Lin <[email protected]>",
"license": "MIT",
"keywords": [
Expand Down
2 changes: 1 addition & 1 deletion packages/astro-loader-tweets/src/config.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { z } from 'astro/zod'
import type { Tweetv2FieldsParams } from 'twitter-api-v2'

export const defaultConfig = {
const defaultConfig = {
storage: 'default' as const,
storePath: 'src/data/tweets.json',
removeTrailingUrls: true,
Expand Down
19 changes: 9 additions & 10 deletions packages/astro-loader-tweets/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { TwitterApi, ApiResponseError } from 'twitter-api-v2'
import { ApiResponseError, TwitterApi } from 'twitter-api-v2'

import pkg from '../package.json' with { type: 'json' }
import { TweetsLoaderConfigSchema, getTweetsApiOptions } from './config.js'
import { processTweets, processTweetText, saveOrUpdateTweets } from './utils.js'
import { TweetSchema } from './schema.js'
import { processTweetText, processTweets, saveOrUpdateTweets } from './utils.js'

import type { Loader } from 'astro/loaders'
import type { TweetsLoaderUserConfig } from './config.js'
Expand All @@ -12,7 +12,7 @@ import type { Tweet } from './schema.js'
const MAX_IDS_PER_REQUEST = 100

/**
* Aatro loader for loading tweets from multiple tweet IDs.
* Astro loader for loading tweets by ID.
*
* @see https://github.com/lin-stephanie/astro-loaders/tree/main/packages/astro-loader-tweets
*/
Expand All @@ -29,13 +29,13 @@ Check out the configuration: ${pkg.homepage}README.md#configuration.`
)
return
}
const parsedUserConfig = parsedConfig.data

const { ids, storage, storePath, authToken, ...processTweetTextConfig } =
parsedUserConfig
parsedConfig.data
const token = authToken || import.meta.env.X_TOKEN

if (ids.length === 0) {
logger.warn('No tweet IDs provided')
logger.warn('No tweet IDs provided and no tweets will be loaded')
return
}

Expand Down Expand Up @@ -91,9 +91,8 @@ Check out the configuration: ${pkg.homepage}README.md#configuration.`
)
} catch (error) {
logger.error(
`Failed to save tweets to '${storePath}': ${(error as Error).message}`
`Failed to save tweets to '${storePath}'. ${(error as Error).message}`
)
return
}
}
} catch (error) {
Expand All @@ -103,10 +102,10 @@ Check out the configuration: ${pkg.homepage}README.md#configuration.`
error.rateLimit
) {
logger.warn(
`Rate limit exceeded with ${error.rateLimit.limit} requests/15min, ${error.rateLimit.remaining} remaining`
`Please try again later as the rate limit of ${error.rateLimit.limit} per 15 minutes is exceeded with ${error.rateLimit.remaining} left`
)
} else {
logger.error(`Failed to load tweets: ${(error as Error).message}`)
logger.error(`Failed to load tweets. ${(error as Error).message}`)
}
}
},
Expand Down
4 changes: 0 additions & 4 deletions packages/astro-loader-tweets/src/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ export const TweetV2Schema = z.object({
title: z.string().optional(),
description: z.string().optional(),
status: z.union([z.string(), z.number()]).optional(),
// mismatch field
media_key: z.string().optional(),
images: z
.array(
Expand Down Expand Up @@ -141,13 +140,11 @@ const UrlEntitySchema = EntitySchema.extend({
})

const HashtagEntitySchema = EntitySchema.extend({
// mismatch field
tag: z.string().optional(),
hashtag: z.string().optional(),
})

const CashtagEntitySchema = EntitySchema.extend({
// mismatch field
tag: z.string().optional(),
cashtag: z.string().optional(),
})
Expand Down Expand Up @@ -186,7 +183,6 @@ const UserV2Schema = z.object({
tweet_count: z.number().optional(),
listed_count: z.number().optional(),
like_count: z.number().optional(),
// mismatch field
media_count: z.number().optional(),
})
.optional(),
Expand Down
4 changes: 2 additions & 2 deletions packages/astro-loader-tweets/src/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import { z } from 'astro/zod'
import type { TweetsLoaderConfigSchema } from './config.js'
import type {
ResIncludes,
TweetV2Schema,
TweetV2ExtendedSchema,
Tweet,
TweetV2ExtendedSchema,
TweetV2Schema,
} from './schema.js'

/**
Expand Down

0 comments on commit f3c237d

Please sign in to comment.