Nextra -> Fumadocs migration

[berekuk]

This is an intermediate step for upcoming changes in how we generate AI prompts and show docs: unlike Nextra, Fumadocs is more flexible and supports source APIs. So, in the following PR, we'll be able to show some docs without generating MDX, and use a single source of truth for everything.

In this PR, only the migration from Nextra is implemented, and doc files are still stored as before, in MDX.

UI changes

There are many minor styling changes, so I'll show some screenshots here.

Top panel

Before:

After:

This is Fumadocs default. Can be replaced if necessary.

Search

Before:

After:

This one is a mixed bag, I like snippets in the old version, but the new version is not bad either, and it's possible to improve.

Footer and header

On /docs/... pages, the footer is hidden, and navigation is collapsed under the dropdown:

This might hurt Squiggle Hub discoverability a little, but probably doesn't matter too much, and dense layout is also nice.

Again, this is customizable if we don't like it.

Descriptions and titles

Unlike Nextra, Fumadocs shows description and title from frontmatter:

So I had to tune the existing MDX docs and generation for API MDX docs.

Other changes

MarkdownViewer

I've removed the support for <details> and <summary>, and for nested code blocks from <MarkdownViewer>. These were used in Squiggle AI logs only.

It's possible to bring them back, but that would require the dependency on rehype-raw, which I'd like to avoid. We can reimplement collapsible blocks in markdown later through some other route (either with collapsible ### headers, or with some special syntax in blockquotes, but it shouldn't be HTML-based).

Small improvements to MarkdownViewer:

• there's no nested <pre> in its HTML
• there's no inline <style> that could affect styles outside of this component
• I suspect that highlighting performance might be better, because we don't depend on rehype-raw anymore, but haven't measured this

generateSummary.ts in AI package now uses ##### headers instead of summaries. This makes log pages slower to load; we can optimize this later if it becomes a problem. (either by implementing collapsible headers, or maybe by optimizing Squiggle components to render lazily on scroll).

Next 15

Fumadocs uses App Router, not pages/. Because it uses RSC, I had to add "use client" to all components in squiggle-components.

Also, due to vercel/next.js#71149 bug in the latest Next 14, I had to upgrade to Next 15, which was just released yesterday. It seems stable though, and dev builds are significantly faster (3x-10x).

I'm interested in upgrading the Hub to v15 too, but that's a separate task.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 78a66ca

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
squiggle-website	✅ Ready (Inspect)	Visit Preview	Oct 26, 2024 4:42pm

3 Skipped Deployments

Name	Status	Preview	Updated (UTC)
quri-hub	⬜️ Ignored (Inspect)	Visit Preview	Oct 26, 2024 4:42pm
quri-ui	⬜️ Ignored (Inspect)	Visit Preview	Oct 26, 2024 4:42pm
squiggle-components	⬜️ Ignored (Inspect)	Visit Preview	Oct 26, 2024 4:42pm

[berekuk]

One more thing:

• Next v15 uses React 19 RC (we'd have to upgrade to it anyway, eventually)
• @headlessui/react used __SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED (uh... Incompatible with React 19 due to use of __SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED tailwindlabs/headlessui#3167)
• @headlessui/react v2.2.0 was released 3 hours ago (yay)
• but versioned-components installed older squiggle versions that dependent on older @headlessui/react, which doesn't build with React 19 anymore

So I had to patch it with pnpm.overrides:

  "pnpm": {
    "overrides": {
      "@headlessui/react": "^2.2.0"
    }
  },

AFAICT older Squiggle playgrounds still render correctly, and we used Headless UI only for StyledTab that was unused there.

[berekuk]

AFAICT this was unused

[berekuk]

Unused

[OAGr]

Cool to see the transition, but there are definitely things to fix, looking at the website.

The code styling is broken:

https://squiggle-website-o07a5slf8-quantified-uncertainty.vercel.app/docs/Ecosystem/BasicPrompt

Page headers seem to have too much margin on bottom.

Also, arguably h2s have too much margin on the top.

Can you remove the border on this header?

Can you remove the left border here? It's kind of interesting, but weird and distracting imo. Or just remove the diagonal lines or something.

Examples could use a stronger background color.

This isn't very important, but would be nice to improve if easy.

BasicPrompt.markdown link is broken.

I assume this would be easy to fix. It's also fine to just remove it.
https://squiggle-website-o07a5slf8-quantified-uncertainty.vercel.app/llms/BasicPrompt.markdown

From this page:
https://squiggle-website-o07a5slf8-quantified-uncertainty.vercel.app/docs/Ecosystem/LanguageModels

[berekuk]

The code styling is broken:

Fixed (was caused by import in BasicPrompt, needed to explicitly pass components)

Page headers seem to have too much margin on bottom.

Fixed for pages without description; others look mostly fine, IMO.

Also, arguably h2s have too much margin on the top.

This can be tuned, but doesn't seem like a regression to me (vertical height on old nextra pages look the same).

Can you remove the border on this header?

Technically, I can, but it would require replacing the entire <Nav> component in fumadocs, or applying fragile tailwind overrides from parent elements... So, too much work to be worth it, sorry.

Can you remove the left border here? It's kind of interesting, but weird and distracting imo. Or just remove the diagonal lines or something.

Yeah, done.

Examples could use a stronger background color.

Fixed (not a regression from Nextra v3, btw, but a regression from Nextra v2 which had white page background).

BasicPrompt.markdown link is broken.

Fixed, thanks (will work once we merge this PR, it links to Github main branch).