Support running myst server behind JupyterHub URL proxy

[fperez]

Which area is this feature request for?

Command Line Tools

Describe the feature you'd like to request

While it's not super convenient, it is possible to build jupyter-books to be viewed in a hosted JuptyerHub instance thanks to the URL proxy it uses. In that case, one has to invoke sphinx manually in order to force a special html_baseurl, via:

jupyter-book config sphinx .
sphinx-build  . _build/html -D html_baseurl=${JUPYTERHUB_SERVICE_PREFIX}/proxy/absolute/8000

But once this is done, one can simply run in the _build/html directory the command:

python -m http.server

and access the URL https://your-jupyterhub.org/user-redirect/proxy/8000/index.html.

When I try to follow the equivalent process with myst, by running the myst server and trying to access .../proxy/3000/index.html, I get redirected to a page that looks like this:

Describe the solution you'd like

Ideally, if the environment variable ${JUPYTERHUB_SERVICE_PREFIX} is set, then the myst server would respond with the proper handling equivalent to the sphinx config example above. And in a future setting where static HTML builds (à la sphix/jbook) are possible, then it would also recognize this.

If doing it automatically isn't desirable, then at least having the option to manually set the base URL like sphinx supports would work.

Describe alternatives you've considered

No response

[welcome]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

Welcome to the EBP community! 🎉

[rowanc1]

Thanks @fperez - figuring out how best to make this work is going to be a bit tricky. In the current app, there are two servers involved (one for content, and one for server-side rendering), both need to be coordinated on the base path, and that is relatively involved in the routing framework that we use.

Understanding the use case better for this: this is to give users of a jupyterhub a local development experience where they can preview the work they are doing?

There might be a few other solutions that are easier: (1) pushing ahead on jupyterlab-myst to make a multi-page experience, as well as support markdown files; (2) having a jupyterlab plugin that exposes a new URL specific for myst websites (not sure how hard this is); (3) create a new template that has routing specific for use in jupyterlab (I think this could suffer the same problems as the current app); (4) focusing on static rendering #188 which you hinted at as well.

I will continue to think on this. I think that some combination of (1) and (4) are probably the most achievable in the short-ish term.

[fperez]

Thx @rowanc1 - for reference, it's for students/users (myself included) to be able to test their site builds "locally" when "locally" means in a cloud-hosted hub, before they push anything to Github. This page summarizes the process for doing it with JupyterBook/Sphinx.

Meeting that need will be pretty critical for users to rely on the myst toolchain regularly when working on hosted hubs.

[fperez]

Understanding the use case better for this: this is to give users of a jupyterhub a local development experience where they can preview the work they are doing?

Yes! (I should have just said that instead of rambling above :)

[rowanc1]

If we made an in-jupyterlab preview (e.g. in a native jupyterlab tab, more similar to the markdown --> markdown preview experience), which allows you to navigate and preview the final rendered book, would that satisfy the use case?

[fperez]

As you know, I'm hugely 👍 on your (4) above, as I think for many use cases that's a very important capability.

But I've also been thinking about how we can ensure that in general, apps that rely on running a local web app work as seamlessly as possible on a hosted hub. Here's an example of setting that up for nbdime that I got running yesterday.

I've been pondering, together with @yuvipanda and @ryanlovett, how we might do this more generically for all such apps. There may be a good convention we could start honoring about recognizing some environment variable that indicates the web app will be proxied, with enough info for the app to adjust its various base URL settings to work on a hosted hub.

Input welcome, this is definitely above my JS/web pay grade. I just want it to work :)

[fperez]

If we made an in-jupyterlab preview (e.g. in a native jupyterlab tab, more similar to the markdown --> markdown preview experience), which allows you to navigate and preview the final rendered book, would that satisfy the use case?

Yes, I think that would go a very long way, and if it's actually reliable it might be enough. I do worry about CSS clashes with Lab itself, and that it may be tricky to get the whole thing to render correctly. But there may be enough iframe tricks that can be played there for it to all work. As you know, my knowledge in this space is near zero, so I trust you!

[rowanc1]

@fperez I think that this might work now with #188 complete.

You should be able to set the BASE_URL property:

export BASE_URL=${JUPYTERHUB_SERVICE_PREFIX}/proxy/absolute/8000
myst build --html
cd _build/html
python -m http.server

Right now, the live server is not setup to do this as there are a few other things that make it less possible, but with static export, this unblocks the use case of at least being able to see the built html pages (in static form). If this works, we can start to streamline it with some of your other suggestions..!

[jmunroe]

I'd like to pick this up as a needed feature to be able have full featured jupyterlab based experience for myst document authoring.

Having a feature like

myst start --base_url "https://jupyterhub.2i2c.cloud/user-redirect/proxy/3000"

that would allow rendered web version for encouraging users to author myst documents entirely within a JupyterLab environment.

Bonus feature request: create a JupyterLab extension that wraps the myst start CLI and opens a window within JupyterLab that serves the myst server in an IFrame.

[ryanlovett]

@jmunroe That would be great, thanks! Running a static server as described above showed the html content, but the CSS was missing. I also tried the non-absolute variant.

Ideally, there could be a jupyter-myst-proxy extension which knew how to launch myst in the right way, so that all users have to do is click on a jupyterlab launcher. jupyter-shiny-proxy is a comparable example. However shiny-server can also render directories for which there is no app configured. It just displays a file listing. The user can navigate through the filesystem until shiny-server sees a shiny app, then it does the right thing. I don't think myst can do that.

To your bonus feature request, in jupyter-server-proxy, if new_browser_tab=True is specified, the proxied app will show up in a jupyterlab tab (in workspace) rather than a new browser tab.

[cboettig]

Hi friends, any update on this?

[agoose77]

Hi @cboettig, not quite yet. Right now, a big focus is on releasing JB2 on top of MyST, so that's where most of my cycles are going. I will take another look shortly, though; it may be a fresh pair of eyes is useful!

[cboettig]

ok no worries. when building directly with

jupyter-book build .

I've found I can just run http.server

python -m http.server --directory ${PWD}/_build/html

and open https://${BASE}${JUPYTERHUB_SERVICE_PREFIX}proxy/8000/index.html on a hub no problem. I did get teh same miss-rendered results as @fperez above doing the myst build from the docs. (I'm still a bit fuzzy about the difference, as I understand it, jupyter-book is ~ sphinx + myst-markdown / yaml config, but I'm still wrapping my head around how it all fits together....

[agoose77]

Jupyter book 1 and mystmd are entirely different tools, joined by a common (mostly) language (MyST flavoured markdown). Jupyter book 2 will be built on MyST, and we intend to make it easy and convenient to preview a full myst site from a hub. At the same time, we intend to improve the jupyterlab preview experience of myst.

[parisni]

Having a feature like
myst start --base_url "https://jupyterhub.2i2c.cloud/user-redirect/proxy/3000"

that would feet our use case which is not related w/ jupyter at all.
it's just about serving some content with myst on a given base-url.

[parisni]

workaround to get a live server with a base url:

• myst start does not honour BASE_URL -> use all this stuff
• myst build --html --watch does not watch -> use watchdog
• myst build --html will keep node working in the background... so kill it myst build --html not shutting down node server #1594
• use the only webserver not affected by Rewrite internal links to include .html for static html (based on config) #950 (python http-server, nginx ... are broken)

export BASE_URL=foobar
nohup watchmedo shell-command -v \
        --patterns='*.md;*.ipynb' \
        --recursive \
        --command='myst build --html && pkill node' \
        --ignore-patterns='./_build/html/build/*;nohup.out' \
        --drop \
        . &
cd _build/html
printf y|npx http-server --port 8000 --cors

[fperez]

Thanks for figuring out that workaround and sharing, @parisni! Ultimately we'll want a cleaner solution to be a default, but I'm perfectly OK using this for now if needed, it definitely isn't trivial to figure out so your help is much appreciated!