Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP merged up uv ports branches #380

Draft
wants to merge 28 commits into
base: master
Choose a base branch
from

Conversation

utnapischtim
Copy link
Contributor

@utnapischtim utnapischtim commented Nov 18, 2024

this PR is a combination of

this PR should not be merged as it is. it is a starting point to discuss improvements!

discussion points

  • some arguable (provoking) performance improvements e.g. reverted. setting it over the invenio call is necessary
  • should we go in and change without backwards compatibility to uv + rspack + pnpm? no, it will be configurable which tools will be used by the developer
  • should the lock file approach be applied completely? b6bbbe2
  • making python packages manager configurable would keep invenio-cli backwards compatible commit yes
  • making javascript packages manager configurable would keep invenio-cli backwards compatible commit needs commit yes
  • making assets builder configurable would keep invenio-cli backwards compatible commit needs commit invenio-assets commit has to be finished if we use the approach (webpack.config.js has to be reverted back to real webpack configuration) yes
  • how make it possible to use invenio-cli installed globally without doing sys.path.insert like

steps to how to use it

1.) check that uv and pnpm are installed
2.) create a new my-site directory with the regular content as usual OR go into your current development my-site directory
2.a) deactivate the active environment
3.) create with uv a virtual environment with uv venv --prompt uv-env && source .venv/env/activate
4.) install invenio-cli with uv pip install "git+https://github.com/utnapischtim/invenio-cli@WIP-merged-up-uv-ports-branches"
5.) copy following text into the .invenio file into the cli section

python_packages_manager = uv
javascript_packages_manager = pnpm
assets_builder = rspack

6.) copy following text into the pyproject.toml file. the file should be created beside the invenio.cfg file

[project]
name = "InvenioRDM"
requires-python = ">= 3.12"
dynamic = ["version"]

dependencies = [
  "invenio-app-rdm[opensearch2]~=13.0.0b2.dev0",
  "uwsgi>=2.0",
  "uwsgitop>=0.11",
  "uwsgi-tools>=1.1.1",

  # keep the custom invenio-cli installed in the virtual environment
  # could possible create problems with global installed invenio-cli
  # because the path to the virtual environmental invenio-cli will not
  # be updated
  "invenio-cli",

  # rspack
  "flask-webpackext",
  "invenio-assets",
  "pywebpack",
  
  # for app-rdm b2.dev0
  # this is necessary because we have still dev packages for flask>=3.0 and sqlalchemy>=2.0
  "invenio-communities>=18.0.0.dev1",
  "invenio-rdm-records>=17.0.0.dev1",
  "invenio-requests>=6.0.0.dev1",
  "invenio-vocabularies>=7.0.0.dev1",
  "invenio-jobs>=3.0.0.dev1",
  "invenio-users-resources>=7.0.0.dev1",
]

[tool.setuptools]
py-modules = [] # necessary to make the packages with setup.py usable with uv

# overrides packages from "dependencies"
[tool.uv.sources]
invenio-cli = { git = "https://github.com/utnapischtim/invenio-cli", branch = "WIP-merged-up-uv-ports-branches" }
flask-webpackext = { git = "https://github.com/utnapischtim/flask-webpackext", branch = "make-ready-for-rspack" }
invenio-assets = { git = "https://github.com/slint/invenio-assets", branch = "rspack" }
pywebpack = { git = "https://github.com/utnapischtim/pywebpack", branch = "add-scripts" }
# invenio-cli = { path = "path/to/invenio-cli", editable = true } # would be local example

7.) run time invenio-cli install to install and to see how long it takes
8.) run time invenio-cli install again to see the behavior with hot cache

utnapischtim and others added 13 commits November 5, 2024 15:20
* this gives a huge performance improvement on installing and setting up
  an environment.
* Explicitly makes the Celery worker listen to the "default" and "low"
  queues.
* the problem was that the packages which are installed editable and
  will be used on the assets build step arent't found if the assets
  build method is called during the invenio-cli install command.
* this commit assumes that the instance path isn't customizable and
  therefore removes the necessity of starting up the app to get the
  instance path.

* the case where the instance path is overridable should be handled in
  another commit.
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from d9fc688 to 128053b Compare November 18, 2024 14:25
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch 4 times, most recently from 94f5361 to 79603c3 Compare January 18, 2025 19:54
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from 79603c3 to bc977c8 Compare January 18, 2025 20:12
* to create the assets create_ui is enough. the assets only need the
  blueprints and they exists in create_ui. creat_ui only creates one
  app and not two as create_app does
* since rspack configuration is not more in webpack.config.js this is
  necessary
@utnapischtim utnapischtim force-pushed the WIP-merged-up-uv-ports-branches branch from f48e721 to 5ddbc3f Compare January 21, 2025 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants