Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MariaDB #110

Merged
merged 10 commits into from
Jul 6, 2023
Merged

MariaDB #110

merged 10 commits into from
Jul 6, 2023

Conversation

Iain-S
Copy link
Collaborator

@Iain-S Iain-S commented Jun 29, 2023

Adds MariaDB support, which might also work for a wider array of DBs (sqlite perhaps?).

DSNs

This PR is a breaking change since we now require users to provide the DB connection DSNs (e.g. postgresql://user:secret@localhost:6789/dbname) rather than building them from SRC_DBNAME, SRC_HOST, SRC_PASSWORD, etc.
I find this less convenient, as an end user, but I think it's safest as the DSN is all we really need to pass to SQLAlchemy and this change

  1. works for mariadb and postgres
  2. future proofs us against other dbms we want to support
  3. allows users to specify weird options, like this two-server postgres dsn (not tested) postgresql://host1:123,host2:456/somedb

Choosing providers

We modify the method of choosing a provider for a given column type. We now use a dict and try to .get() a match based on the column type and whether the column has a length attribute. If that fails, we iterate over all known column types and check issubclass(our_column_type, dict_column_type). This should result in fewer Unsupported type... runtime errors.

Loan tutorial

We add a short tutorial page to describe how to run SSG on this dataset.

SQLite3

Further to my speculation above, I have tested SSG with a simple SQLite3 DB using these env vars:

src_dsn="sqlite:////Users/me/somedir/src.sqlite"
src_dsn="sqlite:////Users/me/somedir/dst.sqlite"

and it works!

@Iain-S Iain-S added the WIP Work in Progress (do not merge) label Jun 29, 2023
@Iain-S Iain-S force-pushed the mariadb branch 2 times, most recently from 57ed8ce to c5e9a27 Compare July 5, 2023 17:06
@Iain-S Iain-S requested a review from cptanalatriste July 5, 2023 17:11
@Iain-S Iain-S removed the WIP Work in Progress (do not merge) label Jul 5, 2023
Copy link
Collaborator

@cptanalatriste cptanalatriste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Only minor re-naming suggestions.

@Iain-S Iain-S mentioned this pull request Jul 6, 2023
@Iain-S Iain-S merged commit e56212e into main Jul 6, 2023
@Iain-S Iain-S deleted the mariadb branch July 6, 2023 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants