Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Processing overhaul #173

Merged
merged 55 commits into from
Aug 13, 2018
Merged

Processing overhaul #173

merged 55 commits into from
Aug 13, 2018

Conversation

andrew-morrison
Copy link
Contributor

@andrew-morrison andrew-morrison commented Aug 13, 2018

Approving this pull request will:

Fix #34
Fix #79
Fix #105
Fix #123
Fix #124
Fix #127
Fix #132
Fix #144
Fix #147
Fix #148
Fix #155
Fix #156
Fix #157
Fix #158
Fix #172

Some of the above require simultaneous changes to the Blacklight configuration, which are controlled by a different, private repository on our internal GitLab system.

So please switch your local copy to the processing-overhaul branch and try running index-all-qa.sh.

If that works, come back here and approve the pull request but do not run index-all-prd.sh.

Then tell me when you are ready to index to production (which will include publishing Merton and all your other recent changes as well as the above fixes and enhancements). I'll then run index-all-prd.sh and deploying the Blacklight configuration at the same time. After that, you'll be able to run it anytime you want.

Both places and organizations are in the same authority file, so it is more efficient to have one script to process that, which will be places.xquery. See #148.
Includes changes for #148 to speed up indexing and #34 to index extra info in the authority files. Renaming people.xquery to persons.xquery (naming scripts consistently with the indexes they generate so it is easier to run them concurrently.)
Instead of hardcoding in the XQuery.
Will allow searching for "Composite manuscript".
Will allow advanced search by language across both indexes.
They'll still be indexed as fulltext, but this might reduce the ranking of manuscripts versus people, works, etc, possibly helping with #158
For example, if no xml:id has been given to a manuscript's root TEI element.
In case filter wanted in works, like in Fihrist.
Get associations from works authority files as well as manuscript TEI files. See #79.
This doesn't work very well in a search across everything so restoring what is on productions. Could add a separate type facet in the future.
So, links to works by their title, authors by their name, etc.
Warnings logged first, before the info messages. Also more concise messages.
Seem to have been lost when merging branches.
Copy link
Collaborator

@holfordm holfordm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indexing works fine on my machine.

@holfordm holfordm merged commit 923cb64 into master Aug 13, 2018
@andrew-morrison andrew-morrison deleted the processing-overhaul branch August 13, 2018 16:47
@andrew-morrison
Copy link
Contributor Author

Great.

I forgot to mention, make sure you switch back to the master branch now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment