Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make load_big_file work with read-only file #1353

Merged
merged 1 commit into from
Jan 14, 2020

Conversation

jjjamie
Copy link
Contributor

@jjjamie jjjamie commented Jan 14, 2020

Use "rb" rather than "r+b" and specify access=mmap.ACCESS_READ

I would like to be able to read model files on a read-only share but current code crashes with permissions problems - this fixes for me

Use "rb" rather than "r+b" and specify access=mmap.ACCESS_READ
@@ -22,16 +22,16 @@
logger = logging.getLogger("flair")


def load_big_file(f):
def load_big_file(f: str) -> mmap.mmap:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And some type annotations!

@jjjamie jjjamie requested a review from alanakbik January 14, 2020 18:59
@djstrong
Copy link
Contributor

It must be made as an option. Models are also loaded for further training.

@alanakbik
Copy link
Collaborator

@jjjamie thanks for adding this!

@djstrong I've done some tests in resuming training on checkpoints or shipped models. It still works with the change because new models after training are written out as separate files.

@djstrong
Copy link
Contributor

@alanakbik Ok, you are right!

@alanakbik
Copy link
Collaborator

👍

@alanakbik alanakbik merged commit da01dff into flairNLP:master Jan 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants