Movielens20m dataset

[zkid18]

Pull Request FAQ

• documentation
• contribution guide
• minimal examples section
• changelog for main framework updates
• Catalyst slack (#__questions channel) for issue discussion

Description

Movielens 20M dataset for RecSys.
Current implementation progress:

[x] download dataset
[x] parser and pre-processing for the rating.csv
[x] user-based train/test splitter
[ ] interaction matrix generation
[ ] seq dataset generation
[ ] upload dataset in sparse format in PyTorch.

I stuck with a proper final representation of the dataset.

Couple words on Movielens20M dataset.
1-5 can be considered as context-based information, while rating provides the user interaction data. Hence we can omit 1-5 at that time.

    The data are contained in six files: 
    1. genome-scores.csv 
    2. genome-tags.csv 
    3. links.csv 
    4. movies.csv
    5. tags.csv  
    6. ratings.csv

rating.csv head -5

   userId  movieId  rating   timestamp
0       1        2     3.5  1112486027
1       1       29     3.5  1112484676
2       1       32     3.5  1112484819
3       1       47     3.5  1112484727
4       1       50     3.5  1112484580

So I have split the dataset randomly by users. I need to better understand how the dataset for the sequential RecSys should look like.

Related Issue

Type of Change

• Examples / docs / tutorials / contributors update
• Bug fix (non-breaking change which fixes an issue)
• Improvement (non-breaking change which improves an existing feature)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Checklist

• Have you updated tests for the new functionality?
• Have you added your new classes/functions to the docs?
• Have you updated the CHANGELOG?
• Have you run colab minimal CI/CD with latest and minimal requirements?
• Have you checked XLA integration with single and multiple processes?

[pep8speaks]

Hello @zkid18! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-12-27 05:25:19 UTC

[Scitator]

it would be great if we could correct the code style and add a few usage examples for the dataset

[mergify]

This pull request is now in conflicts. @zkid18, could you fix it? 🙏

[zkid18]

https://colab.research.google.com/drive/1l6mpAWBKbkEWILnz9ThwpAoTQDCXkG-M?authuser=1#scrollTo=HkzzfT1moccj&uniqifier=1

Colab to test MovieLens20m

[CLAassistant]

All committers have signed the CLA.

[mergify]

This pull request is now in conflicts. @zkid18, could you fix it? 🙏

[Scitator]

todo: torch>=1.3.0 ;)

Pull request has been modified.

[Scitator]

1.0?

[Scitator]

1.0? or 0.25.0

Pull request has been modified.

[Scitator]

could we use TemporaryDirectory? like https://github.com/catalyst-team/catalyst/blob/e1d78b7dd568c4d5cd94be417d6d8b93ef27b2e5/tests/catalyst/runners/test_train_flags.py

[Scitator]

comments ;)

Pull request has been modified.