Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Sync] Readme #6

Merged
merged 18 commits into from
Oct 18, 2021
Merged

[Sync] Readme #6

merged 18 commits into from
Oct 18, 2021

Conversation

blefaudeux
Copy link
Contributor

What does this PR do?

Fixes # (issue).

Before submitting

  • Did you have fun?
    • Make sure you had fun coding 🙃
  • Did you read the contributor guideline?
  • Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
    • N/A
  • Did you make sure to update the docs?
    • N/A
  • Did you write any new necessary tests?
    • N/A
  • Did you update the changelog? (if needed)
    • N/A

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

blefaudeux and others added 18 commits October 13, 2021 20:46
@blefaudeux
[hotfix] doc build requires numpy (#316)
0f8f5b8
@pradeep90
[chore] Set up basic Pyre configuration.
b89c527 
+ Added `pyre-check` to `requirements-test.txt`.
+ Added a `.watchmanconfig` as that is needed for fast incremental results.
+ Added empty stubs for missing imports to `stubs/`.
  This is the equivalent of mypy's `--ignore-missing-imports`.

This PR has nothing to do with shape types.

Remaining work:
There are currently 85 Pyre errors. I will address the non-Tensor ones
in the next commit. The rest will be addressed when we add stubs for
Tensor shape types.
@blefaudeux
[fix] Code of conduct (#318)
d0a5742 
* adding the missing coc
* opportunist doc fix
@pradeep90
[chore] Fix other Pyre errors.
11e2aa0 
This fixes all of the valid errors.

The remaining 56 errors are stub issues that will be fixed when we move
to the Tensor shape type stubs.
@pradeep90
Merge pull request #310 from pradeep90/main
cf55c70 
[chore] Set up basic Pyre configuration.
@blefaudeux
[minor] adding an issue template (#319)
51a2bca
take in key padding mask in blocksparse attn
8102435
flag specifying whether masks must be passed in separately
2f1a42e
@blefaudeux
[fix] Adding missing license headers (#320)
5b495a1 
* adding missing licences, missing are sputnik but will set an exception
fix linting error, not related to previous changes
2869445
fix
904daee
fix default to False
f3de7d0
@dianaml0
Merge pull request #321 from fairinternal/key_padding_blocksparse
e763b07 
Take in key padding mask in blocksparse attention
@suchenzang
[feat] Bump triton to 1.1.1 in requirements-benchmark (#322)
70c7999
@blefaudeux
[feat][LN] Gracefully handle non contiguous tensors + unit test (#323)
92b3ad5
@blefaudeux
[minor] Triton layernorm integration in the encoder/decoder blocks (#317
18a6b1d 
)
@Vittorio-Caggiano @blefaudeux
Update README.md (#262)
b7c61f8 
* Update README.md

* adding a Transformer illustration, minor improvements on the repo map

* adding a `key features` section

* wip populating the foldable links

* Update README.md

* Update README.md

* revert accidental removal of references

* rebase onto main, add benchmark plots

* small additions and markdown linting

* go through some old FIXMEs

* adding the benchmark requirements
* adding more installation instructions

* more examples, restructuring a little

* Expanding on the HOWTO

* adding a note about Triton cache

* add a reference to the license in the README, as requested

* update figure size

* update badges

* adding a runnable example for sparse attention

* clearing up the message on FusedLinear kernel

* adding another small example + some explanations

* missing saved file, getting there

* fixing the xformers citation

* updating the plots, adding Layer norm

Co-authored-by: Benjamin Lefaudeux <[email protected]>
Co-authored-by: Benjamin Lefaudeux <[email protected]>
Co-authored-by: Benjamin Lefaudeux <[email protected]>
@blefaudeux
Merge branch 'origin_main' into sync_readme
da7995e
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 18, 2021
@blefaudeux blefaudeux merged commit 133595c into main Oct 18, 2021
@blefaudeux blefaudeux deleted the sync_readme branch October 18, 2021 17:38
danthe3rd pushed a commit that referenced this pull request Dec 8, 2022
Update on "Fix CI after torchmetrics update"
d119652 
It now takes an argument: https://torchmetrics.readthedocs.io/en/stable/classification/accuracy.html

Change in pytorch lightning:
Lightning-AI/torchmetrics@20eab43

Somehow this is failing with a SEGFAULT on my A100 (in a triton kernel):
```
#0  0x00007fffc0f62e10 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#1  0x00007fffc0f9303c in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#2  0x00007fffc0f2ea13 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#3  0x00007fffc0f94603 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#4  0x00007fffc119e4a0 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#5  0x00007fffc0f3728f in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#6  0x00007fffc0f3999f in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#7  0x00007fffc0fdb1c2 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#8  0x00007fff502234c0 in _launch ()
   from /data/home/XXXXX/.triton/cache/704a3e6949e60326bc68d18a620bee50/layer_norm_fw.so
#9  0x00007fff3c0eea25 in launch ()
   from /data/home/XXXXX/.triton/cache/2cebb5590a024a2e06fe9de08c6b7079/k_dropout_bw.so
#10 0x0000555555698422 in cfunction_call (func=0x7fff3c6e5760, args=<optimized out>, kwargs=<optimized out>)
    at /usr/local/src/conda/python-3.10.6/Objects/methodobject.c:552
```

[ghstack-poisoned]
danthe3rd pushed a commit that referenced this pull request Dec 8, 2022
Update on "Fix CI after torchmetrics update"
29ebaa4 
It now takes an argument: https://torchmetrics.readthedocs.io/en/stable/classification/accuracy.html

Change in pytorch lightning:
Lightning-AI/torchmetrics@20eab43

Somehow this is failing with a SEGFAULT on my A100 (in a triton kernel):
```
#0  0x00007fffc0f62e10 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#1  0x00007fffc0f9303c in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#2  0x00007fffc0f2ea13 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#3  0x00007fffc0f94603 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#4  0x00007fffc119e4a0 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#5  0x00007fffc0f3728f in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#6  0x00007fffc0f3999f in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#7  0x00007fffc0fdb1c2 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#8  0x00007fff502234c0 in _launch ()
   from /data/home/XXXXX/.triton/cache/704a3e6949e60326bc68d18a620bee50/layer_norm_fw.so
#9  0x00007fff3c0eea25 in launch ()
   from /data/home/XXXXX/.triton/cache/2cebb5590a024a2e06fe9de08c6b7079/k_dropout_bw.so
#10 0x0000555555698422 in cfunction_call (func=0x7fff3c6e5760, args=<optimized out>, kwargs=<optimized out>)
    at /usr/local/src/conda/python-3.10.6/Objects/methodobject.c:552
```

[ghstack-poisoned]
danthe3rd pushed a commit that referenced this pull request Dec 9, 2022
Update base for Update on "Fix CI after torchmetrics update"
d523e26 
It now takes an argument: https://torchmetrics.readthedocs.io/en/stable/classification/accuracy.html

Change in pytorch lightning:
Lightning-AI/torchmetrics@20eab43

Somehow this is failing with a SEGFAULT on my A100 (in a triton kernel):
```
#0  0x00007fffc0f62e10 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#1  0x00007fffc0f9303c in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#2  0x00007fffc0f2ea13 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#3  0x00007fffc0f94603 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#4  0x00007fffc119e4a0 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#5  0x00007fffc0f3728f in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#6  0x00007fffc0f3999f in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#7  0x00007fffc0fdb1c2 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#8  0x00007fff502234c0 in _launch ()
   from /data/home/XXXXX/.triton/cache/704a3e6949e60326bc68d18a620bee50/layer_norm_fw.so
#9  0x00007fff3c0eea25 in launch ()
   from /data/home/XXXXX/.triton/cache/2cebb5590a024a2e06fe9de08c6b7079/k_dropout_bw.so
#10 0x0000555555698422 in cfunction_call (func=0x7fff3c6e5760, args=<optimized out>, kwargs=<optimized out>)
    at /usr/local/src/conda/python-3.10.6/Objects/methodobject.c:552
```

[ghstack-poisoned]
danthe3rd pushed a commit that referenced this pull request Dec 9, 2022
Update on "Fix CI after torchmetrics update"
8a6130d 
It now takes an argument: https://torchmetrics.readthedocs.io/en/stable/classification/accuracy.html

Change in pytorch lightning:
Lightning-AI/torchmetrics@20eab43

Somehow this is failing with a SEGFAULT on my A100 (in a triton kernel):
```
#0  0x00007fffc0f62e10 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#1  0x00007fffc0f9303c in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#2  0x00007fffc0f2ea13 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#3  0x00007fffc0f94603 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#4  0x00007fffc119e4a0 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#5  0x00007fffc0f3728f in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#6  0x00007fffc0f3999f in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#7  0x00007fffc0fdb1c2 in ?? () from /lib/x86_64-linux-gnu/libcuda.so
#8  0x00007fff502234c0 in _launch ()
   from /data/home/XXXXX/.triton/cache/704a3e6949e60326bc68d18a620bee50/layer_norm_fw.so
#9  0x00007fff3c0eea25 in launch ()
   from /data/home/XXXXX/.triton/cache/2cebb5590a024a2e06fe9de08c6b7079/k_dropout_bw.so
#10 0x0000555555698422 in cfunction_call (func=0x7fff3c6e5760, args=<optimized out>, kwargs=<optimized out>)
    at /usr/local/src/conda/python-3.10.6/Objects/methodobject.c:552
```

[ghstack-poisoned]
qianfengz added a commit to qianfengz/xformers that referenced this pull request Feb 7, 2024
@qianfengz
Merge pull request facebookresearch#6 from sgrigory/add-triton-fa2
43e7797 
Add Triton Flash Attention 2 forward op
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@blefaudeux @facebook-github-bot @pradeep90 @dianaml0 @suchenzang @Vittorio-Caggiano