Skip to content

Commit

Permalink
updated readme
Browse files Browse the repository at this point in the history
  • Loading branch information
Vishwa44 committed May 13, 2024
1 parent d9ceed4 commit efddab9
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Our project is to improve the PatchTST model, this model is a Transformer model
- Developing a custom triton kernel:
To reduce computation overhead for the smaller model size of PatchTST we developed a custom kernel which reads attention scores directly from froward pass which utilized in backward pass.

The custom kerenl implmentation is in a py notebook format in the Kernel folder.

You can access the Flash attention and updated attention kernel on the compare_flash_attention pynotebook.
You can run it ny running the cells.
## Results:
-Our kernel has a 1.64 speed up for backward pass and 0.70 speed down for forward pass.
-This kernel specifically performs better on smaller context length models.
Expand Down

0 comments on commit efddab9

Please sign in to comment.