Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Layer improvement #90

Merged
merged 26 commits into from
Jul 26, 2024
Merged

Layer improvement #90

merged 26 commits into from
Jul 26, 2024

Conversation

AnFreTh
Copy link
Collaborator

@AnFreTh AnFreTh commented Jul 26, 2024

  • add embedding layer
  • add MambaTab
  • add RegLu and Attention for FTTransformer

AnFreTh and others added 26 commits July 15, 2024 13:53
@AnFreTh
add learnable ple encodings
dc4c313
@AnFreTh
add rotary embeddings and attention_net utils
f6aa9ac
@AnFreTh
add mambatab, rotaryft and rnn configs
04101f9
@AnFreTh
adapt mambatab config to paper hparams
fe40c47
@AnFreTh
include RNN and MambaTab
30d35e4
@AnFreTh
add mambatab, rnn, rotery and basisexpandFT to init
21b0afb
@AnFreTh
include basemodel basisexpansion
8b19f92
@AnFreTh
adding util embedding alyer for decluttering
bad3db6
@AnFreTh
adapted embedding layer for when no cat features
a2c3e71
@AnFreTh
adapted mambular, FT and TabTransformer for new embedding layer class
5f15d5f
@AnFreTh
included possible embeddings into MLP and ResNet
fa8cc00
@AnFreTh
adapted configs of models
6efc326
@AnFreTh
adapt documentation to new configs
ae0bf97
@AnFreTh
add missing line in MLP
6be79d1
@AnFreTh
added seq len to emebedding layer
3d840b9
@AnFreTh
added shuffling embeddings before being passed to mamba blocks
ae84257
@AnFreTh
shuffle embeddings option in config
e1f5c40
@AnFreTh
shuffling along second axis
8af86a2
@AnFreTh
add .built attr to model classed
2029700
@AnFreTh
delete expansion
c7459c4
@AnFreTh
delete basis expansion
96e9449
@AnFreTh
delete self.paper attribute from mambatab
864d4f0
@AnFreTh
adapt build model func
d636b28
@AnFreTh
adapt build_method
5a5a7cd
@AnFreTh
add embedding layer and delete unused models
283a10b
@AnFreTh
Merge branch 'develop' into layer_improvement
19b760c
@AnFreTh AnFreTh merged commit 56801dd into develop Jul 26, 2024
1 check passed
@AnFreTh AnFreTh deleted the layer_improvement branch July 26, 2024 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@AnFreTh