Unofficial implementations of Block Dropping and Layer Dropping in What Matters in Transformers? Not All Attention is Needed

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
experiments		experiments
finetune_utils		finetune_utils
phi_attn_pruned		phi_attn_pruned
qwen2_attn_pruned		qwen2_attn_pruned
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
attn_drop.py		attn_drop.py
decode_layer_drop.py		decode_layer_drop.py
finetune_attn_pruned.py		finetune_attn_pruned.py
hfd.sh		hfd.sh
recovery_finetune.py		recovery_finetune.py
run_attn_drop.py		run_attn_drop.py
run_decode_layer_drop.py		run_decode_layer_drop.py
utils.py		utils.py
wrapper.py		wrapper.py

Provide feedback