DeepSeek #11971

cuichenx · 2025-01-28T05:49:27Z

What does this PR do ?

Add DeepSeek V2-Lite, V2, and V3 (including R1) models.
Support model import (from HF), model export (to HF), SFT/LoRA recipes.

Currently not supported: SFT/LoRA with packed sequence, pretraining,

Collection: [Note which collection this PR will affect]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

nemo/collections/llm/__init__.py

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

Signed-off-by: Chen Cui <[email protected]>

This reverts commit 4e5929b.

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

Signed-off-by: Chen Cui <[email protected]>

# Conflicts: # nemo/collections/llm/gpt/model/deepseek.py

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

cuichenx · 2025-02-19T16:52:27Z

@JRD971000 @yaoyu-33 could you review the changes to ssm.py and mllama.py

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

Signed-off-by: Chen Cui <[email protected]>

nemo/collections/llm/recipes/deepseek_v2.py

nemo/collections/llm/recipes/deepseek_v2_lite.py

nemo/collections/llm/recipes/deepseek_v3.py

Signed-off-by: Chen Cui <[email protected]>

nemo/collections/llm/gpt/model/deepseek.py

nemo/collections/llm/recipes/deepseek_v2.py

nemo/collections/llm/recipes/deepseek_v2_lite.py

hemildesai

LGTM for llm/api and nemo.lightning related changes

Signed-off-by: Chen Cui <[email protected]>

Signed-off-by: cuichenx <[email protected]>

Signed-off-by: Chen Cui <[email protected]>

github-actions · 2025-02-22T04:05:47Z

[🤖]: Hi @cuichenx 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g

* initial commit Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * clean Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * fix mscale and remove debug code Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * remove MTP to avoid HF warning Signed-off-by: Chen Cui <[email protected]> * Revert "remove MTP to avoid HF warning" This reverts commit 4e5929b. * guard v3 args Signed-off-by: Chen Cui <[email protected]> * guard one more v3 arg Signed-off-by: Chen Cui <[email protected]> * add recipes (wip) Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * update recipes Signed-off-by: Chen Cui <[email protected]> * update recipes Signed-off-by: Chen Cui <[email protected]> * update to latest mcore Signed-off-by: Chen Cui <[email protected]> * update to latest mcore Signed-off-by: Chen Cui <[email protected]> * support lora for TELinear Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * support V2 lite Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * exporter Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * memory-efficient hf export Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * code scanning Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * comment out cli factory for pretraining recipes Signed-off-by: Chen Cui <[email protected]> * Support non-layernom column parallel layer for LoRA SP in MLA Signed-off-by: Chen Cui <[email protected]> * add v2-lite recipe Signed-off-by: Chen Cui <[email protected]> * recipe typos Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * linting Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * update recipes Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * guard packed sequence=false Signed-off-by: Chen Cui <[email protected]> * unused imports Signed-off-by: Chen Cui <[email protected]> * fix lora Signed-off-by: Chen Cui <[email protected]> * address comments Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * fix test Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Co-authored-by: cuichenx <[email protected]> Signed-off-by: oliver könig <[email protected]>

* DeepSeek (#11971) * initial commit Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * clean Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * fix mscale and remove debug code Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * remove MTP to avoid HF warning Signed-off-by: Chen Cui <[email protected]> * Revert "remove MTP to avoid HF warning" This reverts commit 4e5929b. * guard v3 args Signed-off-by: Chen Cui <[email protected]> * guard one more v3 arg Signed-off-by: Chen Cui <[email protected]> * add recipes (wip) Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * update recipes Signed-off-by: Chen Cui <[email protected]> * update recipes Signed-off-by: Chen Cui <[email protected]> * update to latest mcore Signed-off-by: Chen Cui <[email protected]> * update to latest mcore Signed-off-by: Chen Cui <[email protected]> * support lora for TELinear Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * support V2 lite Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * exporter Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * memory-efficient hf export Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * code scanning Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * comment out cli factory for pretraining recipes Signed-off-by: Chen Cui <[email protected]> * Support non-layernom column parallel layer for LoRA SP in MLA Signed-off-by: Chen Cui <[email protected]> * add v2-lite recipe Signed-off-by: Chen Cui <[email protected]> * recipe typos Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * linting Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * update recipes Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * guard packed sequence=false Signed-off-by: Chen Cui <[email protected]> * unused imports Signed-off-by: Chen Cui <[email protected]> * fix lora Signed-off-by: Chen Cui <[email protected]> * address comments Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * fix test Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Co-authored-by: cuichenx <[email protected]> Signed-off-by: oliver könig <[email protected]> * missing imports Signed-off-by: oliver könig <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Signed-off-by: oliver könig <[email protected]> Co-authored-by: Chen Cui <[email protected]> Co-authored-by: cuichenx <[email protected]>

cuichenx and others added 2 commits January 27, 2025 21:47

initial commit

e4d0cd9

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

325db93

Signed-off-by: cuichenx <[email protected]>

github-advanced-security bot found potential problems Jan 28, 2025

View reviewed changes

nemo/collections/llm/__init__.py Fixed Show fixed Hide fixed

cuichenx and others added 20 commits January 28, 2025 09:42

clean

c34778e

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

87836f6

Signed-off-by: cuichenx <[email protected]>

fix mscale and remove debug code

0fcdaa5

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

31b0f23

Signed-off-by: cuichenx <[email protected]>

remove MTP to avoid HF warning

4e5929b

Signed-off-by: Chen Cui <[email protected]>

Revert "remove MTP to avoid HF warning"

b3a7328

This reverts commit 4e5929b.

guard v3 args

d851bc7

Signed-off-by: Chen Cui <[email protected]>

guard one more v3 arg

f194158

Signed-off-by: Chen Cui <[email protected]>

add recipes (wip)

fb73013

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

fb90521

Signed-off-by: cuichenx <[email protected]>

update recipes

52f0778

Signed-off-by: Chen Cui <[email protected]>

update recipes

b92f6be

Signed-off-by: Chen Cui <[email protected]>

update to latest mcore

90c5eb1

Signed-off-by: Chen Cui <[email protected]>

update to latest mcore

fa67d3b

Signed-off-by: Chen Cui <[email protected]>

support lora for TELinear

21e3993

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

163ffe9

Signed-off-by: cuichenx <[email protected]>

support V2 lite

6d3708f

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

ad73088

Signed-off-by: cuichenx <[email protected]>

exporter

1b7ed6f

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

cec4326

Signed-off-by: cuichenx <[email protected]>

NVIDIA deleted a comment from github-actions bot Feb 12, 2025

cuichenx and others added 4 commits February 13, 2025 22:47

memory-efficient hf export

7b8940c

Signed-off-by: Chen Cui <[email protected]>

Merge remote-tracking branch 'origin/chcui/deepseek' into chcui/deepseek

f2c95c0

# Conflicts: # nemo/collections/llm/gpt/model/deepseek.py

Merge branch 'main' into chcui/deepseek

955e7ef

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

6937ae5

Signed-off-by: cuichenx <[email protected]>

cuichenx marked this pull request as ready for review February 14, 2025 03:53

cuichenx added the skip-linting label Feb 19, 2025

cuichenx and others added 3 commits February 20, 2025 15:30

update recipes

af49891

Signed-off-by: Chen Cui <[email protected]>

Apply isort and black reformatting

39f4f49

Signed-off-by: cuichenx <[email protected]>

guard packed sequence=false

198d7be

Signed-off-by: Chen Cui <[email protected]>

github-advanced-security bot found potential problems Feb 20, 2025

View reviewed changes

nemo/collections/llm/recipes/deepseek_v2.py Fixed Show fixed Hide fixed

nemo/collections/llm/recipes/deepseek_v2_lite.py Fixed Show fixed Hide fixed

nemo/collections/llm/recipes/deepseek_v3.py Fixed Show fixed Hide fixed

unused imports

74353dd

Signed-off-by: Chen Cui <[email protected]>

cuichenx added Run CICD r2.2.0 labels Feb 21, 2025

ko3n1g added Run CICD and removed Run CICD labels Feb 21, 2025

suiyoubi reviewed Feb 21, 2025

View reviewed changes

nemo/collections/llm/gpt/model/deepseek.py Show resolved Hide resolved

suiyoubi reviewed Feb 21, 2025

View reviewed changes

nemo/collections/llm/recipes/deepseek_v2.py Outdated Show resolved Hide resolved

suiyoubi reviewed Feb 21, 2025

View reviewed changes

nemo/collections/llm/recipes/deepseek_v2_lite.py Outdated Show resolved Hide resolved

hemildesai previously approved these changes Feb 21, 2025

View reviewed changes

cuichenx added 2 commits February 21, 2025 15:54

fix lora

556c864

Signed-off-by: Chen Cui <[email protected]>

address comments

8343149

Signed-off-by: Chen Cui <[email protected]>

cuichenx dismissed hemildesai’s stale review via 8343149 February 21, 2025 21:04

Apply isort and black reformatting

ae394ff

Signed-off-by: cuichenx <[email protected]>

cuichenx added Run CICD and removed Run CICD labels Feb 21, 2025

fix test

616f76d

Signed-off-by: Chen Cui <[email protected]>

cuichenx added Run CICD and removed Run CICD labels Feb 21, 2025

ko3n1g approved these changes Feb 22, 2025

View reviewed changes

ko3n1g merged commit b1dd398 into main Feb 22, 2025
192 checks passed

ko3n1g deleted the chcui/deepseek branch February 22, 2025 10:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DeepSeek #11971

DeepSeek #11971

cuichenx commented Jan 28, 2025 •

edited

Loading

cuichenx commented Feb 19, 2025

hemildesai left a comment

github-actions bot commented Feb 22, 2025

DeepSeek #11971

DeepSeek #11971

Conversation

cuichenx commented Jan 28, 2025 • edited Loading

What does this PR do ?

Changelog

Usage

GitHub Actions CI

Before your PR is "Ready for review"

Who can review?

Additional Information

cuichenx commented Feb 19, 2025

hemildesai left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 22, 2025

cuichenx commented Jan 28, 2025 •

edited

Loading