opensora v1.2 i2v oom

[fenghe12]

[2025-01-12 09:47:20] INFO: Sampling captions:
[2025-01-12 09:47:20] INFO: In an ornate, historical hall, a massive tidal wave peaks and begins to crash. Two surfers, seizing the moment, skillfully navigate the face of the wave.
0%| | 0/30 [00:00<?, ?it/s][WARNING] PRE_ACT(145615,fffdb4551180,python):2025-01-12-09:48:10.866.728 [mindspore/ccsrc/backend/common/somas/somas.cc:479] VerifySomasResult] Mismatch ref node size 2301 vs 2305
[WARNING] PRE_ACT(145615,fffdb4551180,python):2025-01-12-09:48:10.866.815 [mindspore/ccsrc/backend/common/somas/somas.cc:441] LoadSomasResult] Verify Somas Result Failed.
[ERROR] PRE_ACT(145615,fffbdf67f120,python):2025-01-12-09:48:13.703.514 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:392] AddMemBlockAndMemBufByEagerFree] TotalUsedMemStatistics : 24725382656 plus TotalUsedByEventMemStatistics : 0 and plus alloc size : 100555779072 is more than total mem size : 61079552000.
[ERROR] PRE_ACT(145615,fffbdf67f120,python):2025-01-12-09:48:13.704.851 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:392] AddMemBlockAndMemBufByEagerFree] TotalUsedMemStatistics : 24753031680 plus TotalUsedByEventMemStatistics : 0 and plus alloc size : 49766401024 is more than total mem size : 61079552000.
[ERROR] RUNTIME_FRAMEWORK(145615,fffbdf67f120,python):2025-01-12-09:48:13.705.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/memory_manager_actor.cc:412] SetOpContextMemoryAllocFail] #umsg#Memory not enough:#umsg#Device(id:0) memory isn't enough and alloc failed, kernel name: kernel_graph_7_MemoryAllocActor, alloc size: 49766400512B.

[fenghe12]

[hadipash]

Can you please provide the command you use to run sampling?

[fenghe12]

python scripts/inference.py --config configs/opensora-v1-2/inference/sample_iv2v.yaml --ckpt_path /path/to/your/opensora-v1-1.ckpt
just what you provided

[fenghe12]

i only changed path to dit weight

[hadipash]

And what MindSpore version are you using?

[Songyuanwei]

看一下日志里有没有jit level O2的相关字样，如果有，unset RANK_TABLE_FILE 后尝试重新运行