Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Enhance the execution speed of CPM dataloaders (#230)
* Update start_pytorch_task.py - Handle non-zero return code in process execution feat: Handle non-zero return code in process execution Refactor the code to check the return code of each process execution. If the return code is non-zero, an exception is raised with a descriptive error message indicating the process ID and suggesting to check the relevant issue for further details. * Enhance the execution speed of CPM dataloaders Enhance the execution speed of CPM dataloaders, potentially reducing the time by around 30 seconds, subject to potential variations due to different environments Initialize jieba library using jieba.initialize()
- Loading branch information