OMP: Error #15: Initializing libiomp5md.dll

[knslee07]

How to reproduce the behaviour

I followed the quick start on the tutorial website. I chose only ner, cpu, and accuracy. Then, I downloaded the base_config and made config.cfg by "python -m spacy init fill-config base_config.cfg config.cfg"
After input "python -m spacy train config.cfg --output ./output --paths.train ./train.spacy --paths.dev ./dev.spacy",
I got the following error. I am not an expert in coding, and do not know what multiple copies of the OpenMP runtime means. I greatly appreciate your time to look into this. Thank you.

ℹ Using CPU

=========================== Initializing pipeline ===========================
[2021-06-11 15:23:10,837] [INFO] Set up nlp object from config
[2021-06-11 15:23:10,844] [INFO] Pipeline: ['tok2vec', 'ner']
[2021-06-11 15:23:10,846] [INFO] Created vocabulary
[2021-06-11 15:23:12,406] [INFO] Added vectors: en_core_web_lg
[2021-06-11 15:23:12,406] [INFO] Finished initializing nlp object
[2021-06-11 15:23:13,075] [INFO] Initialized pipeline components: ['tok2vec', 'ner']
✔ Initialized pipeline

============================= Training pipeline =============================
ℹ Pipeline: ['tok2vec', 'ner']
ℹ Initial learn rate: 0.001
E    #       LOSS TOK2VEC  LOSS NER  ENTS_F  ENTS_P  ENTS_R  SCORE
---  ------  ------------  --------  ------  ------  ------  ------
OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

Your Environment

spaCy version 3.0.6
Location C:\ProgramData\Anaconda3\envs\spacy\lib\site-packages\spacy
Platform Windows-10-10.0.19042-SP0
Python version 3.8.10
Pipelines en_core_web_lg (3.0.0), en_core_web_trf (3.0.0)

[polm]

Hey, sorry you're having trouble with this. We've seen this several times before, specifically with Anaconda on Windows, and it seems to be an issue with your environment rather than with spaCy. Like the error message says you can make the error go away like this:

import os
os.environ["KMP_DUPLICATE_LIB_OK"] = "True"

Or by otherwise setting that environment variable. That can potentially cause problems but we haven't had reports of any.

Note #7664 is the same issue.

[knslee07]

Hello Polm,

Thanks for your answer. I will figure out myself by following the projects. Thanks.

[Dhilen-Chin]

Hey, sorry you're having trouble with this. We've seen this several times before, specifically with Anaconda on Windows, and it seems to be an issue with your environment rather than with spaCy. Like the error message says you can make the error go away like this:

import os
os.environ["KMP_DUPLICATE_LIB_OK"] = "True"

Or by otherwise setting that environment variable. That can potentially cause problems but we haven't had reports of any.

Note #7664 is the same issue.

Where are we meant to add these two lines?

[polm]

You can add them at the top of any code you've written that's giving this error.

If you're using something like spacy train, it's just an environment variable, so you can do something like:

KMP_DUPLICATE_LIB_OK=True spacy train ... whatever ...

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.