当我给pytorch的dataloader设置num_workers=4进行并行时，出现的warning。

To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)
The current process just got forked. Disabling parallelism to avoid deadlocks...
To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)
The current process just got forked. Disabling parallelism to avoid deadlocks...
To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)
The current process just got forked. Disabling parallelism to avoid deadlocks...
To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)

解决方法warning已经告诉我们了，为了避免死锁，需要我们手动显式设置该变量TOKENIZERS_PARALLELISM是否支持tokens并行。即在程序前面加上以下代码行就可以。

import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"

或

import os
os.environ["TOKENIZERS_PARALLELISM"] = "true"