网站优化推广怎么做,英语卷子哪个网站可以做,网站建设有什么品牌,四平seo记录使用pyinstaller打包含有pytorch和transformers库的程序时遇到的问题和解决方法。
环境和版本信息
操作系统#xff1a;Windows 11
Python#xff1a;3.10.12
pyinstaller#xff1a;5.13.0
torch#xff1a;2.2.2
transformers#xff1a;4.40.1
打包过程和问…记录使用pyinstaller打包含有pytorch和transformers库的程序时遇到的问题和解决方法。
环境和版本信息
操作系统Windows 11
Python3.10.12
pyinstaller5.13.0
torch2.2.2
transformers4.40.1
打包过程和问题
打包命令pyinstaller -w -F mainwindow.py
问题1transformers找不到相关python包的元数据metadata
打包完成后在mainwindow.py所在目录下会生成一个dist文件夹里面是打包生成的exe文件这里是mainwindow.exe直接双击执行该程序会出现如下错误 Traceback (most recent call last):File transformers\utils\versions.py, line 102, in require_versionFile importlib\metadata\__init__.py, line 996, in versionFile importlib\metadata\__init__.py, line 969, in distributionFile importlib\metadata\__init__.py, line 548, in from_name
importlib.metadata.PackageNotFoundError: No package metadata was found for tqdmDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File mainwindow.py, line 10, in moduleFile PyInstaller\loader\pyimod02_importers.py, line 385, in exec_moduleFile task_thread.py, line 3, in moduleFile PyInstaller\loader\pyimod02_importers.py, line 385, in exec_moduleFile image_deduplicate.py, line 15, in moduleFile PyInstaller\loader\pyimod02_importers.py, line 385, in exec_moduleFile transformers\__init__.py, line 26, in moduleFile PyInstaller\loader\pyimod02_importers.py, line 385, in exec_moduleFile transformers\dependency_versions_check.py, line 57, in moduleFile transformers\utils\versions.py, line 117, in require_version_coreFile transformers\utils\versions.py, line 104, in require_version
importlib.metadata.PackageNotFoundError: No package metadata was found for The tqdm4.27 distribution was not found and is required by this application.
Try: pip install transformers -U or pip install -e .[dev] if youre working with git main从错误信息可以看出错误原因是找不到包tqdm的元数据应该是打包时没有把这些数据一起打包过去看了下pyinstaller的使用说明有个参数可以解决这个问题
--copy-metadata PACKAGENAMECopy metadata for the specified package. This option can be used multiple times.在打包命令中加了该参数并指定tqdm包后又出现了其他没有找到元数据的包重复多次后才将所有这些包的元数据都添加进去这些包包括tqdm、regex、requests、packaging、filelock、numpy、huggingface-hub、safetensors、pyyaml最后的打包命令如下
pyinstaller -w -F --copy-metadata tqdm --copy-metadata regex --copy-metadata requests --copy-metadata packaging --copy-metadata filelock --copy-metadata numpy --copy-metadata huggingface-hub --copy-metadata safetensors --copy-metadata pyyaml mainwindow.py但是像这样在打包命令中一个一个加参数比较麻烦其实在这个过程中可以发现执行打包命令时会先生成一个spec文件这个文件是打包时pyinstaller根据传递给它的参数生成的一个python文件里面说明了把一个py文件打包成exe程序需要执行的操作。在打包命令中添加--copy-metadata会相应地在生成spec文件中添加如下代码
from PyInstaller.utils.hooks import copy_metadatadatas []
datas copy_metadata(tqdm)
datas copy_metadata(regex)
datas copy_metadata(requests)
datas copy_metadata(packaging)
datas copy_metadata(filelock)
datas copy_metadata(numpy)
datas copy_metadata(huggingface-hub)
datas copy_metadata(safetensors)
datas copy_metadata(pyyaml)所以可以先使用pyinstaller -w -F mainwindow.py先生成spec文件再对这个文件进行修改。使用spec文件打包只需要执行命令pyinstaller mainwindow.spec即可不需要添加其他参数因为参数对应的操作已经编码在spec文件中了。
问题2没有transformers/__init__.py文件
上述问题解决后再执行打包后的exe程序又会出现以下错误 Traceback (most recent call last):File transformers\utils\import_utils.py, line 1510, in _get_moduleFile importlib\__init__.py, line 126, in import_moduleFile frozen importlib._bootstrap, line 1050, in _gcd_importFile frozen importlib._bootstrap, line 1027, in _find_and_loadFile frozen importlib._bootstrap, line 1006, in _find_and_load_unlockedFile frozen importlib._bootstrap, line 688, in _load_unlockedFile PyInstaller\loader\pyimod02_importers.py, line 385, in exec_moduleFile transformers\models\auto\processing_auto.py, line 28, in moduleFile frozen importlib._bootstrap, line 1027, in _find_and_loadFile frozen importlib._bootstrap, line 1006, in _find_and_load_unlockedFile frozen importlib._bootstrap, line 688, in _load_unlockedFile PyInstaller\loader\pyimod02_importers.py, line 385, in exec_moduleFile transformers\processing_utils.py, line 46, in moduleFile transformers\utils\import_utils.py, line 1539, in direct_transformers_importFile frozen importlib._bootstrap_external, line 879, in exec_moduleFile frozen importlib._bootstrap_external, line 1016, in get_codeFile frozen importlib._bootstrap_external, line 1073, in get_data
FileNotFoundError: [Errno 2] No such file or directory: C:\\Users\\yuany\\AppData\\Local\\Temp\\_MEI264762\\transformers\\__init__.pyThe above exception was the direct cause of the following exception:Traceback (most recent call last):File mainwindow.py, line 10, in moduleFile frozen importlib._bootstrap, line 1027, in _find_and_loadFile frozen importlib._bootstrap, line 1006, in _find_and_load_unlockedFile frozen importlib._bootstrap, line 688, in _load_unlockedFile PyInstaller\loader\pyimod02_importers.py, line 385, in exec_moduleFile task_thread.py, line 3, in moduleFile frozen importlib._bootstrap, line 1027, in _find_and_loadFile frozen importlib._bootstrap, line 1006, in _find_and_load_unlockedFile frozen importlib._bootstrap, line 688, in _load_unlockedFile PyInstaller\loader\pyimod02_importers.py, line 385, in exec_moduleFile image_deduplicate.py, line 15, in moduleFile frozen importlib._bootstrap, line 1075, in _handle_fromlistFile transformers\utils\import_utils.py, line 1501, in __getattr__File transformers\utils\import_utils.py, line 1500, in __getattr__File transformers\utils\import_utils.py, line 1512, in _get_module
RuntimeError: Failed to import transformers.models.auto.processing_auto because of the following error (look up to see its traceback):
[Errno 2] No such file or directory: C:\\Users\\yuany\\AppData\\Local\\Temp\\_MEI264762\\transformers\\__init__.py
这个错误说明没有把transformers相关的文件打包进去可以通过在spec文件中添加如下代码解决该问题
from PyInstaller.utils.hooks import copy_metadata, collect_data_files datas []
datas copy_metadata(tqdm)
datas copy_metadata(regex)
datas copy_metadata(requests)
datas copy_metadata(packaging)
datas copy_metadata(filelock)
datas copy_metadata(numpy)
datas copy_metadata(huggingface-hub)
datas copy_metadata(safetensors)
datas copy_metadata(pyyaml)
datas collect_data_files(transformers, include_py_filesTrue, includes[**/*.py])现在可以通过执行命令pyinstaller mainwindow.spec再次打包成exe程序。
问题3找不到PyTorch和Tokenizers
再次启动生成的exe程序又会出现如下错误 None of PyTorch,TensorFlow 2.0,or Flax have been found. Models wont be available and only tokenizers, configuration and file/data utilities can be used.ImportError:
CLIPTokenizerFast requires the Tokenizers library but it was not found in your environment. You can install it with:
pip install tokenizers
说明找不到pytorch和tokenizers库以通过在spec文件中添加如下代码解决该问题
from PyInstaller.utils.hooks import copy_metadata, collect_data_filesdatas []
datas copy_metadata(tqdm)
datas copy_metadata(regex)
datas copy_metadata(requests)
datas copy_metadata(packaging)
datas copy_metadata(filelock)
datas copy_metadata(numpy)
datas copy_metadata(huggingface-hub)
datas copy_metadata(safetensors)
datas copy_metadata(pyyaml)
datas copy_metadata(tokenizers)
datas copy_metadata(torch)
datas collect_data_files(transformers, include_py_filesTrue, includes[**/*.py])
datas collect_data_files(tokenizers, include_py_filesTrue, includes[**/*.py])
datas collect_data_files(torch, include_py_filesTrue, includes[**/*.py])总结
最终的spec文件如下记得需要把datas传入到Analysis和EXE中
# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import copy_metadata, collect_data_filesdatas []
datas copy_metadata(tqdm)
datas copy_metadata(regex)
datas copy_metadata(requests)
datas copy_metadata(packaging)
datas copy_metadata(filelock)
datas copy_metadata(numpy)
datas copy_metadata(huggingface-hub)
datas copy_metadata(safetensors)
datas copy_metadata(pyyaml)
datas copy_metadata(tokenizers)
datas copy_metadata(torch)
datas collect_data_files(transformers, include_py_filesTrue, includes[**/*.py])
datas collect_data_files(tokenizers, include_py_filesTrue, includes[**/*.py])
datas collect_data_files(torch, include_py_filesTrue, includes[**/*.py])block_cipher Nonea Analysis([mainwindow.py],pathex[],binaries[],datasdatas,hiddenimports[],hookspath[],hooksconfig{},runtime_hooks[],excludes[],win_no_prefer_redirectsFalse,win_private_assembliesFalse,cipherblock_cipher,noarchiveFalse,
)
pyz PYZ(a.pure, a.zipped_data, cipherblock_cipher)exe EXE(pyz,a.scripts,a.binaries,a.zipfiles,a.datas,[],namemainwindow,debugFalse,bootloader_ignore_signalsFalse,stripFalse,upxTrue,upx_exclude[],runtime_tmpdirNone,consoleFalse,disable_windowed_tracebackFalse,argv_emulationFalse,target_archNone,codesign_identityNone,entitlements_fileNone,
)
