參考:#
在上一次中,我們提到了利用 tool.uv.index 為 Pytorch 配置單獨針對 cpu 或 cuda 版本的 torch 和 torchaudio.。
其中有個老哥提問說能不能配置 uv 的 Python 鏡像源。我正好也有這個需求,因為有時候我發現 uv 吃不到我的代理,我用 pip 直接下載的速度是 6MB/s, 但是,我用 uv 下載一個 20MB 的 Numpy 要 7 分鐘。
當時我發現,我們 torch 指定的是一個index-url
,而實際上,我們在 pip 換源的時候,換的也是index-url
。
諸如:
#清華源
https://pypi.tuna.tsinghua.edu.cn/simple
# 阿里源
https://mirrors.aliyun.com/pypi/simple/
# 騰訊源
http://mirrors.cloud.tencent.com/pypi/simple
# 豆瓣源
http://pypi.douban.com/simple/
這些實際上也都是 index-url。於是乎我們參考上一次配置 torch 的 index-url:
dependencies = [
"funasr==1.2.4",
"pyaml==25.1.0",
"torch==2.1.0",
"torchaudio==2.1.0",
]
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[tool.uv.sources]
torch = [
{ index = "pytorch-cpu" },
]
torchaudio = [
{ index = "pytorch-cpu" },
]
可以很容易得出,我們可以這樣配置鏡像源:
dependencies = [
"numpy==1.26.4",
"matplotlib==3.10.0"
]
[[tool.uv.index]]
name = "tsinghua"
url = "https://pypi.tuna.tsinghua.edu.cn/simple"
explicit = true
[[tool.uv.index]]
name = "aliyun"
url = "https://mirrors.aliyun.com/pypi/simple/"
explicit = true
[tool.uv.sources]
numpy = [
{ index = "tsinghua"},
]
matplotlib = [
{ index = "aliyun"}
]
意思是我們用阿里源下載 matplotlib , 用清華源下載 numpy。
然後我們再寫一個簡單的腳本測試一下。
但是這麼做,我發現,uv.lock
中,我們指定的包確實都轉向了tsinghua
和aliyun
,但是所有的依賴包並不會被解析為使用鏡像源,依然採用pypi.org
。
[[package]]
name = "matplotlib"
version = "3.10.0"
source = { registry = "https://mirrors.aliyun.com/pypi/simple/" }
dependencies = [
{ name = "contourpy" },
{ name = "cycler" },
{ name = "fonttools" },
{ name = "kiwisolver" },
{ name = "numpy" },
{ name = "packaging" },
{ name = "pillow" },
{ name = "pyparsing" },
{ name = "python-dateutil" },
]
sdist = { url = "https://mirrors.aliyun.com/pypi/packages/68/dd/fa2e1a45fce2d09f4aea3cee169760e672c8262325aa5796c49d543dc7e6/matplotlib-3.10.0.tar.gz", hash = "sha256:b886d02a581b96704c9d1ffe55709e49b4d2d52709ccebc4be42db856e511278" }
wheels = [
{ url = "https://mirrors.aliyun.com/pypi/packages/09/ec/3cdff7b5239adaaacefcc4f77c316dfbbdf853c4ed2beec467e0fec31b9f/matplotlib-3.10.0-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:2c5829a5a1dd5a71f0e31e6e8bb449bc0ee9dbfb05ad28fc0c6b55101b3a4be6" },
...
[[package]]
name = "numpy"
version = "1.26.4"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/65/6e/09db70a523a96d25e115e71cc56a6f9031e7b8cd166c1ac8438307c14058/numpy-1.26.4.tar.gz", hash = "sha256:2a02aba9ed12e4ac4eb3ea9421c420301a0c6460d9830d74a9df87efa4912010", size = 15786129 }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a7/94/ace0fdea5241a27d13543ee117cbc65868e82213fb31a8eb7fe9ff23f313/numpy-1.26.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:9ff0f4f29c51e2803569d7a51c2304de5554655a60c5d776e35b4a41413830d0", size = 20631468 },
...
[[package]]
name = "packaging"
version = "24.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d0/63/68dbb6eb2de9cb10ee4c9c14a0148804425e13c4fb20d61cce69f53106da/packaging-24.2.tar.gz", hash = "sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f", size = 163950 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl", hash = "sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759", size = 65451 },
]
這是因為什麼?我發現,這是因為我英語不好。
explicit = true
是排除的意思。
An index can be marked as explicit = true to prevent packages from being installed from that index unless explicitly pinned to it. For example, to ensure that torch is installed from the pytorch index, but all other packages are installed from PyPI, add the following to your pyproject.toml:
可以將一個索引標記為 explicit = true ,以防止軟件包從該索引安裝,除非明確地釘在該索引上。例如,要確保 torch 從 pytorch 索引安裝,而所有其他軟件包都從 PyPI 安裝,可在 pyproject.toml :
所以,在我們只希望偶爾指定一下,那麼我們用explicit = true
參數。那麼在正常的時候,它還是用我們的pypi
。
以及,它還提供了一個defualt
參數。
By default, uv includes the Python Package Index (PyPI) as the "default" index, i.e., the index used when a package is not found on any other index. To exclude PyPI from the list of indexes, set default = true on another index entry (or use the --default-index command-line option):
默認情況下,uv 將 Python 軟件包索引 (PyPI) 列為 "默認 "索引,即在其他索引中找不到軟件包時使用的索引。要從索引列表中排除 PyPI,可在其他索引條目中設置 default = true (或使用 --default-index 命令行選項):
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cpu"
default = true
defualt 參數會把pypi
源直接取代掉並且排除它。不再使用pypi
源,即使在你設定的鏡像源中找不到包,也不會搜索pypi
。
如果只設定了defualt
的話實際上這存在一些問題。因為有時候一些包沒有被及時維護更新到,或者鏡像站偶爾抽風了就會導致你無法正常工作。
[tool.uv.index]
** 還有不帶參數的選項。** 即這麼寫:
[[tool.uv.index]]
# Optional name for the index.
name = "pytorch"
# Required URL for the index.
url = "https://download.pytorch.org/whl/cpu"
Indexes are prioritized in the order in which they’re defined, such that the first index listed in the configuration file is the first index consulted when resolving dependencies, with indexes provided via the command line taking precedence over those in the configuration file.
索引按其定義順序排列優先級,因此在解析依賴關係時,配置文件中列出的第一個索引是第一個被查詢的索引,通過命令行提供的索引優先於配置文件中的索引。
By default, uv includes the Python Package Index (PyPI) as the "default" index, i.e., the index used when a package is not found on any other index. To exclude PyPI from the list of indexes, set default = true on another index entry (or use the --default-index command-line option):
默認情況下,uv 將 Python 軟件包索引 (PyPI) 列為 "默認 "索引,即在其他索引中找不到軟件包時使用的索引。要從索引列表中排除 PyPI,可在其他索引條目中設置 default = true (或使用 --default-index 命令行選項):
就是不帶default
和explicit
參數。這樣就像什麼,就像 conda channels
:
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
它會根據你定義的順序作為它的優先級順序,最上面的最優先,然後都找不到的話,就會使用default
,默認是pypi
,可以自己設定。你也可以用explicit
來為特定的包指定 index, 但需要指定 source,像這樣:
[tool.uv.sources]
torch = [
{ index = "pytorch-cpu" },
]
torchaudio = [
{ index = "pytorch-cpu" },
]
相當完美和簡潔的設計。
於是乎最終我們定下來的配置是這樣的:
[[tool.uv.index]]
# Optional name for the index. 這個可以不寫
name = "tsinghua"
url = "https://pypi.tuna.tsinghua.edu.cn/simple"
default = true
[[tool.uv.index]]
name = "aliyun"
url = "https://mirrors.aliyun.com/pypi/simple/"
我們用清華源替代 pypi 並且排除 pypi, 然後給了 aliyun 作為備胎。當我們的清華源(它定義在 aliyun 上面,所以優先級更高)找不到時,就會嘗試在阿里源進行搜索。
可以看到https://github.com/MrXnneHang/test-uv-index/blob/master/uv.lock
這次 uv.lock 中,清華源是我們的主角。=-=