pyenv to uv

2025-01-02 . 1890 words . 10 min

Back story: When I read "How fast is your shell?" ¹ by Thorsten Ball, almost an year ago, I immediately jumped my seat to check my shell latency. I use zsh as daily driver, not because of choice, but rather because it comes as default shell on MacOS. When I started my career, choice of shell was an unknown unknown for me. I didn't know that since macOS Catalina, Apple switched from bash to zsh to avoid licensing obligations of GPLv3 for newer versions of bash. I was also not aware of POSIX compliance of shells and maintaining portability for scripts to run anywhere. Shell scripts was as means to an end and it worked. Over the years, I'm more inclined to terminal based interfaces and reading the article struck a chord somewhere deep and I was compelled to fix my shell startup time.

I haven't version controlled my dotfiles on current system and hence, I don't have the benchmarks documented but from what I remember when I started with the optimization, my shell would take ~700ms to start; which was a horrible number. My goal was to get below ~100ms and I used below command to check after stripping out parts of my ~/.zshrc.

for i in $(seq 1 10); do time $SHELL -i -c exit; done
# ~700ms before optimizations
# now, ~330ms
#$SHELL -i -c exit  0.19s user 0.12s system 81% cpu 0.374 total
#$SHELL -i -c exit  0.18s user 0.10s system 88% cpu 0.323 total
#$SHELL -i -c exit  0.18s user 0.10s system 87% cpu 0.325 total
#$SHELL -i -c exit  0.18s user 0.10s system 86% cpu 0.329 total
#$SHELL -i -c exit  0.18s user 0.10s system 86% cpu 0.330 total
#$SHELL -i -c exit  0.18s user 0.10s system 85% cpu 0.335 total
#$SHELL -i -c exit  0.18s user 0.10s system 87% cpu 0.325 total
#$SHELL -i -c exit  0.18s user 0.10s system 87% cpu 0.325 total
#$SHELL -i -c exit  0.18s user 0.10s system 87% cpu 0.323 total
#$SHELL -i -c exit  0.18s user 0.10s system 86% cpu 0.330 total

The reason I could not go down below 300ms was because of pyenv and comfort it provided for managing python versions and virtual environments for most of my projects. Without pyenv initialization in my ~/.zshrc, I was able to get startup time down to ~60s, which is what I wanted.

for i in $(seq 1 10); do time $SHELL -i -c exit; done
#$SHELL -i -c exit  0.02s user 0.03s system 52% cpu 0.067 total
#$SHELL -i -c exit  0.01s user 0.02s system 52% cpu 0.057 total
#$SHELL -i -c exit  0.01s user 0.02s system 52% cpu 0.057 total
#$SHELL -i -c exit  0.01s user 0.02s system 49% cpu 0.059 total
#$SHELL -i -c exit  0.01s user 0.02s system 53% cpu 0.057 total
#$SHELL -i -c exit  0.01s user 0.02s system 52% cpu 0.056 total
#$SHELL -i -c exit  0.01s user 0.02s system 53% cpu 0.055 total
#$SHELL -i -c exit  0.01s user 0.02s system 52% cpu 0.056 total
#$SHELL -i -c exit  0.01s user 0.02s system 51% cpu 0.058 total
#$SHELL -i -c exit  0.01s user 0.02s system 50% cpu 0.058 total

The cost to benefit ratio for pyenv was low for keeping pyenv because I used it for all Python projects and the effort I would need to invest to finding alternatives and making it work for same level of convinience had huge inertia. In retrospect, I didn't spend too much time into optimizing ~/.zshrc to get pyenv to work while deferring this as part of the shell startup.

Here's how to defer instantializing pyenv while starting shell up. I learned about unfunction while writing this post.

pyenv_init() {
  export PYENV_ROOT="$HOME/.pyenv"
  [[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
  eval "$(pyenv init -)"

  unfunction "$0"
}

python() {
  pyenv_init
  command python "$@"
}

pyenv() {
  pyenv_init
  command pyenv "$@"
}

Nevertheless, enter uv.

uv is Astral's project, which solves Python package and project management and provides alternatives to a lot of tools - pip, pipdeptree, pip-tools, virtualenv, standalone script dependencies, and python installations as well. It checks more boxes than I need from my current ecosystem but there are some pitfalls I get in return. So, instead of completely removing pyenv I'll run a beta replacement for it for Jan 2025 and switch completely post that.

I have been using uv in personal projects since the 0.4.0 release hype, but before I start to use it seriously I did an update to uv itself. There is a convinient API available.

uv self update
#info: Checking for updates...
#success: Upgraded uv from v0.5.10 to v0.5.13! https://github.com/astral-sh/uv/releases/tag/0.5.13

After that sorted, first step was to list all the Python versions and pyenv-virtualenv I use from pyenv.

pyenv versions
#* system (set by /Users/HSingh/.pyenv/version)
#  3.7.13
#  3.7.16
#  3.7.16/envs/[REDACTED]
#  3.9.16
#  3.10.6
#  3.11.6
#  3.11.6/envs/[REDACTED]
#  3.11.6/envs/[REDACTED]
#  3.11.7
#  3.11.7/envs/[REDACTED]
#  3.11.7/envs/[REDACTED]
#  3.11.8
#  3.11.8/envs/[REDACTED]
#  3.11.9
#  3.11.9/envs/[REDACTED]
#  3.11.9/envs/[REDACTED]
#  3.12.0
#  3.12.2
#  3.12.2/envs/[REDACTED]
#  3.12.2/envs/[REDACTED]
#  3.12.2/envs/[REDACTED]
#  3.12.2/envs/[REDACTED]
#  3.12.2/envs/[REDACTED]

In retrospect, I work on a lot of different python versions and have venv for different projects for same Python version. In 2025, I'm going to move away from Python 3.7 for most of my projects (I hope).

To do the same with uv, I did

uv python install 3.8 3.9 3.10 3.11 3.12 3.13
#Installed 6 versions in 3.22s
# + cpython-3.8.20-macos-aarch64-none
# + cpython-3.9.21-macos-aarch64-none
# + cpython-3.10.16-macos-aarch64-none
# + cpython-3.11.11-macos-aarch64-none
# + cpython-3.12.8-macos-aarch64-none
# + cpython-3.13.1-macos-aarch64-none

and then added aliases for Python version

alias python3.8='uv run --python=3.8 python3'
alias python3.9='uv run --python=3.9 python3'
alias python3.10='uv run --python=3.10 python3'
alias python3.11='uv run --python=3.11 python3'
alias python3.12='uv run --python=3.12 python3'
alias python3.13='uv run --python=3.13 python3'
alias python3=python3.12

With Python installation completed above, I have 6+3 Python installations. 6 of them from the above step, 2 of them from Homebrew namely 3.12.8 and 3.13.1 and 1 (default) on MacOS installation i.e. 3.9.6.

uv python list --only-installed
#cpython-3.13.1-macos-aarch64-none     /opt/homebrew/opt/[email protected]/bin/python3.13 -> ../Frameworks/Python.framework/Versions/3.13/bin/python3.13
#cpython-3.13.1-macos-aarch64-none     /Users/HSingh/.local/share/uv/python/cpython-3.13.1-macos-aarch64-none/bin/python3.13
#cpython-3.12.8-macos-aarch64-none     /opt/homebrew/opt/[email protected]/bin/python3.12 -> ../Frameworks/Python.framework/Versions/3.12/bin/python3.12
#cpython-3.12.8-macos-aarch64-none     /Users/HSingh/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/bin/python3.12
#cpython-3.11.11-macos-aarch64-none    /Users/HSingh/.local/share/uv/python/cpython-3.11.11-macos-aarch64-none/bin/python3.11
#cpython-3.10.16-macos-aarch64-none    /Users/HSingh/.local/share/uv/python/cpython-3.10.16-macos-aarch64-none/bin/python3.10
#cpython-3.9.21-macos-aarch64-none     /Users/HSingh/.local/share/uv/python/cpython-3.9.21-macos-aarch64-none/bin/python3.9
#cpython-3.9.6-macos-aarch64-none      /Library/Developer/CommandLineTools/usr/bin/python3 -> ../../Library/Frameworks/Python3.framework/Versions/3.9/bin/python3
#cpython-3.8.20-macos-aarch64-none     /Users/HSingh/.local/share/uv/python/cpython-3.8.20-macos-aarch64-none/bin/python3.8

I like to keep the Python environment clean and install any tool for project in a virtual environment, hence I didn't need to use the uv tool install. In future I can do global package installation by doing uv tool install pre-commit. Skipping this for now, as we are in evaluation mode and I can live with global pre-commit from Homebrew. Mentioned this point to make a note to clean such tools later.

To check tools, I listed all packages I've installed with pipx as well and to my surprise there was only 1 i.e. harlequin.

pipx list
#venvs are in /Users/[REDACTED]/.local/pipx/venvs
#apps are exposed on your $PATH at /Users/[REDACTED]/.local/bin
#manual pages are exposed at /Users/[REDACTED]/.local/share/man
#   package harlequin 1.20.0, installed using Python 3.12.3
#    - harlequin

pipx uninstall harlequin
#uninstalled harlequin! ✨ 🌟 ✨

pipx list
#nothing has been installed with pipx 😴

Okay, now lets talk about project management and for this I chose a work project where I created a new virtualenv and ran a build step

cd [PROJECT_NAME]

rm -rf .python-version

uv python install 3.12.6
#Installed Python 3.12.6 in 2.61s
# + cpython-3.12.6-macos-aarch64-none

uv venv --python 3.12.6
#Using CPython 3.12.6
#Creating virtual environment at: .venv
#Activate with: source .venv/bin/activate

source .venv/bin/activate

python --version
#Python 3.12.6

uv pip install '.[dev]'
#Using Python 3.12.6 environment at: /Users/HSingh/code/github.com/majidalfuttaim/de-airflow/.venv
#Resolved 37 packages in 41.46s
#[REDACTED]
#Prepared 37 packages in 10.49s
#Installed 37 packages in 177ms

#Run a functionality to find it works as expected like
#pytest
#OR a custom script that I had
#python build.py

There might be better ways of doing above but, this works for me, so YMMV.

I'm going to delete .python-version files from projects (eventual consistency) and when it is time to let go of pyenv completely, I can do below

• remove pyenv init from ~/.zshrc
• remove custom function to recreate pyenv-virtualenv from ~/.zshrc
• rm -rf $(pyenv root)
• brew uninstall pyenv pyenv-virtualenv

Ok, now talking about confusions.

First thing that I need to get out of my muscle memory is pip install TOOL. I keep doing pip list and other pip-tools command often which is engrained to my brain. I need to learn to unlearn and then get used to new way of usage. Some temporary hacks is to create aliases but I can think many places this will become an issue.

Secondly, when using uv python we are basically using the awesome python-build-standalone project. Astral took stewardship of this project over from Gregory Szorc and they are already contributing steadily to it. There are some documented behavior quirks ² of PBS and none of them are affecting me right now. We will need to keep watch for this though.

Thirdly, I'm not comfortable using unstable APIs at work. In this case, since I'm changing only my development workflow, I'm less worried about it. At the time of writing this article, latest uv version is 0.5.13 and I couldn't find a roadmap or a low-fidelity-bullet-point-path-to-1.0 in versioning policy ³ or otherwise. It is a thankless job to answer questions on an open-source project but this is an important question for adoption of uv; hope Astral team finds time to document a low fidelity roadmap for people like me to be comfortable using and pitching uv.

EOF