Python Tips and Tricks

Creating Virtual Environments

#

Standard Python Operations

#

Creation

#

Here are some common code examples 👇

# Create a virtual environment
python -m venv your_env_name

# Create a virtual environment with a specific Python version (if Python is installed in the default location)
python -m venv your_env_name --python=python3.11

# If Python is installed in a custom location
D:\Python\Python311\python.exe -m venv your_env_name

Below are some optional parameters for creating customized virtual environments:

Activation

#

By default, the virtual environment is inactive. In the “your_env_name/Scripts/” directory, there will be a file named “activate,” which can be executed via the command line.

# Activate the virtual environment
your_env_name/Scripts/activate

Poetry

#

Poetry is a Python package management tool. According to its changelog, it has been in development since February 2018, making it not a new tool but one that has gained significant popularity in recent years 💫.

Its advertised highlights include:

• More comprehensive third-party dependency analysis
• Automated virtual environment creation
• More intuitive commands 🤔

Installation

#

Straightforward: pip install poetry for a global installation.

Global Configuration

#

Poetry has some unique mechanisms that may differ from tools like pip. You might need to configure certain settings as follows:

# List all configuration items
poetry config --list

# By default, Poetry creates virtual environments in a dedicated folder rather than the project directory
# Use the following command to create the environment folder in the project directory
poetry config virtualenvs.in-project true

# Poetry can automatically create virtual environments when none exist in the project
# For reassurance, you can run this command
poetry config virtualenvs.create true

# Switch to a domestic mirror
poetry config repositories.tencentyun https://mirrors.tencentyun.com/pypi/simple

# Other settings can generally be left as defaults

Initializing a Project

#

If you’re starting a new project from scratch, you’ll need to: create a project folder, navigate to it in the command line, and simply run:

poetry init

An interactive command-line interface will guide you through creating a new project, generating a pyproject.toml file to record dependencies, and creating Poetry’s signature poetry.lock file to lock package versions. Those familiar with Node will find this familiar 😝.

If you’re using someone else’s project, you’ll typically only find the pyproject.toml file, which is compatible across different package management tools. Some authors may also include the poetry.lock file. In such cases, you can choose to: 1) delete the lock file and use another package manager, or 2) keep the lock file and use Poetry to manage dependencies.

If you need a simple directory structure, you can use:

poetry new my-package

This will automatically create a directory structure suitable for most projects:

my-package
├── pyproject.toml
├── README.md
├── my_package
│   └── __init__.py
└── tests
    └── __init__.py

For more complex needs: new Command.

Creating a Virtual Environment

#

Here are a few scenarios to consider.

1. You’re building your project from scratch:

poetry env use python # Remember how you installed Poetry? This uses the default Python version.

poetry env use python3.7 # Uses Python 3.7 from your system.

poetry env use 3.7 # For the lazy.

poetry env use /full/path/to/python # If the shorthand doesn't work, use this.

2. You’re using someone else’s project:

There are two very similar commands, and the comments below highlight their differences. The official recommendation is the sync command, as it avoids installing packages not tracked by poetry.lock, which might have been accidentally added by the original developer.

However, in practice, install is often more reliable, as sync can sometimes produce strange errors, like removing itself and crashing the command line 🤔.

poetry install # Automatically creates a virtual environment and installs all dependencies listed in pyproject.toml.

# Ensure virtualenvs.create is set to true.
# Automatically creates a virtual environment and installs all dependencies listed in poetry.lock, removing any extra packages from pyproject.toml.
poetry sync

Then, enter the virtual environment with:

poetry shell

Adding Packages

#

Poetry allows developers to distinguish between production and development dependencies.

# Defaults to the production group.
poetry add your-package

# Adds to the development group.
poetry add your-package -D

Updating Packages

#

poetry update # Automatically analyzes dependencies and updates all possible packages.

poetry update your-package1 your-package2 # Updates only the listed packages.

Removing Packages

#

This is one of Poetry’s standout features: pip installs a package and its dependencies but only removes the specified package, leaving its dependencies behind.

Poetry, however, can safely and completely remove a package and its dependencies without affecting other packages.

# Removes a production dependency.
poetry remove your-package

# Removes a development dependency.
poetry remove your-package -D

Displaying Dependencies

#

poetry show --tree # Displays the dependency tree from pyproject.toml.

poetry show your-package --tree # Displays the dependency tree for a specific package.

UV

#

A package management tool written in Rust, with a command structure very similar to poetry.

Installation

#

First, configure environment variables:

cd ~ # Ensure you're in the default directory.
ls -a # Check if the .bashrc file exists.
vim .bashrc # Edit the file with vim.

# Append these three lines to the end of the file.
# Domestic mirror for the installer.
export UV_INSTALLER_GHE_BASE_URL=https://ghproxy.cn/https://github.com
# Domestic mirror for Python installation.
export UV_PYTHON_INSTALL_MIRROR=https://ghproxy.cn/https://github.com/indygreg/python-build-standalone/releases/download
# Domestic mirror for Python packages.
export UV_DEFAULT_INDEX=https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple

• Installation via curl (recommended):

# Replace "/custom/path" with your desired installation location.
curl -LsSf https://astral.sh/uv/install.sh | env UV_INSTALL_DIR="/custom/path" sh

• Installation via pip/pipx (recommended):

pipx install uv # Install uv using pipx.

pip install uv # Install uv using pip.

Usage

#

Usage is very similar to poetry.

Conda

#

I personally don’t use Conda much for environment management, but since many lab servers default to using Conda, I’ll document the basics here for reference.

Usage

#

The following commands are fundamental for checking environment information. They’re typically used when accessing a new server environment or troubleshooting environment-related issues.

########## Checking Environment Information ##########

arch # View the system architecture

conda --version # Get the Conda version

python --version # Get the Python version

conda env list # List all available environments

conda activate xxx # Activate the virtual environment named "xxx"

conda deactivate # Exit the current virtual environment

whereis python # Check the location of Python in the currently activated environment

nvidia-smi # Check CUDA status (not a Conda command but frequently used)

conda config --show channels # View Conda's download sources

conda remove -n xxx # Delete the virtual environment named "xxx"

conda create -n env_name # Create a virtual environment with the specified name, using the default Python version

Adding domestic (Chinese) download sources:

conda config --show channels # View Conda's download sources  

# Add Tsinghua's mirror source  
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2  

Managing Python packages:

# Install a specific package
conda install package_name

# Remove a specific package  
conda remove package_name  

# Update a specific package  
conda update package_name  

# Search for a specific package  
conda search package_name  

# List all installed packages and their versions  
conda list  

# Clean unused packages and cache to save disk space  
conda clean --all  

# Install packages from a requirements.txt file  
conda install -f requirements.txt  

Manual installation of dependency packages for handling extreme offline situations:

# Directly use pip download command to download all related whl files
pip download scikit-image --dest ./scikit_image_files --only-binary :all: --python-version 3.13 --platform manylinux_2_17_x86_64 --implementation cp --abi cp313

# Manually transfer to the server, switch to the target folder, and run (note the whl files)
# --find-links specifies the folder containing the whl files
pip install scikit_image-0.25.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl --no-index --find-links=./

Passwordless Login via VSCode

#

For students without a local GPU, editing and running files directly on a server is very convenient. The VSCode extension remote-ssh can also display a VSCode interface directly on the server. To avoid the hassle of repeatedly entering passwords, you can set up passwordless login as follows 😄.

For a local Windows machine, SSH-related configuration files are usually stored in the C:\Users\<User_name>\.ssh folder. Passwordless login can be achieved by modifying files in this directory.

1. Configure the config file. A sample configuration is as follows:

# Replace <User_name> with your local machine's username  
Host 3090  
    HostName xx.xxx.xx.xx  
    User morethan  
    IdentityFile "C:\Users\<User_name>\.ssh\id_rsa"  

2. Generate the authentication file id_rsa.pub.

3. Locally create an authorized_keys file and copy the contents of id_rsa.pub into it.

4. On the server, create a .ssh folder in the default directory and copy the authorized_keys file from your local machine into it.

Now, when logging into the server via VSCode, you’ll find that passwordless login is enabled.

Packaging Applications

#

We often need to share Python programs we’ve written. However, sharing only the source code can be frustrating for users unfamiliar with coding, as running the source requires setting up a local environment. Hence, packaging tools come into play.

pyinstaller

#

Installation is straightforward, just like any other Python package: pip install pyinstaller. To use it, navigate to the directory containing the target file in the terminal and run the command.

Features: Fast packaging speed; larger resulting executable size.

Common commands:

# Package main.py into a standalone executable; disable the console window when running.
pyinstaller -F -w main.py

# Package main.py into a project folder; enable the console window when running.
pyinstaller -D main.py

Nuitka

#

Converts Python code into an exe executable by first translating Python to C and then compiling the C code.

Features: Very slow packaging speed; requires a C compiler (though installation can be automated, it may not suit users with strict memory constraints); resulting executable is very small (about one-tenth the size of pyinstaller’s output).

Installation command:

pip install -U nuitka

Common usage commands:

# Package main.py into a single exe file with link-time optimization and clean up temporary files afterward.
python -m nuitka --lto=yes --remove-output --onefile main.py

References

#

• Poetry-related:

  • Poetry Beginner’s Guide_Using Poetry-CSDN Blog Very detailed resource.
  • Poetry
  • How to Change Poetry’s Domestic Mirror - Data Science SourceResearch

• UV-related:

  • Installing and Using Python Project and Package Manager UV - Deep Sea Xiao Tao One of the few blogs that includes the domestic mirror address for uv python install.

• Conda-related：

  • Summary of Commonly Used Conda Commands - Zhihu