Software engineer Jordan Mora introduces us to Python Poetry and its implementation for convenient and relaxed dependency management.
Have you ever joined a new Python project or reviewed an old one and, while setting up your environment, you started getting those “ can't install dependency So you've checked the requirements file and discovered that the offending dependency doesn't have a pinned version, or worse, doesn't even exist. Well, you're not alone; It happened to me once or twice.
When this happens, the best solution is always:
- Check the version with another coworker
- Add it to the requirements file
- Run the installation process again
- I hope this doesn't happen with the next dependency
- Repeat
You can see it's very reactive because it doesn't stop it from happening again and sooner or later it will. That thought remained in my head. Is there a solution to this common problem? Something like npm for front-end projects?
The old way
The usual way to install dependencies in a Python project is using pip . With pip, you add a dependency with pip install you need to run pip install -r requirements.txt to install them.
The problem with this approach is that when dependencies also have subdependencies, you may not always get the same version because they are not specified in the requirements.txt file. Another problem with this process is that you may forget to add the dependency to the requirements file. This is where Python Poetry comes in .
What Makes Python Poetry Attractive
Poetry uses a TOML file instead of a text file to track dependencies and a lock file to track dependencies and subdependencies. Each time project dependencies are installed, you will always get the same lock file versions. Poetry goes further and also saves the hash of a package, as some maintainers can update the package without changing the version.
Poetry comes with a convenient CLI for installing dependencies, so pip is no longer necessary. When installing dependencies, they are automatically added to the TOML file, avoiding the terrible task of remembering the packages added to the project.
Python Poetry reinforces the use of virtual environments. If you're not already using one, it creates a new virtual environment to isolate dependencies between different projects. If you are using Docker, you can disable this feature, as there is no point in using virtual environments inside containers.
Unlike the old way, Poetry gives you the ability to update all your packages, and depending on the dependency specification you are using, it can be a patch or a minor update (a thorough search is recommended for larger updates).
The dependency specification refers to the version restriction that you apply to your dependencies. You can use caret, tilde, or wildcard restrictions, or even specify a range of acceptable versions. You can also use git, path and URL, and source dependencies.
Another cool feature of Poetry is that when you remove a dependency you no longer need, it also removes its subdependencies, which is not the case with pip. This will avoid orphaned dependencies in your environment.
Finally, Poetry gives you the ability to separate core dependencies, those you need to make your project work, from development dependencies, which are primarily used for debugging purposes. This results in a production environment free of unnecessary dependencies.
I should note that Poetry is also capable of packaging and publishing libraries, but I'll just focus on dependency management.
Composing your first poem
There's no better way to understand something than to see it for yourself, so we'll follow a short tutorial on how to migrate an existing project to Python Poetry. I assume you have already installed Python version 3.7+ and git. If you're feeling adventurous, you can use a project you own or are working on, but I'll provide a mockup of the Django project for this tutorial. The version of Python Poetry at the time of writing this article is 1.3.1.
Firstly, you can install Python Poetry by running the command:
curl -sSL python3-
You can check if Poetry was installed successfully by running poetry –version. You can also run Poetry List to display the list of available commands.
Next, go to this demo GitHub repository and clone it to your local machine; it's just an empty Django project with a requirements.txt file.
Now that we have the scenario ready, let's proceed with the migration.
Initializing Poetry
The first step is to initialize Python Poetry in your project, so go ahead and run initial poetry within the repository. An interactive prompt will appear to configure the TOML file. You can use the default suggestions, but make sure your Python version matches the Supported Python Version field. When you get to “ Would you like to define your main dependencies interactively? ” section, just send “ no. ” It will then ask if you want to define your development dependencies interactively; again, write “ no. ”Finally, you will be asked to confirm your settings; send “ yes. ”
Migrating your dependencies
Since you already have a requirements.txt file, the easiest way to migrate your dependencies is by running the command:
cat requirements.txt xargs poetry add
Poetry will add each dependency to the TOML file and attempt to resolve any conflicts, then write to the lock file. If you are not using a virtual environment, the dependencies will be installed in a newly created virtual environment.
Now that Poetry has all its dependencies in its list, you can check it by running Poetry Show.
Since the requirements.txt file is no longer needed, you can get rid of it.
Cleaning up subdependencies
Since you probably have subdependencies listed as main dependencies, we can get rid of them. When you want to update high-level dependencies, there will be no restrictions.
Run the command:
poetry show --tree
And you will see which are subdependencies.
Making dependencies updatable
The final step is to make your dependencies updatable. Personally, I only use patch updates as they are usually fixes and security patches. I make minor and major updates only after reading the changelogs and making sure it's something we want to add to the project.
You can use the dependency specification (mentioned earlier in the article) that suits you best. You can open the TOML file again and edit the version of each dependency by adding the version constraint (tilde, caret, wildcard, etc.) to the version number. In my case, I used tilde constraints.
Now, Django and Djangorestframework dependencies can be patched. The next time an update is released for any of these dependencies, you can update them with the command:
poetry update
This will attempt to update all dependencies. You can also specify the dependencies you want to update. If updates are found, Poetry will resolve the dependency conflicts and perform the update. After that it will update the lock file.
In this case, if you run the update and, assuming you made the same changes as I did, you will see Djangorestframework will be updated from version 3.10.0 to 3.10.3, which is the latest patch at the time of writing this article.
You can check out the final product here in the migrated branch of the repository .
Next steps
From now on, you can use Poetry to add and remove dependencies to and from your project, just using the CLI. If you upgrade outside of your constraint, you will need to run the add command with the new version.
For project beginners, getting exactly the same package versions as the last person who updated the lock file is as easy as running poetry install.
Conclusion
Similar to npm, Python Poetry is a powerful dependency management tool that solves many of the problems present in using the requirements.txt file together with pip. It also brings more functionality and the CLI is very easy to use. It can be applied to new and existing Python projects, making it easy to add, remove, and update dependencies as the project evolves.
I hope this tool can be used in many projects going forward. It's proven to be one of the best tools I've ever used to help me and my team keep a project up to date without it becoming a complicated task.
If you're interested in learning more about Python Poetry, check out the documentation by following this link .
If you liked this article, check out one of our other Python articles.
- 8 Best Python GUI Frameworks
- 8 Best Python Libraries for Data Science
- Best Python Libraries for Modern Developers
- 8 Best Sentiment Analysis Libraries in Python
- 4 best web scraping libraries in Python
Source: BairesDev
