xemu/python/README.rst

88 lines
3.8 KiB
ReStructuredText
Raw Normal View History

QEMU Python Tooling
===================
This directory houses Python tooling used by the QEMU project to build,
configure, and test QEMU. It is organized by namespace (``qemu``), and
then by package (e.g. ``qemu/machine``, ``qemu/qmp``, etc).
``setup.py`` is used by ``pip`` to install this tooling to the current
environment. ``setup.cfg`` provides the packaging configuration used by
``setup.py``. You will generally invoke it by doing one of the following:
1. ``pip3 install .`` will install these packages to your current
environment. If you are inside a virtual environment, they will
install there. If you are not, it will attempt to install to the
global environment, which is **not recommended**.
2. ``pip3 install --user .`` will install these packages to your user's
local python packages. If you are inside of a virtual environment,
this will fail; you want the first invocation above.
If you append the ``--editable`` or ``-e`` argument to either invocation
above, pip will install in "editable" mode. This installs the package as
a forwarder ("qemu.egg-link") that points to the source tree. In so
doing, the installed package always reflects the latest version in your
source tree.
Installing ".[devel]" instead of "." will additionally pull in required
packages for testing this package. They are not runtime requirements,
and are not needed to simply use these libraries.
Running ``make develop`` will pull in all testing dependencies and
install QEMU in editable mode to the current environment.
(It is a shortcut for ``pip3 install -e .[devel]``.)
See `Installing packages using pip and virtual environments
<https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/>`_
for more information.
Using these packages without installing them
--------------------------------------------
These packages may be used without installing them first, by using one
of two tricks:
1. Set your PYTHONPATH environment variable to include this source
directory, e.g. ``~/src/qemu/python``. See
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
2. Inside a Python script, use ``sys.path`` to forcibly include a search
path prior to importing the ``qemu`` namespace. See
https://docs.python.org/3/library/sys.html#sys.path
A strong downside to both approaches is that they generally interfere
with static analysis tools being able to locate and analyze the code
being imported.
Package installation also normally provides executable console scripts,
so that tools like ``qmp-shell`` are always available via $PATH. To
invoke them without installation, you can invoke e.g.:
``> PYTHONPATH=~/src/qemu/python python3 -m qemu.qmp.qmp_shell``
The mappings between console script name and python module path can be
found in ``setup.cfg``.
Files in this directory
-----------------------
- ``qemu/`` Python 'qemu' namespace package source directory.
python: add avocado-framework and tests Try using avocado to manage our various tests; even though right now they're only invoking shell scripts and not really running any python-native code. Create tests/, and add shell scripts which call out to mypy, flake8, pylint and isort to enforce the standards in this directory. Add avocado-framework to the setup.cfg development dependencies, and add avocado.cfg to store some preferences for how we'd like the test output to look. Finally, add avocado-framework to the Pipfile environment and lock the new dependencies. We are using avocado >= 87.0 here to take advantage of some features that Cleber has helpfully added to make the test output here *very* friendly and easy to read for developers that might chance upon the output in Gitlab CI. [Note: ALL of the dependencies get updated to the most modern versions that exist at the time of this writing. No way around it that I have seen. Not ideal, but so it goes.] Provided you have the right development dependencies (mypy, flake8, isort, pylint, and now avocado-framework) You should be able to run "avocado --config avocado.cfg run tests/" from the python folder to run all of these linters with the correct arguments. (A forthcoming commit adds the much easier 'make check'.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-28-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-05-27 21:17:11 +00:00
- ``tests/`` Python package tests directory.
- ``avocado.cfg`` Configuration for the Avocado test-runner.
Used by ``make check`` et al.
- ``Makefile`` provides some common testing/installation invocations.
Try ``make help`` to see available targets.
- ``MANIFEST.in`` is read by python setuptools, it specifies additional files
that should be included by a source distribution.
- ``PACKAGE.rst`` is used as the README file that is visible on PyPI.org.
python: Add pipenv support pipenv is a tool used for managing virtual environments with pinned, explicit dependencies. It is used for precisely recreating python virtual environments. pipenv uses two files to do this: (1) Pipfile, which is similar in purpose and scope to what setup.cfg lists. It specifies the requisite minimum to get a functional environment for using this package. (2) Pipfile.lock, which is similar in purpose to `pip freeze > requirements.txt`. It specifies a canonical virtual environment used for deployment or testing. This ensures that all users have repeatable results. The primary benefit of using this tool is to ensure *rock solid* repeatable CI results with a known set of packages. Although I endeavor to support as many versions as I can, the fluid nature of the Python toolchain often means tailoring code for fairly specific versions. Note that pipenv is *not* required to install or use this module; this is purely for the sake of repeatable testing by CI or developers. Here, a "blank" pipfile is added with no dependencies, but specifies Python 3.6 for the virtual environment. Pipfile will specify our version minimums, while Pipfile.lock specifies an exact loadout of packages that were known to operate correctly. This latter file provides the real value for easy setup of container images and CI environments. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-15-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-05-27 21:16:58 +00:00
- ``Pipfile`` is used by Pipenv to generate ``Pipfile.lock``.
- ``Pipfile.lock`` is a set of pinned package dependencies that this package
is tested under in our CI suite. It is used by ``make check-pipenv``.
- ``README.rst`` you are here!
- ``VERSION`` contains the PEP-440 compliant version used to describe
this package; it is referenced by ``setup.cfg``.
- ``setup.cfg`` houses setuptools package configuration.
- ``setup.py`` is the setuptools installer used by pip; See above.