Building a release¶

Release notes
- Release note subsubsections
PRs and issues in release notes
Preparing the release commit
Isolated anaconda environment
Preparing the distributions
Docs
Test PyPI
Release to PyPI
Release to github
Web page update

Release notes ¶

The release notes should be updated with the date of release and checked for completeness and accuracy. The release series (x.y) is formatted as a section, the actual release (x.y.z) as a subsection, and the possible subsubsections follow here.

Normally release notes should be updated as functionality is added/changed, not only at release time.

Release note subsubsections ¶

If any of these is not used for a particular release, it may be omitted.

New features¶

Brief description of any features that were added in this release. Link to appropriate documentation of the feature (e.g. user documentation such as the script, or API documentation). Link to either the Github pull request that introduced the feature or the enhancement request issue related to it, whichever has the most information.

This includes added functionality in existing scripts, functions, and classes.

Deprecations and removals¶

Any functionality that has been removed or is planned to be removed (deprecated). This includes API deprecations. For deprecated features, link their documentation; for deprecated or removed, link the appropriate replacement.

This also includes any changes in support for older database versions.

Dependency requirements¶

Any changes to dependencies (e.g. new dependencies, new minimum versions.)

Major bugfixes¶

Fixed bugs that are likely to be of interest to a substantial fraction of dbprocessing users. Omit very obscure bugs that are unlikely to be hit; they will be included in the full list of closed issues.

Normally link to the relevant Github issue, although in some cases the pull request that closed the bug may be more appropriate.

Other changes¶

Any other changes that are likely to be of interest to a substantial fraction of users and do not fit the above categories. In particular, any changes in behavior or expectations that are not new features or bug fixes.

Pull requests merged this release¶

Should be generated and pasted into release notes at release time; see below.

Other issues closed this release¶

Also generated at release time.

PRs and issues in release notes ¶

The list of pull requests and issues closed in a release is created from the script git_notes.py in developer/scripts.

This script requires GitPython and PyGithub. On Ubuntu these are packaged as python-git and python-github (or python3-git and python3-github.)

You need a Github token to avoid throttling. No scopes are required.

Run the script with your token and the name of the last release tag. The relevant rST will be sent to stdout; redirect to a file, and copy/paste into the release notes. Do not do this on a shared machine, as the token will be visible in the process table.

python ./git_notes.py -t GITHUB_TOKEN -r release-0.1

Note that the PR including the release should also be in the list; you can open a draft PR to get the PR number, and manually add that to the list. Similarly, the commit against gh-pages that updates the website should be in the list.

Preparing the release commit ¶

The following changes should result in either the final release version, or the appropriate release candidate (i.e. add “rc1” if building rc).

Change version in setup.py, in setup_kwargs near bottom.

Change __version__ around line 209 of dbprocessing/__init__.py.

Commit these changes.

If this is the release version, make a second commit setting all the versions to the next version number and rc0. Then submit the PR. The tagging is done in github and will be done from the last commit before the rc0 commit.

Isolated anaconda environment ¶

You may wish to perform all the steps after this within an isolated Anaconda environment, separate from your machine’s Python setup. In that case, download Miniconda, install it, and use that environment:

bash ./Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda
~/miniconda/bin/conda create -y -n dbp_build python=3
export PYTHONNOUSERSITE=1
export PYTHONPATH=
source ~/miniconda/bin/activate dbp_build
conda install sqlalchemy python-dateutil sphinx numpydoc twine numpy wheel

Note numpy is only required for the reports module (and thus its documentation).

Preparing the distributions ¶

python setup.py sdist --formats=gztar,zip
python setup.py bdist_wheel --universal

Note this will perform a “build” and then rebuild the documentation (in sphinx/build) for the souce distribution. Tarball, zip, and wheel are in the dist directory.

Docs ¶

From the sphinx directory:

make latexpdf
cp build/latex/dbprocessing.pdf dbprocessing-x.y.z-doc.pdf
cd build/html
zip -r ../../dbprocessing-x.y.z-doc.zip *

Test PyPI ¶

Test PyPI allows the upload and test of a complete build.

A release candidate build should be uploaded first. rc0 is used as the version number throughout development; actual release candidates should start with rc1.

Make a .pypirc file

[distutils]
index-servers =
    pypi
    testpypi

[pypi]
username: <username>

[testpypi]
repository: https://test.pypi.org/legacy/
username: <username>

Put all the builds (source dists) into one directory and then do the upload:

twine upload -r testpypi dbprocessing-*.zip dbprocessing-*.whl

PyPI can only take one source distribution (zip or tar.gz), so we use zip.

Test installing with:

pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ dbprocessing

You can use the --pre flag to install the RC version; in that case, probably want to use --no-deps so don’t get RC version of dependencies! (Or can specify the rc version, e.g. dbprocessing==0.1.0rc1).

Release to PyPI ¶

See PyPI upload directions

twine upload dbprocessing-*.zip dbprocessing-*.whl

Do not upload the .tar.gz since can only upload one source package per release.

There’s no longer any capability to edit information on PyPI, it’s straight from the setup.py metadata.

Release to github ¶

See GitHub directions.

On the code tab, click on “n releases” (on the right column, below “about”). Click “Draft a new release.” Make the tag “release-x.y.z” and select the appropriate commit as the target (the one where the version number was updated).

Use just “x.y.z” as the title. The “describe” should link the release notes on the website and also have a brief version of the release notes included.

Click in the “upload binaries” area and upload all the files: source distributions (zip and tar), wheel, documentation PDF (dbprocessing-x.y.z-doc.pdf) and a zip (dbprocessing-x.y.z-doc.zip).

Web page update ¶

Check out the gh-pages branch. Right now the root of the branch is basically the root of the sphinx/build/html output. Copy all the freshly-built docs there, commit, submit PR against the gh-pages branch.

Release: 0.1.0 Doc generation date: Feb 10, 2022

dbprocessing