Azure-Pipelines¶
Azure-Pipelines is a service which allows you to test and deploy your projects. We are only interested in the first aspect, testing, as Azure-Pipelines allows us to run a complete battery of tests every time a new commit is made on the master branch or every time a pull-request is updated. This ensures that we are gradually improving the project and do not introduce bugs or style issues in areas where we already have tests. Also, we can run the test battery on the three major operating system, Linux, MacOS and Windows.
Broadly speaking, there are two categories of tests we are implementing in a research project. The first category is about testing our data to ensure that the source files are the same, intermediate results did not change, etc.. Most of the time, researchers are bounded by confidentiality agreements to keep their data private. In this case you cannot use Azure-Pipelines to test your data and you need to skip this part of the testing battery.
The second category of tests concerns the code which does not normally fall under the former constraint and can be given into the hands of private company.
Installation¶
To enable testing, go to https://azure.microsoft.com/de-de/services/devops/pipelines/
and choose to log in with Github. Then, create a project. After that, choose to create
pipeline based on an existing .yaml
in one of your repositories. Now, link to that
repository and navigate to the configuration file via branch and path. That’s it!
To get an impression of a configuration file, take the following example of the Azure-Pipelines configuration of this template.
pr: none
variables:
CI: true
jobs:
- job:
displayName: Linux
pool:
vmImage: "ubuntu-16.04"
strategy:
matrix:
Python36:
python.version: "3.6"
Python37:
python.version: "3.7"
steps:
- bash: |
sudo apt-get -qq update
wget https://github.com/jgm/pandoc/releases/download/2.7.3/pandoc-2.7.3-1-amd64.deb -O $HOME/pandoc.deb
sudo dpkg -i $HOME/pandoc.deb
displayName: Install Pandoc.
- bash: echo "##vso[task.prependpath]$CONDA/bin"
displayName: Add conda to PATH.
- bash: conda update conda --yes --quiet
displayName: Update conda.
- bash: conda create --yes --quiet --name {{ cookiecutter.project_slug }} -c defaults -c conda-forge python=$PYTHON_VERSION tox
displayName: Create Anaconda environment.
- bash: |
source activate {{ cookiecutter.project_slug }}
tox -e pytest
tox -e sphinx -e linting -p auto
- job:
displayName: Windows
pool:
vmImage: "windows-2019"
strategy:
matrix:
Python36:
python.version: "3.6"
Python37:
python.version: "3.7"
steps:
- powershell: Write-Host "##vso[task.prependpath]$env:CONDA\Scripts"
displayName: Add conda to PATH.
- powershell: conda update conda --yes --quiet
displayName: Update conda.
- powershell: conda create --yes --quiet --name {{ cookiecutter.project_slug }} -c defaults -c conda-forge python=$env:PYTHON_VERSION tox
displayName: Create Anaconda environment.
- script: |
call activate {{ cookiecutter.project_slug }}
tox -e pytest
- job:
displayName: MacOS
pool:
vmImage: "macOS-10.13"
strategy:
matrix:
Python36:
python.version: "3.6"
Python37:
python.version: "3.7"
steps:
- bash: echo "##vso[task.prependpath]$CONDA/bin"
displayName: Add conda to PATH.
# On hosted macOS, the agent user doesn't have ownership of Miniconda's installation
# directory. We need to take ownership if we want to update conda or install packages
# globally.
- bash: sudo chown -R $USER $CONDA
displayName: Take ownership of conda installation
- bash: conda update conda --yes --quiet
displayName: Update conda.
- bash: conda info -a
- bash: conda create --yes --quiet --name {{ cookiecutter.project_slug }} -c defaults -c conda-forge python=$PYTHON_VERSION tox
displayName: Create Anaconda environment.
- bash: conda list
- bash: |
source activate {{ cookiecutter.project_slug }}
tox -e pytest