Testing and Deploying Python Projects with Travis CI

Introduction

When working on projects, especially those that others or yourself may depend upon, it is important to test to make sure that everything is working as expected. Furthermore, being able to deploy your code/packages to a central repository with a package manager makes distribution easier. Although this can all be done manually, it can also be automated. Both testing and deployment can be automated using Travis CI which makes the entire process as easy as pushing your most recent changes to GitHub. In this writeup, I will go over how to create a Python project with unit tests and deploy to PyPI. A sample project can be found at this link

Prerequisites

Install virtualenv

sudo pip install virtualenv

Create The Project

For the test project, I will create a module that performs adding, subtracting, increment and decrement operations.

Define Folder Structure

We start out by creating a directory for our project.

mkdir travistest

Inside that directory, we want to have have a directory for our module as well as for our tests. Therefore, we need to create a directory for both of them.

mkdir travistest
mkdir test

Finally, we want to initialize the virtual environment for our project. To do so, we can use the virtualenv package. For the project python 3.5 is the version that will be used.

virtualenv -p python3.5 ENV

After installation, a folder with the name ENV should appear in the root directory of the project. The final directory structure should look like so:

travistest
|_ENV
|_travistest
|_test

Install Modules

For this project, I'll be using pytest for unit testing. Before installing anything however, I'll need to activate the virtual environment.

source ENV/bin/activate

Once our virtual environment is activated, we can install pytest.

pip install pytest

After installation, we can persist installed packages inside a requirements.txt file with the freeze command.

pip freeze > requirements.txt

Create The Module

Inside the travistest module directory, the easiest way to create a module is to include an __init__.py file inside the directory. It's okay if it's empty.

Therefore, we can start by creating the __init__ file in that directory.

touch __init__.py

Once that's created, we can begin writing the main functionality of our module. Inside a file called Operations.py, we can put the following code in.

class Operations:
    def __init__(self):
        pass
    
    def add(self,x,y):
        return x + y

    def subtract(self,x,y):
        return x - y

    def increment(self,x):
        return x + 1

    def decrement(self,x):
    	return x - 1

Unit Test

Once we have our code, we need to write tests for it. Navigating to the test directory, we can add the following code to the test_operations.py file.

from pytest import fixture

@fixture
def op():
    from travistest.Operations import Operations
    return Operations()

def test_add(op):
    assert op.add(1,2) == 3

def test_subtract(op):
    assert op.subtract(2,1) == 1

def test_increment(op):
    assert op.increment(1) == 2

def test_decrement(op):
assert op.decrement(2) == 1

To make sure everything is working correctly, from the project's root directory, we can run the pytest command. If all goes well, an output similar to the one below should appear.

============================= test session starts ==============================
platform linux -- Python 3.5.2, pytest-3.4.0, py-1.5.2, pluggy-0.6.0

collected 4 items                                                              

test/test_operations.py ....                                             [100%]

=========================== 4 passed in 0.04 seconds ===========================

Prepare For Deployment

To prepare for deployment and uploading to PyPI, we need to add a setup.py file to the root directory of our project. The contents of this file for our purposes are mostly metadata that will populate information in PyPI.

from distutils.core import setup

setup(
    name='travistest',
    packages=['travistest'],
    version='0.0.7',
    description='Test project to get acquainted with TravisCI',
    url='https://github.com/lqdev/TravisTest',    
)

Setup Travis

Enable Repository

Assuming that you have a GitHub login and a repository has been created for your project:

  1. Log into travis-ci.org with your GitHub credentials.
  2. Once all of your repositores are synced, toggle the switch next to the repository containing your project.

Configure .travis.yml

Once the project has been enabled, we need fo configure Travis. This is all done using the .travis.yml file.

In this file we'll tell Travis that the language of our project is Python version 3.5 and that we'll be using a virtual environment. Additionally we'll require sudo priviliges and target the Ubuntu Xenial 16.04 distribution. All of these configurations can be done as follows.

sudo: required
dist: xenial
language: python
virtualenv:
  system_site_packages: true
python:
- '3.5'

Once that is set up, we can tell it to install all of the dependencies stored in our requirements.txt file.

install:
- pip install -r requirements.txt

After this, we need to tell Travis to run our tests just like we would on our local machine.

script: pytest

Once our tests have run, we need to make sure we are back in the root directory of our project for deployment.

after_script: cd ~

We're done with the automated testing script portion of our project. Now we need to setup deployment options. This section will mainly contain the credentials of your PyPI account.

deploy:
  provider: pypi
  user: "YOURUSERNAME"

After setting the provider and user, we need to set the password. Because this will be a public repository DO NOT enter your password on this file. Instead, we can set an encrypted version that only Travis can decrypt. To do so, while in the root directory of our project, we can enter the following command into the terminal.

travis encrypt --add deployment.password

Type your password into the terminal and press Ctrl + D.

If you take a look at your .travis.yml file you should see something like the following in your deploy options

deploy:
  provider: pypi
  user: "YOURUSERNAME"
  password:
    secure: "YOURPASSWORD"

The final .travis.yml file should like like so

sudo: required
dist: xenial
language: python
virtualenv:
  system_site_packages: true
python:
- '3.5'
install:
- pip install -r requirements.txt
script: pytest
after_script: cd ~
deploy:
  provider: pypi
  user: "YOURUSERNAME"
  password:
    secure: "YOURPASSWORD"

Deploy

Now that we have everything set up, deployment should be relatively easy. A build is triggered when changes are pushed to the repository on GitHub. Therefore, pushing your local changes to the remote GitHub repository should initialize the build. To track progress, visit the project's page on Travis CI.

NOTE: WHEN PUSHING NEW CHANGES, INCREMENT VERSION NUMBER IN THE SETUP.PY FILE SO THAT EXISTING FILE ERRORS DO NOT CRASH BUILDS

Conclusion

In this writeup, we created a Python package that performs basic operations and set up automated testing and deployment with Travis CI. Configurations can be further customized and refined to adapt more complex build processes.

Send me a message or webmention