Skip to content

add feature: container environment variables #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lundybernard wants to merge 1 commit into
base: master
Choose a base branch
from
Open

add feature: container environment variables #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 19 additions & 18 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ a python [LTI](http://developers.imsglobal.org/) app (Django) and an external se
is a standard that allows external apps to be embedded within an LMS's website by
means of an iframe. This makes it very difficult to do functional tests in
isolation. So I created a docker image for spinning up a dev instance of Canvas
and also this TestCase mixin for the purposes of automating it's execution
during my tests. It's a pretty narrow use case, but the hope is that it can be
and also this TestCase mixin for the purposes of automating it's execution
during my tests. It's a pretty narrow use case, but the hope is that it can be
useful for other types of containers.

### Getting started
Expand All @@ -24,7 +24,7 @@ useful for other types of containers.

### Writing the test case class

To create a functional test that relies on a docker container you'll need to
To create a functional test that relies on a docker container you'll need to
include a few additions to to your TestCase subclass:

1. Insert `PythonDockerTestMixin` at the beginning of your TestCase inheiritance
Expand All @@ -34,15 +34,15 @@ chain. See [here](http://nedbatchelder.com/blog/201210/multiple_inheritance_is_h
image you wish to run. If the specified image is not found in your local docker
instance it will be pulled from the public registry.

1. Optional but recommended, define a `container_ready_callback` class method.
This method will be called from the thread that handles running the container.
1. Optional but recommended, define a `container_ready_callback` class method.
This method will be called from the thread that handles running the container.
Within this method you should do things to confirm that whatever's running in
your container is ready to run whatever your tests are exercising. The method
should simply return if everything is all set, otherwise raise a
`python_docker_test.ContainerNotReady`. The method will be called with a
positional argument of a dict structure containing the result of docker-py's
[`inspect_container`](http://docker-py.readthedocs.org/en/latest/api/#inspect_container), so you can know things like the container's ip and gateway
address.
should simply return if everything is all set, otherwise raise a
`python_docker_test.ContainerNotReady`. The method will be called with a
positional argument of a dict structure containing the result of docker-py's
[`inspect_container`](http://docker-py.readthedocs.org/en/latest/api/#inspect_container), so you can know things like the container's ip and gateway
address.

1. Optionally set the `CONTAINER_READY_TRIES` and `CONTAINER_READY_SLEEP` class
attributes to control how many times your `container_ready_callback` method is
Expand All @@ -57,11 +57,12 @@ called before giving up, and how long the thread will sleep between calls.
from python_docker_test import PythonDockerTestMixin

class MyTests(PythonDockerTestMixin, unittest.TestCase):

CONTAINER_IMAGE = 'foobar/my_image'
CONTAINER_READY_TRIES = 3
CONTAINER_READY_SLEEP = 10

CONTAINER_ENVIRONMENT = {'MYSQL_ALLOW_EMPTY_PASSWORD': True}

@classmethod
def container_ready_callback(cls, container_data):
try:
Expand All @@ -71,8 +72,8 @@ called before giving up, and how long the thread will sleep between calls.
assert resp.status_code == 302
return
except (requests.ConnectionError, AssertionError):
raise ContainerNotReady()
raise ContainerNotReady()

def test_something(self):
# container should be running; test some stuff!
...
Expand All @@ -81,16 +82,16 @@ called before giving up, and how long the thread will sleep between calls.
### App <-> Container networking

If, like me, you need the service(s) in the container to communicate back to the
app under test there is an additional hurdle of making the app accessible from
within the container. The simplest approach that I found is to bind the app to
app under test there is an additional hurdle of making the app accessible from
within the container. The simplest approach that I found is to bind the app to
the container's gateway IP. By default, using docker's "bridge" networking mode,
the gateway address is `172.17.42.1`. So, assuming a Django app, prior to
the gateway address is `172.17.42.1`. So, assuming a Django app, prior to
executing the functional tests you'll need to start an instance of your app like
so:

`python manage.py runserver 172.17.42.1:8000`

You should then be able to access the app at that address from within the
You should then be able to access the app at that address from within the
container.

### Automating the app server
Expand Down
69 changes: 48 additions & 21 deletions python_docker_test/mixin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,66 +1,84 @@
# -*- coding: utf-8 -*-
from __future__ import print_function

import logging
import sys
import threading
from time import sleep
import docker
from docker.errors import APIError
from requests import ConnectionError
from future.utils import raise_

__all__ = ['PythonDockerTestMixin', 'ConfigurationError', 'ContainerNotReady']

log = logging.getLogger(__name__)

DEFAULT_READY_TRIES = 10
DEFAULT_READY_SLEEP = 3


class ConfigurationError(Exception):
pass


class ContainerNotReady(Exception):
pass


class ContainerStartThread(threading.Thread):

def __init__(self, image, ready_callback, ready_tries, ready_sleep):
def __init__(
self, image, ready_callback, ready_tries, ready_sleep,
environment=None
):
self.is_ready = threading.Event()
self.error = None
self.image = image
self.ready_tries = ready_tries
self.ready_sleep = ready_sleep
self.ready_callback = ready_callback

self.environment = environment

super(ContainerStartThread, self).__init__()

def run(self):

log.debug("ContainerStartThread.run() executed")
try:
try:
self.client = docker.Client(version='auto')
self.client.ping()
except ConnectionError, e:
except ConnectionError as e:
self.error = "Can't connect to docker. Is it installed/running?"
raise

# confirm that the image we want to run is present and pull if not
try:
self.client.inspect_image(self.image)
except APIError, e:
except APIError as e:
if '404' in str(e.message):
print >>sys.stderr, "%s image not found; pulling..." % self.image
print("{} image not found; pulling...".format(self.image),
file=sys.stderr)
result = self.client.pull(self.image)
if 'error' in result:
raise ConfigurationError(result['error'])

run_args = {'image': self.image, 'environment': self.environment}

# create and start the container
self.container = self.client.create_container(self.image)
self.container = self.client.create_container(**run_args)
self.client.start(self.container)
self.container_data = self.client.inspect_container(self.container)

if self.ready_callback is not None:
# wait for the container to be "ready"
print >>sys.stderr, "Waiting for container to start..."
print("Waiting for container to start...", file=sys.stderr)
tries = self.ready_tries
while tries > 0:
try:
print >>sys.stderr, "Number of tries left: {}".format(tries)
print("Number of tries left: {}".format(tries),
file=sys.stderr)
self.ready_callback(self.container_data)
break
except ContainerNotReady:
Expand All @@ -69,20 +87,18 @@ def run(self):

self.is_ready.set()

except Exception, e:
except Exception as e:
self.exc_info = sys.exc_info()
if self.error is None:
self.error = e.message
self.error = e
self.is_ready.set()


def terminate(self):
if hasattr(self, 'container'):
self.client.stop(self.container)
self.client.remove_container(self.container)



class PythonDockerTestMixin(object):

@classmethod
Expand All @@ -93,15 +109,28 @@ def setUpClass(cls):
the container in a separate thread to allow for better cleanup if
exceptions occur during test setup.
"""
log.debug("custom setup class executed")

if not hasattr(cls, 'CONTAINER_IMAGE'):
raise ConfigurationError("Test class missing CONTAINER_IMAGE attribute")
raise ConfigurationError(
"Test class missing CONTAINER_IMAGE attribute"
)

ready_tries = getattr(cls, 'CONTAINER_READY_TRIES', DEFAULT_READY_TRIES)
ready_sleep = getattr(cls, 'CONTAINER_READY_SLEEP', DEFAULT_READY_SLEEP)
ready_tries = getattr(
cls, 'CONTAINER_READY_TRIES', DEFAULT_READY_TRIES
)
ready_sleep = getattr(
cls, 'CONTAINER_READY_SLEEP', DEFAULT_READY_SLEEP
)
ready_callback = getattr(cls, 'container_ready_callback')
environment = getattr(cls, 'CONTAINER_ENVIRONMENT', None)

cls.container_start_thread = ContainerStartThread(
cls.CONTAINER_IMAGE, ready_callback, ready_tries, ready_sleep
cls.CONTAINER_IMAGE,
ready_callback,
ready_tries,
ready_sleep,
environment
)
cls.container_start_thread.daemon = True
cls.container_start_thread.start()
Expand All @@ -110,16 +139,15 @@ def setUpClass(cls):
cls.container_start_thread.is_ready.wait()
if cls.container_start_thread.error:
exc_info = cls.container_start_thread.exc_info
# Clean up behind ourselves, since tearDownClass won't get called in
# case of errors.
# Clean up behind ourselves,
# since tearDownClass won't get called in case of errors.
cls._tearDownClassInternal()
raise exc_info[1], None, exc_info[2]
raise raise_(exc_info[1], None, exc_info[2])

cls.container_data = cls.container_start_thread.container_data

super(PythonDockerTestMixin, cls).setUpClass()


@classmethod
def _tearDownClassInternal(cls):
if hasattr(cls, 'container_start_thread'):
Expand All @@ -135,4 +163,3 @@ def tearDownClass(cls):
def setUp(self):
self.container_ip = self.container_data['NetworkSettings']['IPAddress']
self.docker_gateway_ip = self.container_data['NetworkSettings']['Gateway']