Skip to content

Home

Jump to bottom Edit New page
benosteen edited this page Jun 20, 2011 · 17 revisions

Dataflow - developer documentation

This is to provide guides, links, information, tips and other documentation on the development of the Dataflow project so that the development can be coherent and open, with everyone able to find out how people are running their test suites, managing their branching strategy, how they configured the continuous integration (CI) and continuous deployments (CD) and so on.

Branching strategy with Git

[Ben:] I very strongly recommend adopting the advice and strategy laid out in the following article: http://nvie.com/posts/a-successful-git-branching-model/ - it is clear, understandable, and battle-tested with multiple developers working on the same repositories.

Please read the documentation that accompanies this to understand how to merge, in which branches to develop, bugfix and to add new features and so on but the key aspects of the strategy can be combined with some simple project-specific house-keeping (issues, etc):

  • For all development, the code and its testcode should be in the same branch. Seems simple, but can be forgotten.
  • master should be treated with reverence. The article put it best: 'We consider master to be the main branch where the source code ... always reflects a production-ready state.' No changes are made directly to this branch. I [Ben] strongly recommend that at least two sets of eyes see any code that is merged into the master or release branches, and preferably, a set of developer eyes from a different area of the project.
  • develop is where much of the rolling development will occur as code should be branched from and merged back into this, before the final testing and review to make sure it is solid enough to be pushed into the master branch:
    • Any new features (hopefully rare in this case) should be developed on in their own named branch, eg sworddeposit
    • Any major blocking bugs that must be fixed can be done in a named branch from master, with the branch name referenced in the bug ticket, and then, once the tests pass, merged back into master (again, recommend code-review before the merge).
    • Refactorings, reviews and other rewrites should take place in their own named branches as well, referenced by an enhancement ticket in the issue tracker preferably.

Advice on how multiple developer efforts can be merged within a given project can be read from the linked article.

Git Branching Strategy

Continuous Integration - Jenkins

By using the age-old tactic of 'whichever is the best to understand and use in the fastest time', Jenkins (a fork of Hudson) won out over Build-bot due to its clear configuration and maintenance options. The wealth of plugins available was also a strong factor in its choice.

Running tests with python

As we are all aiming to use a continuous integration system, it is important that the tool which runs the tests outputs its report in a form that the continuous integration server can interpret.

Jenkins expects the test report to be in a JUnit XML schema. Certain tools can provide this as an output from running a python testsuite:

  • 'nosetests TEST_ITEMS --with-xunit' - the XML report will be in './nosetests.xml' by default
Add a custom footer
Add a custom sidebar

Clone this wiki locally