Skip to content

Commit 510c2fa

Browse files
committed
more info on python harness README
1 parent 528838d commit 510c2fa

File tree

1 file changed

+37
-1
lines changed

1 file changed

+37
-1
lines changed

universal/README.md

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,10 +140,46 @@ cases have higher priority.
140140
Finally, any hint caused by a test *error* rather than a test *failure* will receive
141141
lower overall priority.
142142

143+
### Tests should FAIL, not ERROR
144+
143145
As much as possible, we want to write grading tests that result in a test
144146
**failure** rather than an error. If your grading test errors given a student
145147
solution, the test should be augmented with additional checks to result in a
146-
failure instead.
148+
failure instead. Here is an example for a test that *could* error:
149+
150+
```
151+
def test_x_is_positive(self):
152+
self.hint("x is not positive")
153+
self.assertGreater(implementation.x, 0)
154+
```
155+
156+
In case the student wrote faulty code where `implementation.x` is, for example,
157+
a string, `assertGreater` will error, because '>' not supported between
158+
instances of 'str' and 'int'. So instead of failing, this test case would error.
159+
Thus, the test suite should check if `x` is a number before checking its value.
160+
161+
```
162+
def test_x_is_positive(self):
163+
self.hint("x is not a number")
164+
self.assertIsInstance(implementation.x, numbers.Number)
165+
self.hint("x is not positive")
166+
self.assertGreater(implementation.x, 0)
167+
```
168+
169+
or more broadly:
170+
171+
```
172+
def test_x_is_positive(self):
173+
self.hint("x is not a positive number")
174+
try:
175+
self.assertGreater(implementation.x, 0)
176+
except:
177+
self.fail()
178+
```
179+
180+
It often makes sense to implement such checks in a separate (non-test) method
181+
and to call that method in every test case. You may similarily be able to use
182+
`setUp()`, but if you do, make sure you call `super().setUp()` inside it.
147183

148184
## Input function
149185

0 commit comments

Comments
 (0)