Ignored tests are executed and fail

[michael-mohemian]

Actual

Test classes which are annotated with @Ignore get executed and fail.

2018-11-07 13:55:07 [SDR.run] Running at.some.app.android.ui.PaymentMethodsFlowTest#null on [emulator-5554]
01:55:07 I/RemoteAndroidTest: Running am instrument -w -r   -e class 'at.some.app.client.android.ui.PaymentMethodsFlowTest#null' at.some.app.integration.test/android.support.test.runner.AndroidJUnitRunner on Nexus_5X_API_27 [emulator-5554]
2018-11-07 13:55:08 [STRL.testRunFailed] errorMessage=There was 1 failure:
1) Fatal exception when running tests
java.lang.NullPointerException: Attempt to invoke interface method 'int java.lang.CharSequence.length()' on a null object reference
at java.util.regex.Matcher.reset(Matcher.java:1052)
at java.util.regex.Matcher.<init>(Matcher.java:180)
at java.util.regex.Pattern.matcher(Pattern.java:1006)
at android.support.test.internal.runner.TestRequestBuilder$MethodFilter.stripParameterizedSuffix(TestRequestBuilder.java:524)
at android.support.test.internal.runner.TestRequestBuilder$MethodFilter.evaluateTest(TestRequestBuilder.java:510)
at android.support.test.internal.runner.TestRequestBuilder$ParentFilter.shouldRun(TestRequestBuilder.java:145)
at android.support.test.internal.runner.TestRequestBuilder$ClassAndMethodFilter.evaluateTest(TestRequestBuilder.java:452)
at android.support.test.internal.runner.TestRequestBuilder$ParentFilter.shouldRun(TestRequestBuilder.java:145)
at org.junit.runner.manipulation.Filter$3.shouldRun(Filter.java:112)
at org.junit.runners.ParentRunner.shouldRun(ParentRunner.java:434)
at org.junit.runners.ParentRunner.filter(ParentRunner.java:382)
at org.junit.runner.manipulation.Filter.apply(Filter.java:97)
at android.support.test.internal.runner.TestRequestBuilder$LenientFilterRequest.getRunner(TestRequestBuilder.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:58)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:375)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2075)

Reproducible Spoon versions: 1.3.0, 1.4.0, 1.4.0-SNAPSHOT

Expected

Ignored tests should not cause an exception.

Environment

Android Gradle plugin: 3.2.1

Observations

• With an older Android Gralde plugin 3.0.1 ignored tests are still executed but they don't cause a crash.

2018-11-07 14:13:29 [SDR.run] Running at.some.app.client.android.ui.PaymentMethodsFlowTest#null on [emulator-5554]
02:13:29 I/RemoteAndroidTest: Running am instrument -w -r   -e class at.some.app.client.android.ui.PaymentMethodsFlowTest#null at.some.app.integration.test/android.support.test.runner.AndroidJUnitRunner on Nexus_5X_API_27 [emulator-5554]

• Running the task connectedAndroidTest works without issues.

Workaround

Add the @Ignore annotation to every method of the test class vs. only to the test class itself.