From 7250f1f246a3d4800703fa070faff748615b5877 Mon Sep 17 00:00:00 2001 From: rachaelbradley Date: Thu, 15 Sep 2022 10:43:59 -0400 Subject: [PATCH 01/10] Move issue severity work into main guidelines document --- guidelines/index.html | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/guidelines/index.html b/guidelines/index.html index 162615ce..01c7122c 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -446,6 +446,36 @@

Technology specific testing

Each outcome includes methods associated with different technologies. Each method contains tests and techniques for satisfying the outcome. The outcome is written so that testers can test the accessibility of new and emerging technologies that do not have related methods based solely on the outcome.

+ +
+

Critical Failures

+ +
+

This section is exploratory. +

Severity rating could contribute towards scoring and prioritizatoin. This is a potential way to replace A/AA/AAA at the test level.

Outstanding questions that need to be addressed include:

+
    +
  1. What to do with non-critical issues?
    2. +
  2. How best to assign severity, particularly if testers have different ideas on what is critical?
    3. +
  3. How do we incorporate context/process/task? Is that part of scoping, or issue severity? Both are important to the end result.
    4. +
  4. Can the matrix inform designation of functional categories? For example, the Text Alternative Available outcome.
    5. +
  5. How will issue severity fit into levels? For example:
      +
    • "Bronze" could be an absence of any critical or high issues;
      • +
    • "Silver" could be an absence of any critical, high, or medium issues.
      • +
    +
    6. +
  6. How to account for cumulative errors becoming critical? +
+
+ +

Tests will include critical failures. This may be done either by:

+ + +
+ + From d4209a2d29fef3c74a4cf03a80efe7fb0de3c11f Mon Sep 17 00:00:00 2001 From: rachaelbradley Date: Thu, 15 Sep 2022 10:50:47 -0400 Subject: [PATCH 02/10] Update index.html --- guidelines/index.html | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/guidelines/index.html b/guidelines/index.html index 01c7122c..805a58d0 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -397,7 +397,7 @@

Conditional tests

-
+

Conventional tests

Conventional tests evaluate the results within a particular context. The tests are still unconditional or conditional tests but the context dictates:

  • which unconditional or conditional tests are used, or
    • @@ -441,14 +441,9 @@

    Procedural tests

    The requirements for what would be evaluated for procedural tests are to be determined.

-
-

Technology specific testing

-

Each outcome includes methods associated with different technologies. Each method contains tests and techniques for satisfying the outcome. The outcome is written so that testers can test the accessibility of new and emerging technologies that do not have related methods based solely on the outcome.

-
-
-

Critical Failures

+

Critical Errors

This section is exploratory. @@ -472,8 +467,12 @@

Critical Failures

  • Categorizing tests that, when they do not meet criteria, always cause a critical issues. Examples of critical failures in tests are at Text Alternative Available and Translates Speech And Non-Speech Audio.
  • Assign a critical failure after testing is complete based on task or type of task
    • -
    +
    +

    Technology specific testing

    +

    Each outcome includes methods associated with different technologies. Each method contains tests and techniques for satisfying the outcome. The outcome is written so that testers can test the accessibility of new and emerging technologies that do not have related methods based solely on the outcome.

    +
    + From 278a124bb9acff9d76d912388a1b7e0c20ec70fd Mon Sep 17 00:00:00 2001 From: rachaelbradley Date: Thu, 15 Sep 2022 10:53:35 -0400 Subject: [PATCH 03/10] Update index.html --- guidelines/index.html | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/guidelines/index.html b/guidelines/index.html index 805a58d0..7f1c3833 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -440,10 +440,15 @@

    Procedural tests

    The requirements for what would be evaluated for procedural tests are to be determined.

    - +
    +

    Technology specific testing

    +

    Each outcome includes methods associated with different technologies. Each method contains tests and techniques for satisfying the outcome. The outcome is written so that testers can test the accessibility of new and emerging technologies that do not have related methods based solely on the outcome.

    +
    + +
    -

    Critical Errors

    +

    Critical Errors

    This section is exploratory. @@ -462,17 +467,14 @@

    Critical Errors

    -

    Tests will include critical failures. This may be done either by:

    +

    Tests will include critical errors. This may be done either by:

    -
    -

    Technology specific testing

    -

    Each outcome includes methods associated with different technologies. Each method contains tests and techniques for satisfying the outcome. The outcome is written so that testers can test the accessibility of new and emerging technologies that do not have related methods based solely on the outcome.

    -
    - + +
    From 77ba917a26b758b46846ea79e29e618dffc17474 Mon Sep 17 00:00:00 2001 From: rachaelbradley Date: Thu, 15 Sep 2022 10:58:46 -0400 Subject: [PATCH 04/10] Update index.html --- guidelines/index.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/guidelines/index.html b/guidelines/index.html index 7f1c3833..cbc911d9 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -447,8 +447,8 @@

    Technology specific testing

    -
    -

    Critical Errors

    +
    +

    Critical Errors

    This section is exploratory. From d58bf90c5c6e959b12485a9a78e1d33a12a06c04 Mon Sep 17 00:00:00 2001 From: rachaelbradley Date: Thu, 15 Sep 2022 10:59:42 -0400 Subject: [PATCH 05/10] Update index.html --- guidelines/index.html | 3 --- 1 file changed, 3 deletions(-) diff --git a/guidelines/index.html b/guidelines/index.html index cbc911d9..f4ab9de8 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -445,7 +445,6 @@

    Technology specific testing

    Each outcome includes methods associated with different technologies. Each method contains tests and techniques for satisfying the outcome. The outcome is written so that testers can test the accessibility of new and emerging technologies that do not have related methods based solely on the outcome.

    -

    Critical Errors

    @@ -475,8 +474,6 @@

    Critical Errors

    -
    -
    From 0c0f34d54242c5a6a824d4efa3b4d0a9eacd4cae Mon Sep 17 00:00:00 2001 From: rachaelbradley Date: Thu, 15 Sep 2022 11:01:02 -0400 Subject: [PATCH 06/10] Update index.html --- guidelines/index.html | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/guidelines/index.html b/guidelines/index.html index f4ab9de8..2767d519 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -444,7 +444,6 @@

    Procedural tests

    Technology specific testing

    Each outcome includes methods associated with different technologies. Each method contains tests and techniques for satisfying the outcome. The outcome is written so that testers can test the accessibility of new and emerging technologies that do not have related methods based solely on the outcome.

    -

    Critical Errors

    @@ -472,7 +471,7 @@

    Critical Errors

  • Assign a critical error after testing is complete based on task or type of task.
    • - + From c25fe378ef57ce51e7bda75207513d528d9b2d90 Mon Sep 17 00:00:00 2001 From: Alastair Campbell Date: Thu, 15 Sep 2022 17:07:10 +0100 Subject: [PATCH 07/10] Updates from meeting --- guidelines/index.html | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/guidelines/index.html b/guidelines/index.html index 2767d519..556a99ca 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -446,11 +446,12 @@

    Technology specific testing

    -

    Critical Errors

    +

    Critical Issues

    This section is exploratory. -

    Severity rating could contribute towards scoring and prioritizatoin. This is a potential way to replace A/AA/AAA at the test level.

    Outstanding questions that need to be addressed include:

    +

    Severity rating could contribute towards scoring and prioritization. This is a potential way to replace how A/AA/AAA levels represented severity by incorporating a mechanism to express severity as a part of testing.

    +

    Outstanding questions that need to be addressed include:

    1. What to do with non-critical issues?
    2. How best to assign severity, particularly if testers have different ideas on what is critical?
      3. @@ -461,14 +462,14 @@

      Critical Errors

    3. "Silver" could be an absence of any critical, high, or medium issues.
      4. -
    4. How to account for cumulative errors becoming critical? +
    5. How to account for cumulative issues becoming critical?
    -

    Tests will include critical errors. This may be done either by:

    +

    Tests will include critical issues. This may be done either by:

    From 0685df12270424fc3edf0d1e9249f23edde1662d Mon Sep 17 00:00:00 2001 From: Alastair Campbell Date: Thu, 22 Sep 2022 12:16:08 +0100 Subject: [PATCH 08/10] Adjusting the alternative approach to crit issues. --- guidelines/index.html | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/guidelines/index.html b/guidelines/index.html index 556a99ca..840b1bd3 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -462,15 +462,12 @@

    Critical Issues

  • "Silver" could be an absence of any critical, high, or medium issues.
    • -
  • How to account for cumulative issues becoming critical? +
  • How to account for cumulative issues becoming critical?
    • +
  • Should another approach be more effective, for example assigning critical issues after testing is complete based on task or type of task rather than by test.
    • -

    Tests will include critical issues. This may be done either by:

    - +

    Tests will include critical issues. Each test will have a category of severity, so some tests will be flagged as causing a critical issue. Examples of critical issues in tests are at Text Alternative Available and Translates Speech And Non-Speech Audio.

    From c4f15d75f75688e4b9b8ec45a4c7f7aad4829822 Mon Sep 17 00:00:00 2001 From: Alastair Campbell Date: Thu, 22 Sep 2022 12:22:37 +0100 Subject: [PATCH 09/10] UPdates from TPAC --- guidelines/index.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/guidelines/index.html b/guidelines/index.html index 840b1bd3..f1cc7485 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -450,7 +450,7 @@

    Critical Issues

    This section is exploratory. -

    Severity rating could contribute towards scoring and prioritization. This is a potential way to replace how A/AA/AAA levels represented severity by incorporating a mechanism to express severity as a part of testing.

    +

    Severity rating could contribute towards scoring and prioritization. This is a potential way to replace how A/AA/AAA levels represented severity by incorporating a mechanism to evaluate severity as a part of testing.

    Outstanding questions that need to be addressed include:

    1. What to do with non-critical issues?
      2. @@ -463,7 +463,7 @@

      Critical Issues

    2. How to account for cumulative issues becoming critical?
      3. -
    3. Should another approach be more effective, for example assigning critical issues after testing is complete based on task or type of task rather than by test.
      4. +
    4. Would another approach be more effective, for example assigning critical issues after testing is complete based on task or type of task rather than by test.
    From 8a6949b75c0be1367d565876f4c95bd4f9c0e6fc Mon Sep 17 00:00:00 2001 From: rachaelbradley Date: Mon, 17 Oct 2022 09:04:18 -0400 Subject: [PATCH 10/10] Update index.html --- guidelines/index.html | 1 + 1 file changed, 1 insertion(+) diff --git a/guidelines/index.html b/guidelines/index.html index f1cc7485..cfa4c3c5 100644 --- a/guidelines/index.html +++ b/guidelines/index.html @@ -456,6 +456,7 @@

    Critical Issues

  • What to do with non-critical issues?
  • How best to assign severity, particularly if testers have different ideas on what is critical?
  • How do we incorporate context/process/task? Is that part of scoping, or issue severity? Both are important to the end result.
    • +
  • If included, how will situations where issue severity depends on context be handled?
  • Can the matrix inform designation of functional categories? For example, the Text Alternative Available outcome.
  • How will issue severity fit into levels? For example:
    • "Bronze" could be an absence of any critical or high issues;