Lesson 3 - Evaluation Tools
Description
Automatic and Manual Checkers
There are a number of programs, such as Adobe Acrobat Pro, PAC 2.0 and CommonLook PDF Global Access (which works together with Acrobat Pro), that can carry out automatic accessibility checks. Human input is still essential in judging the number and severity of issues as they relate to a WCAG or PDF/UA passing or failing, but these tools can still be very effective and useful in finding accessibility issues. For example, a document may contain tags that have been added automatically, in which case missing tags will not be flagged in the automatic review. But these tags can be incorrect, out of order, or missing for some of the content in the document.
Automated checkers provide a good way for discovering problems that can be programmatically determined, such as:
- Missing document tags
- Incorrect tag structure
- Missing “alt” attributes for images
- Missing document titles
- Language of page not set
Comparison of Selected PDF Accessibility Evaluation Tools
PDF accessibility evaluation tools vary greatly in the standards that they use, the completeness of their results, and their ease of use. We will focus on three of the most commonly used accessibility checkers:
- Adobe Acrobat Pro XI Full Checker,
- CommonLook PDF Global Access, and
- PAC 2.0.
The most appropriate automated accessibility checker depends primarily on the set of guidelines that need to be met. A document satisfying the Acrobat Pro Full Checker and prompted manual checks will provide reasonable accessibility for many users, within the limits of the current state of the PDF format and its viewers. However, Acrobat Pro’s checker is not based on any official set of guidelines, and although many of the issues checked for are covered by both the WCAG 2.0, and PDF/UA, the results of the check will not provide enough insight to establish whether or not documents meet these guidelines. On the other hand, PAC 2.0 and CommonLook PDF Global Access base their automated checks on recognized international accessibility standards. PAC 2.0 uses the Matterhorn Protocol to establish if PDF/UA standards have been met, and CommonLook PGA allows the user to select from several sets of standards, including both PDF/UA and WCAG 2.0. A document that passes an evaluation by either one of these tools, along with the related manual checks, will not only ensure that it is reasonably accessible, but also indicate which criteria are met and which require remediation or further investigation.
Table 1: Comparison of PDF Accessibility Evaluation Tools
Detail | Adobe Acrobat Pro XI Full Checker | CommonLook PDF Global Access | PAC 2.0 |
Description | An automated accessibility checker built into Adobe Acrobat Pro (since version 7). A version number is included because Adobe periodically adjusts what the tool. Not included in the free Adobe Reader. Referenced in the WCAG 2.0 PDF Techniques. | An Adobe Acrobat Pro plugin developed by Netcentric Technologies Inc.. The plugin is both an accessibility checker and remediation tool. Referenced in the WCAG 2.0 PDF Techniques. | A free automated accessibility checker developed by the Swiss non-profit organization Access for All. No remediation is provided. Referenced in the WCAG 2.0 PDF Techniques. |
Accessibility standard used | Checks for a number of common accessibility barriers, such as presence of tags, contrast, proper language and title identification Does not fully cover PDF/UA or WCAG 2.0. Prompts to perform manual tests are included. | User can select between WCAG 2.0, PDF/UA, section 508 and HHS standards as the basis for accessibility checks. Full or selective partial checks can be carried out. Prompts to perform manual tests are included. | Machine-checkable items in the Matterhorn protocol (Checklist of PDF/UA issues). No prompts to perform manual tests are included. |
Error description | The accessibility check report provides brief descriptions of each error. More information regarding the errors as well as some guidance on fixing these can be found by right-clicking on the error and following the ‘Explain’ and ‘Fix’ links. | Error descriptions are brief and technical. No additional information is available. | Descriptions are technical and unintuitive. No additional information is available. |
Remediation support provided | One-click fixes are provided for some errors. These can be accessed by right-clicking on the error and selecting “Fix” when available. Some errors have to be fixed manually. | One click remediation is provided for most issues. Issues are cleared once resolved making the process clear and easy to follow. | No remediation support provided. |
Locating error within document | In many cases, the checker does not identify where in the document problems occur. | Highlights content that causes errors in document, as well as the corresponding tag in the tag structure tree. | Content that causes errors is highlighted in error details. |
Cost | The checker is included with the Adobe Acrobat Pro software. Prices vary between the cloud-based subscription model and desktop version, and range between $15/month and $450 US for one-time purchase. | Quote request required (but likely substantially more than Acrobat Pro, which is also required) | Free. |
Limitations | Does not guarantee compliance with specific guidelines or official standards. | Costly. Requires Adobe Acrobat Pro. | The tool doesn’t provide any remediation help. Little information provided about errors. The cause, importance, and impact of errors is not always clear. Does not assess WCAG 2.0 compliance. |
General comments | Although fairly basic, this tool is easy to use and is helpful for solving a number of issues. | CL-PGA is commonly considered the highest quality tool for assessment and remediation of PDF documents. It provides a full list of passed and failed criteria across a variety of standards, easy step-by-step remediation, and the most sophisticated table structure editor. | PAC 2.0 is free and relatively easy to use. Has additional useful features for checking accessibility, such as the Screen reader Preview document view, Document Statistics and Logical Structure. |
Using Adobe Acrobat Pro’s Full Checker
- To launch the accessibility check select Tool > Accessibility > Full Check.
- Check all boxes and run the accessibility check on the entire document.
- The results of the check will appear in a side panel, with drop-down menus for each error category.
- The accessibility check results provide an explanation of each error that it can detect and also support for fixing most errors.
- The Acrobat Full Checker includes reminders to manually check the reading order and certain content-level accessibility issues (e.g. colour contrast) that cannot be automatically checked.
Using CommonLook PDF Global Access (“CommonLook PGA”)
CommonLook PGA is a plug-in for Adobe Acrobat Pro. Once the plugin is installed, open a document you would like to check in Acrobat Pro.
- Launch CommonLook from the plugin menu.
- Once in CommonLook, in the WIndows tab, choose Standards.
- In the Standards panel, you can choose from one of four standards; Section 508, WCAG 2.0, PDF/UA and HHS. For each of these, you can choose to run a full check, or an incremental check for a specific guideline or criterion.
- The results of the check will be displayed in a results panel. Filter options are available for viewing the results of the accessibility check. Since only the failed checkpoints need addressing, selecting the Failed filter is easiest when remediating a document. Items will automatically be removed from the list as they are fixed. Selecting an individual issue will display the corresponding content in the document viewer and highlight the related tag in the Tags pane. The Fix Wizard will appear in a pop-up window displaying the error description along with the standard and checkpoint name as well as fix options.
Using PAC 2.0
PAC 2.0 is free and relatively simple to use. However, it only serves as an accessibility testing tool and does not provide remediation support. Satisfying PAC 2.0 check indicates compliance to the automatable aspects of the Matterhorn protocol (a checklist of PDF/UA).
- Launch PAC 2.0 and locate the file you would like to check by selecting the Browse button and locating it on your computer.
- Click the Start button
- Click the Report button to view details.
- Use the drop down toggles to view individual error details.
PAC 2.0 does not include reminders to manually check certain content-level accessibility issues that cannot be automatically checked.
Manual Tools
There are also manual tools that can help you with an audit, such as TPG Colour Contrast Analyzer – A small desktop app that allows you to enter a foreground colour and a background colour and checks the contrast level as it relates to WCAG guidelines. Colour Contrast Analyzer appears as floating windows (one window for Windows, two for Mac OS) that can be used within any application. Using the eyedropper tool, select foreground and background colours. The Analyzer will check if they meet the WCAG requirements.
Testing with Assistive Technology
Testing a PDF document using assistive technology (AT) is an excellent way to discover some of the more hidden accessibility issues, and observe the experience that an AT user might encounter when interacting with the document. Screen reader testing will help discover accessibility issues such as:
- Missing alternative text
- Missing tags
- Missing input labels
- Confusing content order
- Areas that are not accessible to keyboard users
There are two screen readers available that are helpful for testing and free to use:
- VoiceOver – a screen reader that comes with the Mac OS that works with Adobe Acrobat Reader. You can turn it on by pressing Command+F5. You can find keyboard shortcuts for VO here: http://webaim.org/articles/voiceover/.
- NVDA – a Windows screen reader that can be downloaded for free and used with Adobe Acrobat Reader. A list of keyboard shortcuts for NVDA can be found here: http://webaim.org/articles/nvda/
Task
- Use CommonLook PGA to inspect a document.
- Use a screen-reader, either VoiceOver or NVDA, to read and interact with the PDF. Remember to use http://webaim.org/articles/voiceover/ for VO keyboard shortcuts, and http://webaim.org/articles/nvda/ for NVDA shortcuts.
- Interact with a PDF document form using only a keyboard.
- Write about what you found interesting or challenging about using a PDF these ways, ensuring that the "Is this for an assignment?" dropdown is set to the name of this lesson.
Continue to Lesson 4 - Basic PDF Accessibility Remediation in Acrobat Pro »
Submissions (2)
-
Natalia Writing submission: 3224.7 days ago
-
Jessica Geboers Writing submission: 3234.7 days ago