Re: FW: [program-l] NVDA add-on development questions
toggle quoted messageShow quoted text
The following message will contain geeky material, so please hold on:
An NVDA object is NVDA’s way of looking at graphical elements. A given NVDA object has a name, role, value, states, location, among other things. But in order to represent what is seen as “real” by users, it also remembers what kind of object it is: MSAA/IAccessible, UI Automation, JAB, and many others, as evidenced by developer info listing attributes coming from these API’s. As Javi pointed out, changing the attributes of NVDA objects will not change what accessibility API’s say about a particular control – it just tells NVDA objects to act like something else, give users something other than what is actually shown on screen, among other tricks.
Technical: an NVDA object is an abstract representation of a user interface element on screen – visible or invisible, differing roles, across apps, across accessibility API’s, and many other things. A base class named NVDAObjects.NVDAObject is responsible for specifying the blueprints of this abstract class: name, role, states, app module, events, among others. Although the base NVDA object class isn’t marked as an abstract base class (ABC), you don’t really want to instantiate this class (technically, you can but under specific circumstances such as providing gestures), and several descriptors in this base class are marked with “NotImplementedError”.
Because the base NVDA object class is primitive at best, you won’t find them in real life; rather ,you’ll interact with one concrete implementation, which serves as the “real” NVDA object: NVDAObjects.window.Window. Every control you see (or can interact with) on screen is a window (including windows that are invisible). Apart from specific exceptions, every window is represented by a window handle (hwnd). Window handles shield apps (including NVDA) from direct manipulation, and without this shield, it introduces security problems. Window handles are also important for NVDA (and the next component described below) as NVDA can use it to query certain things about a window or a control such as window text and such. Unless you need a way to direct Windows to talk to controls in a controlled fashion, you won’t have to interact with window objects.
Window handle is also needed to construct what’s termed “API class”. An API class is a specification of an NVDA object based on an accessibility API’s notion of an element. Each API class (or for that matter, accessibility API’s) have differing notions as to what the screen element should say to screen readers (technically termed “accessibility clients”) and how apps and assistive technologies communicate. In short, an accessibility API such as MSAA and UI Automation serves as a “bridge” between apps (termed “servers”) and screen readers such as NVDA (clients), and as such, one of their duties is to provide information about a graphical element to NVDA based on what the app reports about a given control (now you know why I and others get frustrated about UIA implementation issues from Microsoft). Again, without a window handle for the element in question, accessibility API’s cannot do its job, and for NVDA, it cannot instantiate API classes.
Another thing an API class must have is a way to talk to the control in question via accessibility API. Each API class includes a reference to the control as seen by accessibility API. For MSAA/IAccessible, it is object.IAccessibleObject; for UIA, it is object.UIAElement. And since API classes are objects representing a control as reported by the app through accessibility API, NVDA receives directions from API’s as to how to present things such as name, role, states, location, value, among other things. Because apps are imperfect (think about what I just said for a second before moving on), information reported by IAccessible/UIA/JAB might be incomplete or incorrect; for this reason, NVDA employs tricks to provide more accurate information to the best of a developer’s knowledge (for a practical example, see Windows 10 App Essentials add-on, and many other examples from community add-ons website). And this last family of classes is what NVDA ships with, and can be used by add-ons for specific purposes (this last point is called an “overlay class”).
There is a special family of NVDA objects that doesn’t take a window handle as a required constructor argument: behaviors. Behavior classes are used to teach NVDA as to how to behave for specific controls such as terminal windows (announce incoming text automatically if given a chance), notification messages and alerts, table navigation, and auto-suggest accessibility (playing a sound when suggestions appear). Because behavior classes are meant to enhance user experience, they don’t require window handles or other fancy footwork to get them to work; rather, they are added to existing API/overlay classes (see UIA console code for practical examples).
From: email@example.com <firstname.lastname@example.org> On Behalf Of Javi Domínguez
Sent: Sunday, May 17, 2020 1:18 PM
Subject: Re: [nvda-addons] FW: [program-l] NVDA add-on development questions
editBox = api.getFocusObject ()
El 15/05/2020 a las 20:44, Adil Shaikh escribió: