Re: FW: [program-l] NVDA add-on development questions


 

Hi,

The following message will contain geeky material, so please hold on:

An NVDA object is NVDA’s way of looking at graphical elements. A given NVDA object has a name, role, value, states, location, among other things. But in order to represent what is seen as “real” by users, it also remembers what kind of object it is: MSAA/IAccessible, UI Automation, JAB, and many others, as evidenced by developer info listing attributes coming from these API’s. As Javi pointed out, changing the attributes of NVDA objects will not change what accessibility API’s say about a particular control – it just tells NVDA objects to act like something else, give users something other than what is actually shown on screen, among other tricks.

Technical: an NVDA object is an abstract representation of a user interface element on screen – visible or invisible, differing roles, across apps, across accessibility API’s, and many other things. A base class named NVDAObjects.NVDAObject is responsible for specifying the blueprints of this abstract class: name, role, states, app module, events, among others. Although the base NVDA object class isn’t marked as an abstract base class (ABC), you don’t really want to instantiate this class (technically, you can but under specific circumstances such as providing gestures), and several descriptors in this base class are marked with “NotImplementedError”.

Because the base NVDA object class is primitive at best, you won’t find them in real life; rather ,you’ll interact with one concrete implementation, which serves as the “real” NVDA object: NVDAObjects.window.Window. Every control you see (or can interact with) on screen is a window (including windows that are invisible). Apart from specific exceptions, every window is represented by a window handle (hwnd). Window handles shield apps (including NVDA) from direct manipulation, and without this shield, it introduces security problems. Window handles are also important for NVDA (and the next component described below) as NVDA can use it to query certain things about a window or a control such as window text and such. Unless you need a way to direct Windows to talk to controls in a controlled fashion, you won’t have to interact with window objects.

Window handle is also needed to construct what’s termed “API class”. An API class is a specification of an NVDA object based on an accessibility API’s notion of an element. Each API class (or for that matter, accessibility API’s) have differing notions as to what the screen element should say to screen readers (technically termed “accessibility clients”) and how apps and assistive technologies communicate. In short, an accessibility API such as MSAA and UI Automation serves as a “bridge” between apps (termed “servers”) and screen readers such as NVDA (clients), and as such, one of their duties is to provide information about a graphical element to NVDA based on what the app reports about a given control (now you know why I and others get frustrated about UIA implementation issues from Microsoft). Again, without a window handle for the element in question, accessibility API’s cannot do its job, and for NVDA, it cannot instantiate API classes.

Another thing an API class must have is a way to talk to the control in question via accessibility API. Each API class includes a reference to the control as seen by accessibility API. For MSAA/IAccessible, it is object.IAccessibleObject; for UIA, it is object.UIAElement. And since API classes are objects representing a control as reported by the app through accessibility API, NVDA receives directions from API’s as to how to present things such as name, role, states, location, value, among other things. Because apps are imperfect (think about what I just said for a second before moving on), information reported by IAccessible/UIA/JAB might be incomplete or incorrect; for this reason, NVDA employs tricks to provide more accurate information to the best of a developer’s knowledge (for a practical example, see Windows 10 App Essentials add-on, and many other examples from community add-ons website). And this last family of classes is what NVDA ships with, and can be used by add-ons for specific purposes (this last point is called an “overlay class”).

There is a special family of NVDA objects that doesn’t take a window handle as a required constructor argument: behaviors. Behavior classes are used to teach NVDA as to how to behave for specific controls such as terminal windows (announce incoming text automatically if given a chance), notification messages and alerts, table navigation, and auto-suggest accessibility (playing a sound when suggestions appear). Because behavior classes are meant to enhance user experience, they don’t require window handles or other fancy footwork to get them to work; rather, they are added to existing API/overlay classes (see UIA console code for practical examples).

Cheers,

Joseph

 

From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of Javi Domínguez
Sent: Sunday, May 17, 2020 1:18 PM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] FW: [program-l] NVDA add-on development questions

 

Hello, Adil.

I think that's a typical beginner mistake. You've done:

editBox = api.getFocusObject ()

and you thin that editBox it is the authentic, original object, that of the applicationn but no, editBox is a virtual object created by NVDA with the attributes that it has been able to extract from the application object.

All NVDAObjects are reflections of the original objects, you can't act on the object of the application modifying the NVDA objects.

If you do:


editBox.value = "some text

you make the user see "some text", but the edit box in the screen, the reflected object, the source, I don't know how to say it, doesn't change.

So you have to call the operating system through the winUser module, as James has explained to you.


This is what I think but there are people here who know much more than I do. Correct me if I'm wrong, please.

Greetings

Javi Dominguez

El 15/05/2020 a las 20:44, Adil Shaikh escribió:

thanks to everyone who answered my query. 
I got that part of the add-on working. now, I am trying to add the selected text to a third party application.  initially, I  was considering to use clipboard to pass the selected text to third party application by just using simple copy and paste. but it doesn't feel quite right to me as it messes up with the clipboard content.  

I've already tried this code:

editBox = api.getFocusObject ()

editBox.value = "some text

it's not working. 

thanks josephly for forwarding my post to this mailing list. 

 

Join nvda-addons@nvda-addons.groups.io to automatically receive all group messages.