Topics

detecting browse mode focus changes in an AppModule


Venkatesh Potluri
 

Hi all,


My team is currently working on an experimental app module add-on for Chrome that involves figuring out when the browse mode cursor has moved into a table. We want to make custom announcements when a user browses a webpage and focuses their browse mode cursor on a table. Though the app module does have event support for focus change with event_focusChange, it doesn’t provide support for navigator or cursor change in browse mode, which makes it impossible to figure out when the cursor is, for example, inside a plain text table where there is nothing to focus on when the user is in browse  mode. We found that in CursorManager.py, the _set_selection function calls some handlers in the braille and vision module which in turn trigger some actions on cursor move, which enables NVDAHighlighter in the vision module to notify custom EventExtentionPoint and highlight the current (focused) navigator object on the screen. We were wondering if there is a way to do something similar (detect when the focus of the browse mode cursor changes) in an app module. Should we be handling some event in the extensionPoint? Any pointers to code in the NVDA source code, or other addons that detect and handle browse mode focus changes would be of great value to us.


Thank you!


Venkatesh Potluri

PhD Student, University of Washington

https://venkateshpotluri.me


Tony Malykh
 

Hi

Starting from NVDA 2020.3 the default browse mode is fast browse mode, that means that the real system focus doesn't move when you navigate around HTML document. So you need to retrieve current element from browse mode.

Take a look at browseMode.py, within class TextInfoQuickNavItem, function def moveTo.
This function is only called from quickNav scripts. To actually update current element it calls:
self.document._set_selection(info,reason=REASON_QUICKNAV)
That function is defined in the same file. I would try to inject a hook into that function, and see whether that hook also triggers when navigating by arrow keys and tab. If that's not the case, then you'd need to read more source code to figure out how cursor navigation via arrows works.

Also CC'ing Stefan Moisei, who recently ran into similar issues.

--Tony


On 11/10/2020 7:37 AM, Venkatesh Potluri wrote:

Hi all,


My team is currently working on an experimental app module add-on for Chrome that involves figuring out when the browse mode cursor has moved into a table. We want to make custom announcements when a user browses a webpage and focuses their browse mode cursor on a table. Though the app module does have event support for focus change with event_focusChange, it doesn’t provide support for navigator or cursor change in browse mode, which makes it impossible to figure out when the cursor is, for example, inside a plain text table where there is nothing to focus on when the user is in browse  mode. We found that in CursorManager.py, the _set_selection function calls some handlers in the braille and vision module which in turn trigger some actions on cursor move, which enables NVDAHighlighter in the vision module to notify custom EventExtentionPoint and highlight the current (focused) navigator object on the screen. We were wondering if there is a way to do something similar (detect when the focus of the browse mode cursor changes) in an app module. Should we be handling some event in the extensionPoint? Any pointers to code in the NVDA source code, or other addons that detect and handle browse mode focus changes would be of great value to us.


Thank you!


Venkatesh Potluri

PhD Student, University of Washington

https://venkateshpotluri.me


Venkatesh Potluri
 

Hi Tony,

Thank you for the information, and pointers to those functions. I have
a follow-up question. Could you help us find pointers (code examples
or other addons) that hook into class methods related to NVDA? That
would help us better understand how to hook into class methods and try
the suggestions in your earlier email.

Best, Venkatesh

On 11/11/20, Tony Malykh <anton.malykh@...> wrote:
Hi

Starting from NVDA 2020.3 the default browse mode is fast browse mode,
that means that the real system focus doesn't move when you navigate
around HTML document. So you need to retrieve current element from
browse mode.

Take a look at browseMode.py, within class TextInfoQuickNavItem,
function def moveTo.
This function is only called from quickNav scripts. To actually update
current element it calls:
self.document._set_selection(info,reason=REASON_QUICKNAV)
That function is defined in the same file. I would try to inject a hook
into that function, and see whether that hook also triggers when
navigating by arrow keys and tab. If that's not the case, then you'd
need to read more source code to figure out how cursor navigation via
arrows works.

Also CC'ing Stefan Moisei, who recently ran into similar issues.

--Tony


On 11/10/2020 7:37 AM, Venkatesh Potluri wrote:

Hi all,


My team is currently working on an experimental app module add-on for
Chrome that involves figuring out when the browse mode cursor has
moved into a table. We want to make custom announcements when a user
browses a webpage and focuses their browse mode cursor on a table.
Though the app module does have event support for focus change with
event_focusChange, it doesn’t provide support for navigator or cursor
change in browse mode, which makes it impossible to figure out when
the cursor is, for example, inside a plain text table where there is
nothing to focus on when the user is in browse  mode. We found that in
CursorManager.py, the _set_selection function calls some handlers in
the braille and vision module which in turn trigger some actions on
cursor move, which enables NVDAHighlighter in the vision module to
notify custom EventExtentionPoint and highlight the current (focused)
navigator object on the screen. We were wondering if there is a way to
do something similar (detect when the focus of the browse mode cursor
changes) in an app module. Should we be handling some event in the
extensionPoint? Any pointers to code in the NVDA source code, or other
addons that detect and handle browse mode focus changes would be of
great value to us.


Thank you!


Venkatesh Potluri

PhD Student, University of Washington

https://venkateshpotluri.me <https://venkateshpotluri.me>






Tony Malykh
 

You can look at my Tony's enhancements add-on as an example of patching functions:

https://github.com/mltony/nvda-tonys-enhancements/

Look at def injectHooks function to begin with.

On 11/12/2020 8:27 AM, Venkatesh Potluri wrote:
Hi Tony,

Thank you for the information, and pointers to those functions. I have
a follow-up question. Could you help us find pointers (code examples
or other addons) that hook into class methods related to NVDA? That
would help us better understand how to hook into class methods and try
the suggestions in your earlier email.

Best, Venkatesh

On 11/11/20, Tony Malykh <anton.malykh@...> wrote:
Hi

Starting from NVDA 2020.3 the default browse mode is fast browse mode,
that means that the real system focus doesn't move when you navigate
around HTML document. So you need to retrieve current element from
browse mode.

Take a look at browseMode.py, within class TextInfoQuickNavItem,
function def moveTo.
This function is only called from quickNav scripts. To actually update
current element it calls:
self.document._set_selection(info,reason=REASON_QUICKNAV)
That function is defined in the same file. I would try to inject a hook
into that function, and see whether that hook also triggers when
navigating by arrow keys and tab. If that's not the case, then you'd
need to read more source code to figure out how cursor navigation via
arrows works.

Also CC'ing Stefan Moisei, who recently ran into similar issues.

--Tony


On 11/10/2020 7:37 AM, Venkatesh Potluri wrote:
Hi all,


My team is currently working on an experimental app module add-on for
Chrome that involves figuring out when the browse mode cursor has
moved into a table. We want to make custom announcements when a user
browses a webpage and focuses their browse mode cursor on a table.
Though the app module does have event support for focus change with
event_focusChange, it doesn’t provide support for navigator or cursor
change in browse mode, which makes it impossible to figure out when
the cursor is, for example, inside a plain text table where there is
nothing to focus on when the user is in browse  mode. We found that in
CursorManager.py, the _set_selection function calls some handlers in
the braille and vision module which in turn trigger some actions on
cursor move, which enables NVDAHighlighter in the vision module to
notify custom EventExtentionPoint and highlight the current (focused)
navigator object on the screen. We were wondering if there is a way to
do something similar (detect when the focus of the browse mode cursor
changes) in an app module. Should we be handling some event in the
extensionPoint? Any pointers to code in the NVDA source code, or other
addons that detect and handle browse mode focus changes would be of
great value to us.


Thank you!


Venkatesh Potluri

PhD Student, University of Washington

https://venkateshpotluri.me <https://venkateshpotluri.me>