requesting review of the SPRITEs Table Navigation for Chrome addon


Venkatesh Potluri
 

Hi list,


I would like to request review of SPRITEs (pronounced Sprites) table navigation for Chrome. The code is hosted on github and a version of the packaged  addon is hosted on the releases page. The internals of this addon are documented in the wiki.


This addon is a research product developed by my team at University of Washington. The addon introduces Sprites mode, a new mode of nonlinear table navigation. To briefly describe, enabling Sprites mode maps the top row of the table to the top rows of keys on the keyboard (grave, keys 1 through equals) and the selected column to the keys towards the left of the keyboard (tab, capslock, left shift, and left control). Users can navigate to a row and column in a table by pressing the corresponding keys and exit the Sprites mode by pressing the escape key. The goal with this approach is to let users explore tables spatially when needed. In addition to table navigation, the addon also has functionality to support searching for information in large tables. We explain the addon in this readme, and designed a tutorial to introduce users to this new interaction technique.



This addon was developed based on earlier research out of our group that explored a set of nonlinear interaction techniques on a keyboard surface (link to the earlier paper). In addition to relying on this early work, we validated the usability of this addon and our tutorial with a blind screen reader user through a pilot study (this five-minute talk by my colleague summarizes the effort). In addition to this validation, I have been contributing to this addon as the lead on this project and have been actively using it as I use NVDA for my screen reading.


We request you to review this addon to publish this on the community website. Please let me know if we can answer any questions.


Best regards, Venkatesh Potluri

PhD Student, University of Washington

 


 

Hi,

Basic review results (based on my initial impressions and reading the source code):

  • License and copyright: pass (MIT license  is compatible with GPL, according to Wikipedia)
  • Documentation: pass
  • User experience: fail (see comments)
  • Security: pass

 

Comments:

  • Documentation is well thought out, complete with a tutorial.
  • Unfortunately, user experience fails because when I try to get NVDA to announce columns and rows in example tutorial (from Chrome 91.0.4472.77), NVDA doesn’t announce cell contents. If the add-on is supposed to announce cell contents after moving to a specific row/column via Sprites mode (as implied from the tutorial), then I consider this a critical error that should be addressed before going any further. You can ignore this if the intention is doing something other than announcing cell contents, in which case user experience passes. Also, NVDA doesn’t seem to add a space between row commands (NVDA says “grawv comma tab comma, etc.), which could introduce issues for braille display users. Also, note that not all keyboards place left Control key at the bottom left corner – Lenovo ThinkPad laptops place the Fn key as the bottom-left corner key, and so does MacBook keyboards (imagine people running Chrome under Windows on top of Bootcamp, and you can see why this could be an issue; as far as docs goes, it is fine to state that the left Control key will move to the fifth row).

 

For future considerations, have you investigated applying Sprites concept to other browsers such as Microsoft Edge (Chromium-based) and Firefox? I would imagine Edge is the easiest to apply as it shares commonality with Chrome (both are Chromium-based). Also, in case people ask for it, I advise looking into supporting UI Automation and web browsers as it may expose different information.

Cheers,

Joseph

 

From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of Venkatesh Potluri
Sent: Tuesday, June 8, 2021 9:16 AM
To: nvda-addons@nvda-addons.groups.io
Subject: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

Hi list,

 

I would like to request review of SPRITEs (pronounced Sprites) table navigation for Chrome. The code is hosted on github and a version of the packaged  addon is hosted on the releases page. The internals of this addon are documented in the wiki.

 

This addon is a research product developed by my team at University of Washington. The addon introduces Sprites mode, a new mode of nonlinear table navigation. To briefly describe, enabling Sprites mode maps the top row of the table to the top rows of keys on the keyboard (grave, keys 1 through equals) and the selected column to the keys towards the left of the keyboard (tab, capslock, left shift, and left control). Users can navigate to a row and column in a table by pressing the corresponding keys and exit the Sprites mode by pressing the escape key. The goal with this approach is to let users explore tables spatially when needed. In addition to table navigation, the addon also has functionality to support searching for information in large tables. We explain the addon in this readme, and designed a tutorial to introduce users to this new interaction technique.

 

This addon was developed based on earlier research out of our group that explored a set of nonlinear interaction techniques on a keyboard surface (link to the earlier paper). In addition to relying on this early work, we validated the usability of this addon and our tutorial with a blind screen reader user through a pilot study (this five-minute talk by my colleague summarizes the effort). In addition to this validation, I have been contributing to this addon as the lead on this project and have been actively using it as I use NVDA for my screen reading.

 

We request you to review this addon to publish this on the community website. Please let me know if we can answer any questions.

 

Best regards, Venkatesh Potluri

PhD Student, University of Washington

 


Rui Fontes
 

In addition to the comments of Joseph Lee, here are some more...

1 - Why not allow Alt+Tab, and other Windows commands like Alt+F4,  to perform his function?

2 - In manifest.ini you declare:

lastTestedNVDAVersion = 2021.1

but it is not true.

In line 432 of __init__.py:

        speech.speakTextInfo(info, formatConfig=formatConfig, reason=controlTypes.REASON_CARET)

for compliance to NVDA 2021.1 you must change to:

        speech.speakTextInfo(info, formatConfig=formatConfig, reason=controlTypes.OutputReason.CARET)


it is because of that issue that the cell contents is not announced.


3 - In my case, the installTasks.py only add to NVDA.ini the following:

[sprites]
    spritesID = 0
    searchID = 0
    logID = 0
    userID = f2014c8c-c881-11eb-9097-b808cf3aee32
    logPath = C:\Users\Utilizador\AppData\Roaming\nvda\sprites
    logStart = 2021-06-08


So, as firstUse param is not defined, it gaves an error everytime I go to Chrome.


Rui Fontes




Às 17:16 de 08/06/2021, Venkatesh Potluri escreveu:

Hi list,


I would like to request review of SPRITEs (pronounced Sprites) table navigation for Chrome. The code is hosted on github and a version of the packaged  addon is hosted on the releases page. The internals of this addon are documented in the wiki.


This addon is a research product developed by my team at University of Washington. The addon introduces Sprites mode, a new mode of nonlinear table navigation. To briefly describe, enabling Sprites mode maps the top row of the table to the top rows of keys on the keyboard (grave, keys 1 through equals) and the selected column to the keys towards the left of the keyboard (tab, capslock, left shift, and left control). Users can navigate to a row and column in a table by pressing the corresponding keys and exit the Sprites mode by pressing the escape key. The goal with this approach is to let users explore tables spatially when needed. In addition to table navigation, the addon also has functionality to support searching for information in large tables. We explain the addon in this readme, and designed a tutorial to introduce users to this new interaction technique.



This addon was developed based on earlier research out of our group that explored a set of nonlinear interaction techniques on a keyboard surface (link to the earlier paper). In addition to relying on this early work, we validated the usability of this addon and our tutorial with a blind screen reader user through a pilot study (this five-minute talk by my colleague summarizes the effort). In addition to this validation, I have been contributing to this addon as the lead on this project and have been actively using it as I use NVDA for my screen reading.


We request you to review this addon to publish this on the community website. Please let me know if we can answer any questions.


Best regards, Venkatesh Potluri

PhD Student, University of Washington

 


 

Hi,

Speech output reason enumeration: ah, makes sense – no wonder cell contents were not announced.

]When declaring support for the latest NVDA release (especially for backwards incompatible version), you MUST make sure the source code doesn’t use deprecated or removed functionality unless one needs to support older releases. In case of output reason enumeration, this was introduced in NVDA 2020.1, so changing to speech output reason enumeration won’t pose a problem for this ad-don (minimum NVDA version is 2020.3, according to the manifest).

Cheers,

Joseph

 

From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of Rui Fontes
Sent: Tuesday, June 8, 2021 12:07 PM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

In addition to the comments of Joseph Lee, here are some more...

1 - Why not allow Alt+Tab, and other Windows commands like Alt+F4,  to perform his function?

2 - In manifest.ini you declare:

lastTestedNVDAVersion = 2021.1

but it is not true.

In line 432 of __init__.py:

        speech.speakTextInfo(info, formatConfig=formatConfig, reason=controlTypes.REASON_CARET)

for compliance to NVDA 2021.1 you must change to:

        speech.speakTextInfo(info, formatConfig=formatConfig, reason=controlTypes.OutputReason.CARET)

 

it is because of that issue that the cell contents is not announced.

 

3 - In my case, the installTasks.py only add to NVDA.ini the following:

[sprites]
    spritesID = 0
    searchID = 0
    logID = 0
    userID = f2014c8c-c881-11eb-9097-b808cf3aee32
    logPath = C:\Users\Utilizador\AppData\Roaming\nvda\sprites
    logStart = 2021-06-08

 

So, as firstUse param is not defined, it gaves an error everytime I go to Chrome.

 

Rui Fontes

 

 

 

Às 17:16 de 08/06/2021, Venkatesh Potluri escreveu:

Hi list,

 

I would like to request review of SPRITEs (pronounced Sprites) table navigation for Chrome. The code is hosted on github and a version of the packaged  addon is hosted on the releases page. The internals of this addon are documented in the wiki.

 

This addon is a research product developed by my team at University of Washington. The addon introduces Sprites mode, a new mode of nonlinear table navigation. To briefly describe, enabling Sprites mode maps the top row of the table to the top rows of keys on the keyboard (grave, keys 1 through equals) and the selected column to the keys towards the left of the keyboard (tab, capslock, left shift, and left control). Users can navigate to a row and column in a table by pressing the corresponding keys and exit the Sprites mode by pressing the escape key. The goal with this approach is to let users explore tables spatially when needed. In addition to table navigation, the addon also has functionality to support searching for information in large tables. We explain the addon in this readme, and designed a tutorial to introduce users to this new interaction technique.

 

This addon was developed based on earlier research out of our group that explored a set of nonlinear interaction techniques on a keyboard surface (link to the earlier paper). In addition to relying on this early work, we validated the usability of this addon and our tutorial with a blind screen reader user through a pilot study (this five-minute talk by my colleague summarizes the effort). In addition to this validation, I have been contributing to this addon as the lead on this project and have been actively using it as I use NVDA for my screen reading.

 

We request you to review this addon to publish this on the community website. Please let me know if we can answer any questions.

 

Best regards, Venkatesh Potluri

PhD Student, University of Washington

 


Rui Fontes
 

Hello!


Some more problems:

1 - Not every one use a US keyboard...

So, 11th and 12th or are not accessible or are in a completly different location...


2 - Braille display keys still work in SPRITES mode;


3 - Both Insert keys announce the cell content, so they are not totally deactivated...


Rui Fontes


Às 20:07 de 08/06/2021, Rui Fontes via groups.io escreveu:

In addition to the comments of Joseph Lee, here are some more...

1 - Why not allow Alt+Tab, and other Windows commands like Alt+F4,  to perform his function?

2 - In manifest.ini you declare:

lastTestedNVDAVersion = 2021.1

but it is not true.

In line 432 of __init__.py:

        speech.speakTextInfo(info, formatConfig=formatConfig, reason=controlTypes.REASON_CARET)

for compliance to NVDA 2021.1 you must change to:

        speech.speakTextInfo(info, formatConfig=formatConfig, reason=controlTypes.OutputReason.CARET)


it is because of that issue that the cell contents is not announced.


3 - In my case, the installTasks.py only add to NVDA.ini the following:

[sprites]
    spritesID = 0
    searchID = 0
    logID = 0
    userID = f2014c8c-c881-11eb-9097-b808cf3aee32
    logPath = C:\Users\Utilizador\AppData\Roaming\nvda\sprites
    logStart = 2021-06-08


So, as firstUse param is not defined, it gaves an error everytime I go to Chrome.


Rui Fontes




Às 17:16 de 08/06/2021, Venkatesh Potluri escreveu:

Hi list,


I would like to request review of SPRITEs (pronounced Sprites) table navigation for Chrome. The code is hosted on github and a version of the packaged  addon is hosted on the releases page. The internals of this addon are documented in the wiki.


This addon is a research product developed by my team at University of Washington. The addon introduces Sprites mode, a new mode of nonlinear table navigation. To briefly describe, enabling Sprites mode maps the top row of the table to the top rows of keys on the keyboard (grave, keys 1 through equals) and the selected column to the keys towards the left of the keyboard (tab, capslock, left shift, and left control). Users can navigate to a row and column in a table by pressing the corresponding keys and exit the Sprites mode by pressing the escape key. The goal with this approach is to let users explore tables spatially when needed. In addition to table navigation, the addon also has functionality to support searching for information in large tables. We explain the addon in this readme, and designed a tutorial to introduce users to this new interaction technique.



This addon was developed based on earlier research out of our group that explored a set of nonlinear interaction techniques on a keyboard surface (link to the earlier paper). In addition to relying on this early work, we validated the usability of this addon and our tutorial with a blind screen reader user through a pilot study (this five-minute talk by my colleague summarizes the effort). In addition to this validation, I have been contributing to this addon as the lead on this project and have been actively using it as I use NVDA for my screen reading.


We request you to review this addon to publish this on the community website. Please let me know if we can answer any questions.


Best regards, Venkatesh Potluri

PhD Student, University of Washington

 


Venkatesh Potluri
 

Hi Joseph and Rui,


Thank you for reviewing our addon and providing feedback. We would like to address the feedback we received so far.

  • User experience feedback from Joseph and error reported by Rui.

We now have a new release on github (tagged v0.9.1-beta) that should fix the chrome announcement issue. We apologize about the confusion regarding the compatibility with NVDA 2021.1. This should address the announcement issue Joseph was talking about, and the error Rui was talking about. Since the previous error was related to first-time use, we request you to uninstall the addon, re-start NVDA and install the new addon. Rui, please let us know if you still get an error with Sprites on chrome. We request you to send us the log file that Sprites generates to help us understand the issue if it still persists (instructions in this issue template).

  • Retaining functionality of tab, alt-F4, etc.

Since sprites mode changes how keyboard interactions work, we made a design decision to prevent the user from switching away from the page on Chrome that they activated Sprites mode on to prevent confusion. The user can exit Sprites mode by pressing Escape, restoring NVDA’s default behavior and functionality, at which point they can switch windows using alt-tab or exit by pressing alt-f4.

  • We were unclear about the comments related to braille display keys working and the cell announcements being problematic. More context would help us address the comment as I do not use a braille display and we do not have access to one to test.

  • Supporting different keyboard layouts.

By keys eleven and 12, were you referring to the - (dash) and =(equals) keys? Could you clarify the keyboard layout that you are using? This perhaps can be addressed to some degree if we are able to have this information. We recognize that we are unable to support all keyboard layouts at this point. We will need to explore feasibility to expand layouts that we can support in the future. We can clarify this limitation in the documentation for now.

  • Braille display and row commands announcement.

Thank you for pointing this out Joseph. We changed the string to use the , character, and have spaces. Could you help us identify if this fixes the issue?

  • Supporting UIA.

Thank you for the advice. We will add this to our roadmap and ensure support for UIA in an update to this addon once it is released. We became aware of NVDA’s UIA support recently so were unable to support it in this release.

  • Supporting Chromium based browsers like edge.

Yes. We considered this, but decided to restrict to Chrome to reduce technical complexity as this is a very new interaction technique, and this is the first time we were building an NVDA addon.

Look forward to hear from you.
Best, Venkatesh


Cyrille
 

Hello Venkatesh

 

Regarding the localization of 11th and 12th key, you may take two alternative example on two add-ons:

  • Outlook Extended: I made this key translated by the add-on’s translators
  • PC Keyboard Braille Input for NVDA: look at the key mappings in the code. It’s not about 11th and 12th key of the first row. But for example in French, you may have remapping of keys A, Z, Y, W, M and ;.

Cheers,

 

Cyrille

 

De : nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> De la part de Venkatesh Potluri
Envoyé : mercredi 9 juin 2021 02:33
À : nvda-addons@nvda-addons.groups.io
Objet : Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

Hi Joseph and Rui,

 

Thank you for reviewing our addon and providing feedback. We would like to address the feedback we received so far.

  •  
  • User experience feedback from Joseph and error reported by Rui.
  •  

We now have a new release on github (tagged v0.9.1-beta) that should fix the chrome announcement issue. We apologize about the confusion regarding the compatibility with NVDA 2021.1. This should address the announcement issue Joseph was talking about, and the error Rui was talking about. Since the previous error was related to first-time use, we request you to uninstall the addon, re-start NVDA and install the new addon. Rui, please let us know if you still get an error with Sprites on chrome. We request you to send us the log file that Sprites generates to help us understand the issue if it still persists (instructions in this issue template).

  •  
  • Retaining functionality of tab, alt-F4, etc.
  •  

Since sprites mode changes how keyboard interactions work, we made a design decision to prevent the user from switching away from the page on Chrome that they activated Sprites mode on to prevent confusion. The user can exit Sprites mode by pressing Escape, restoring NVDA’s default behavior and functionality, at which point they can switch windows using alt-tab or exit by pressing alt-f4.

  •  
  • We were unclear about the comments related to braille display keys working and the cell announcements being problematic. More context would help us address the comment as I do not use a braille display and we do not have access to one to test.
  •  
  •  
  • Supporting different keyboard layouts.
  •  

By keys eleven and 12, were you referring to the - (dash) and =(equals) keys? Could you clarify the keyboard layout that you are using? This perhaps can be addressed to some degree if we are able to have this information. We recognize that we are unable to support all keyboard layouts at this point. We will need to explore feasibility to expand layouts that we can support in the future. We can clarify this limitation in the documentation for now.

  •  
  • Braille display and row commands announcement.
  •  

Thank you for pointing this out Joseph. We changed the string to use the , character, and have spaces. Could you help us identify if this fixes the issue?

  •  
  • Supporting UIA.
  •  

Thank you for the advice. We will add this to our roadmap and ensure support for UIA in an update to this addon once it is released. We became aware of NVDA’s UIA support recently so were unable to support it in this release.

  •  
  • Supporting Chromium based browsers like edge.
  •  

Yes. We considered this, but decided to restrict to Chrome to reduce technical complexity as this is a very new interaction technique, and this is the first time we were building an NVDA addon.

Look forward to hear from you.
Best, Venkatesh


Rui Fontes
 

Hi Venkatesh!


1 - After removing SPRITES and restarting NVDA the following informations were not removed from nvda.ini:

[sprites]
    spritesID = 0
    searchID = 0
    logID = 0
    userID = f2014c8c-c881-11eb-9097-b808cf3aee32
    logPath = C:\Users\Utilizador\AppData\Roaming\nvda\sprites
    logStart = 2021-06-08
    firstUse = 1


The last line was added by me before checking it was a boolean value...


2 - After installation, the firstUse error is still present.

But is slightly different... Nothing was wrote on nvda.ini!


3 - The SPRITEs initial screen with info about logging, for me, should have a Ok or Close button...


4 - Setting firstUse to True, the welcome screen is shown.

Is not changed firstUse parameter from True to False.


5 - The log folder and log file are created but they are not used, supposedelly due to the next error...


6 - Important NVDA log fragments refering to SPRITEs

After changing to a Chrome window:

DEBUGWARNING - Python warning (11:13:43.369) - MainThread (5624):
C:\Users\Utilizador\AppData\Roaming\nvda\addons\sprites-nvda\appModules\chrome\__init__.py:30: DeprecationWarning: invalid escape sequence \s
  path = os.path.join(os.environ['APPDATA'], 'nvda\sprites')
DEBUGWARNING - Python warning (11:13:43.389) - MainThread (5624):
C:\Users\Utilizador\AppData\Roaming\nvda\addons\sprites-nvda\appModules\chrome\logHelper.py:13: DeprecationWarning: invalid escape sequence \l
  self.logFileName = self.path + '\log.txt'


Fragment refering the firstUse error:

error in appModule 'chrome'
Traceback (most recent call last):
  File "appModuleHandler.pyc", line 172, in fetchAppModule
  File "C:\Users\Utilizador\AppData\Roaming\nvda\addons\sprites-nvda\appModules\chrome\__init__.py", line 74, in __init__
    if config.conf['sprites']['firstUse'] == 'True':
  File "config\__init__.pyc", line 932, in __getitem__
KeyError: 'firstUse'


7 - Regarding:

• Braille display and row commands announcement.
Thank you for pointing this out Joseph. We changed the string to use the , character, and have spaces. Could you help us identify if this fixes the issue?


The statement:

We changed the string to use the , character, and have spaces.

is not true. You have changed to use ' comma ' and it was not the intended...


The line 239 is:

        rowCombo = ' comma '.join(mappedRowRange)

and should be:

        rowCombo = ', '.join(mappedRowRange)


8 - • Supporting different keyboard layouts.
The suggestion of Cyrille to allow translators to define the keys and its names is the better!


9 - • Retaining functionality of tab, alt-F4, etc.

If you have the following in the __init__.py:

    def event_appModule_loseFocus(self):
        '''
        Set inputCore to use default captureFunc if not focus on chrome
        '''
        self.toggleSpritesGesture(False)
        self.removeHooks()

    def event_appModule_gainFocus(self):
        '''
        Determine which captureFunc to use when focus back on chrome
        '''
        if self.spritesMode:
            self.toggleSpritesGesture(True)
        self.injectHooks()

it is not enough to avoid any problem with the SPRITEs commands scheme?


10 - • We were unclear about the comments related to braille display keys working and the cell announcements being problematic.

The Braille displays can have keys to gave commands to the screen reader and they keep working as advised even with the SPRITEs loaded...

By instance, I am in a table and activate SPRITEs mode.

If I press Tab in the PC keyboard I will move to second line.

If I press a Braille display key to go to next line, it works as advised avoiding the SPRITEs disabling of other commands.


By now, it is all...


Rui Fontes



Às 01:33 de 09/06/2021, Venkatesh Potluri escreveu:

Hi Joseph and Rui,


Thank you for reviewing our addon and providing feedback. We would like to address the feedback we received so far.

  • User experience feedback from Joseph and error reported by Rui.

We now have a new release on github (tagged v0.9.1-beta) that should fix the chrome announcement issue. We apologize about the confusion regarding the compatibility with NVDA 2021.1. This should address the announcement issue Joseph was talking about, and the error Rui was talking about. Since the previous error was related to first-time use, we request you to uninstall the addon, re-start NVDA and install the new addon. Rui, please let us know if you still get an error with Sprites on chrome. We request you to send us the log file that Sprites generates to help us understand the issue if it still persists (instructions in this issue template).

  • Retaining functionality of tab, alt-F4, etc.

Since sprites mode changes how keyboard interactions work, we made a design decision to prevent the user from switching away from the page on Chrome that they activated Sprites mode on to prevent confusion. The user can exit Sprites mode by pressing Escape, restoring NVDA’s default behavior and functionality, at which point they can switch windows using alt-tab or exit by pressing alt-f4.

  • We were unclear about the comments related to braille display keys working and the cell announcements being problematic. More context would help us address the comment as I do not use a braille display and we do not have access to one to test.

  • Supporting different keyboard layouts.

By keys eleven and 12, were you referring to the - (dash) and =(equals) keys? Could you clarify the keyboard layout that you are using? This perhaps can be addressed to some degree if we are able to have this information. We recognize that we are unable to support all keyboard layouts at this point. We will need to explore feasibility to expand layouts that we can support in the future. We can clarify this limitation in the documentation for now.

  • Braille display and row commands announcement.

Thank you for pointing this out Joseph. We changed the string to use the , character, and have spaces. Could you help us identify if this fixes the issue?

  • Supporting UIA.

Thank you for the advice. We will add this to our roadmap and ensure support for UIA in an update to this addon once it is released. We became aware of NVDA’s UIA support recently so were unable to support it in this release.

  • Supporting Chromium based browsers like edge.

Yes. We considered this, but decided to restrict to Chrome to reduce technical complexity as this is a very new interaction technique, and this is the first time we were building an NVDA addon.

Look forward to hear from you.
Best, Venkatesh


Venkatesh Potluri
 
Edited

Thank you Cyrille and Rui for the pointers and feedback. We attempted to address the feedback and released 0.9.2 of the addon. Please note, if you are facing issues related to the first time use not showing up, please do the following before installing 0.9.2.

  1. Uninstall Sprites.

  2. Restart NVDA.

  3. Open the developer console.

  4. Import config

  5. Config.conf[‘sprites’] = {}

  6. Config.conf.spec[‘sprites’] = {}

  7. Restart NVDA.

  8. Install 0.9.2 and restart NVDA.


Here are more comments on the feedback that was given to us.

  1. Following the above steps and installing 0.9.2 should fix the issue.

  2. The error should also be fixed. Please send us the log if this error still persists. We are unable to reproduce this issue, so we added additional print statements to get some information that should help us debug.

  3. We replaced the logging window that shows up on launching Chrome for the first time to open a new Chrome tab. We also have a button to go to the tutorial if the user wants to learn how to use Sprites. Please let us know if this is acceptable to pass the review.

  4. This should also be fixed now.

  5. Thank you for pointing this out. This also should be fixed now. Please let us know if you still see the issue.

  6. The key error should also be fixed. Please be sure to follow the steps to reset the config.conf above. Please let us know if you are still able to see this issue. We are unable to reproduce it.

  7. Other feedback.

  • Braille display and , character. This should be fixed. Please let us know if the command messages are better now.

  • Keyboard layout feedback. We incorporated the suggestion given to us to translate keys 11 and 12. Does this address the issue you were facing?
    8. While we have the code you pointed us to, we use the tab key to navigate the table. Therefore, we are unsure if processing alt+tab would be possible as we replace the _captureFunc. Further, this may create a confusing experience. We would like to retain the behavior we have now if it is an acceptable behavior as users get familiar with this new interaction technique.
    9. 
    Sprites, by design, relies on the spatial arrangement of the keys on the keyboard. Consequently, we are unable to support Braille display key-based interactions at this point.


Best, Venkatesh

 


 

Hi,

At least concerns I raised were resolved. I vote to pass user experience portion.

Anyone else have any comments? If not, I propose a green light (approved for distribution with add-on update key being “sprites”).

Cheers,

Joseph

 

From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of Venkatesh Potluri
Sent: Thursday, June 10, 2021 11:45 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

Thank you Cyrille and Rui for the pointers and feedback. We attempted to address the feedback and released 0.9.2 of the addon. Please note, if you are facing issues related to the first time use not showing up, please do the following before installing 0.9.2.

  1.  
  2. Uninstall Sprites.
  3.  
  4.  
  5. Restart NVDA.
  6.  
  7.  
  8. Open the developer console.
  9.  
  10.  
  11. Import config
  12.  
  13.  
  14. Config.conf[‘sprites’] = {}
  15.  
  16.  
  17. Config.conf.spec[‘sprites’] = {}
  18.  
  19.  
  20. Restart NVDA.
  21.  
  22.  
  23. Install 0.9.2 and restart NVDA.
  24.  

 

Here are more comments on the feedback that was given to us.

  1.  
  2. Following the above steps and installing 0.9.2 should fix the issue.
  3.  
  4.  
  5. The error should also be fixed. Please send us the log if this error still persists. We are unable to reproduce this issue, so we added additional print statements to get some information that should help us debug.
  6.  
  7.  
  8. We replaced the logging window that shows up on launching Chrome for the first time to open a new Chrome tab. We also have a button to go to the tutorial if the user wants to learn how to use Sprites. Please let us know if this is acceptable to pass the review.
  9.  
  10.  
  11. This should also be fixed now.
  12.  
  13.  
  14. Thank you for pointing this out. This also should be fixed now. Please let us know if you still see the issue.
  15.  
  16.  
  17. The key error should also be fixed. Please be sure to follow the steps to reset the config.conf above. Please let us know if you are still able to see this issue. We are unable to reproduce it.
  18.  
  19.  
  20. Other feedback.
  21.  

·          

·         Braille display and , character. This should be fixed. Please let us know if the command messages are better now.

·          

·          

·         Keyboard layout feedback. We incorporated the suggestion given to us to translate keys 11 and 12. Does this address the issue you were facing? 

·          

  1.  
  2. While we have the code you pointed us to, we use the tab key to navigate the table. Therefore, we are unsure if processing alt+tab would be possible as we replace the _captureFunc. Further, this may create a confusing experience. We would like to retain the behavior we have now if it is an acceptable behavior as users get familiar with this new interaction technique.
  3.  

11.   

    1.  
    2. Sprites, by design, relies on the spatial arrangement of the keys on the keyboard. Consequently, we are unable to support Braille display key-based interactions at this point.
    3.  

12.   

  1.  

 

Best, Venkatesh

 


Venkatesh Potluri
 

Thank you Joseph.

Best, Venkatesh


James Scholes
 

From line 29 of installTasks.py:

path = os.path.join(os.environ['APPDATA'], 'nvda\\sprites')
Please don't do this, because it is a fundamentally broken approach. Ask the running copy of NVDA for the user's config path, and then store logs there. If a user is running a portable copy of NVDA, one of two things will happen with your current code:

1. The add-on will write logs to appdata, even though no data from a portable copy should ever persist on the system. Or...
2, Worse, the user will run a portable copy on a completely different computer that has a different username, and the add-on will fail to log (or create directories for a non-existent user).

Note: the same applies to line 30 of __init__.py in your Chrome app module.

Until this is fixed, this add-on should not be given a green light, and it is slightly disappointing that this hasn't been highlighted until now.

Regards,

James Scholes

On 10/06/2021 at 5:32 pm, Venkatesh Potluri wrote:
Thank you Joseph.

Best, Venkatesh


 

Hi,
Uh oh, that's a potential security issue. Please address this as soon as possible.
Cheers,
Joseph

-----Original Message-----
From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of James Scholes
Sent: Thursday, June 10, 2021 4:54 PM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

From line 29 of installTasks.py:

path = os.path.join(os.environ['APPDATA'], 'nvda\\sprites')
Please don't do this, because it is a fundamentally broken approach. Ask the running copy of NVDA for the user's config path, and then store logs there. If a user is running a portable copy of NVDA, one of two things will happen with your current code:

1. The add-on will write logs to appdata, even though no data from a portable copy should ever persist on the system. Or...
2, Worse, the user will run a portable copy on a completely different computer that has a different username, and the add-on will fail to log (or create directories for a non-existent user).

Note: the same applies to line 30 of __init__.py in your Chrome app module.

Until this is fixed, this add-on should not be given a green light, and it is slightly disappointing that this hasn't been highlighted until now.

Regards,

James Scholes

On 10/06/2021 at 5:32 pm, Venkatesh Potluri wrote:
Thank you Joseph.

Best, Venkatesh


Rui Fontes
 

Nop!

I am still with a lot of problems with this add-on..:

- Now, nothing is write on NVDA.ini;

-The grav key do not work and I can't replace it with none..


Rui Fontes


Às 19:59 de 10/06/2021, Joseph Lee escreveu:

Hi,

At least concerns I raised were resolved. I vote to pass user experience portion.

Anyone else have any comments? If not, I propose a green light (approved for distribution with add-on update key being “sprites”).

Cheers,

Joseph

 

From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of Venkatesh Potluri
Sent: Thursday, June 10, 2021 11:45 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

Thank you Cyrille and Rui for the pointers and feedback. We attempted to address the feedback and released 0.9.2 of the addon. Please note, if you are facing issues related to the first time use not showing up, please do the following before installing 0.9.2.

  1.  
  2. Uninstall Sprites.
  3.  
  4.  
  5. Restart NVDA.
  6.  
  7.  
  8. Open the developer console.
  9.  
  10.  
  11. Import config
  12.  
  13.  
  14. Config.conf[‘sprites’] = {}
  15.  
  16.  
  17. Config.conf.spec[‘sprites’] = {}
  18.  
  19.  
  20. Restart NVDA.
  21.  
  22.  
  23. Install 0.9.2 and restart NVDA.
  24.  

 

Here are more comments on the feedback that was given to us.

  1.  
  2. Following the above steps and installing 0.9.2 should fix the issue.
  3.  
  4.  
  5. The error should also be fixed. Please send us the log if this error still persists. We are unable to reproduce this issue, so we added additional print statements to get some information that should help us debug.
  6.  
  7.  
  8. We replaced the logging window that shows up on launching Chrome for the first time to open a new Chrome tab. We also have a button to go to the tutorial if the user wants to learn how to use Sprites. Please let us know if this is acceptable to pass the review.
  9.  
  10.  
  11. This should also be fixed now.
  12.  
  13.  
  14. Thank you for pointing this out. This also should be fixed now. Please let us know if you still see the issue.
  15.  
  16.  
  17. The key error should also be fixed. Please be sure to follow the steps to reset the config.conf above. Please let us know if you are still able to see this issue. We are unable to reproduce it.
  18.  
  19.  
  20. Other feedback.
  21.  

·          

·         Braille display and , character. This should be fixed. Please let us know if the command messages are better now.

·          

·          

·         Keyboard layout feedback. We incorporated the suggestion given to us to translate keys 11 and 12. Does this address the issue you were facing? 

·          

  1.  
  2. While we have the code you pointed us to, we use the tab key to navigate the table. Therefore, we are unsure if processing alt+tab would be possible as we replace the _captureFunc. Further, this may create a confusing experience. We would like to retain the behavior we have now if it is an acceptable behavior as users get familiar with this new interaction technique.
  3.  

11.   

    1.  
    2. Sprites, by design, relies on the spatial arrangement of the keys on the keyboard. Consequently, we are unable to support Braille display key-based interactions at this point.
    3.  

12.   

  1.  

 

Best, Venkatesh

 


Wen Qiu
 

Hi everyone! My name is Wen and I am Venkatesh’s teammate on the Sprites add-on development. 


Thank you James for pointing out the security issue, and we investigated other add-ons for ways to address it. Since we still want to keep the log file to be in the same folder as the NVDA config file, one example we have found is ReadFeeds, which obtains the config file path via globalVars.appArgs.configPath. In that case, instead of writing 

path = os.path.join(os.environ['APPDATA'], 'nvda\\sprites')

We will have something like:

path = os.path.join(globalVars.appArgs.configPath, 'sprites')

Could you please let us know if this solution would address the portable copy issue? If not, it would be great if you could suggest an alternative or point us to some add-on examples that safely write to a local file on a user’s computer.


And Rui, sorry to hear that the add-on is still not working for you. If you could answer the following questions and give us more details about your add-on usage, it would be greatly appreciated:

  1. How were you testing the add-on? Did you install a build of the add-on or copy the add-on folder into the scratchpad?

  2. Are there any exceptions shown in the NVDA logs? If not, it is possible that the exceptions have been intercepted by our logging mechanism, so it would be great if you can send us the content of your log file, which should be in the sprites folder in the same directory as the NVDA config file. You can also go to the NVDA preference menu, then Settings, then the Sprites submenu, and there should be a button that opens the folder where the log file is located.


Thank you so much again for your patience and feedback. 


Best wishes, Wen



From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> on behalf of Rui Fontes <rui.fontes@...>
Sent: Thursday, June 10, 2021 5:14 PM
To: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io>
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon
 

Nop!

I am still with a lot of problems with this add-on..:

- Now, nothing is write on NVDA.ini;

-The grav key do not work and I can't replace it with none..


Rui Fontes


Às 19:59 de 10/06/2021, Joseph Lee escreveu:

Hi,

At least concerns I raised were resolved. I vote to pass user experience portion.

Anyone else have any comments? If not, I propose a green light (approved for distribution with add-on update key being “sprites”).

Cheers,

Joseph

 

From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of Venkatesh Potluri
Sent: Thursday, June 10, 2021 11:45 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

Thank you Cyrille and Rui for the pointers and feedback. We attempted to address the feedback and released 0.9.2 of the addon. Please note, if you are facing issues related to the first time use not showing up, please do the following before installing 0.9.2.

  1.  
  2. Uninstall Sprites.
  3.  
  4.  
  5. Restart NVDA.
  6.  
  7.  
  8. Open the developer console.
  9.  
  10.  
  11. Import config
  12.  
  13.  
  14. Config.conf[‘sprites’] = {}
  15.  
  16.  
  17. Config.conf.spec[‘sprites’] = {}
  18.  
  19.  
  20. Restart NVDA.
  21.  
  22.  
  23. Install 0.9.2 and restart NVDA.
  24.  

 

Here are more comments on the feedback that was given to us.

  1.  
  2. Following the above steps and installing 0.9.2 should fix the issue.
  3.  
  4.  
  5. The error should also be fixed. Please send us the log if this error still persists. We are unable to reproduce this issue, so we added additional print statements to get some information that should help us debug.
  6.  
  7.  
  8. We replaced the logging window that shows up on launching Chrome for the first time to open a new Chrome tab. We also have a button to go to the tutorial if the user wants to learn how to use Sprites. Please let us know if this is acceptable to pass the review.
  9.  
  10.  
  11. This should also be fixed now.
  12.  
  13.  
  14. Thank you for pointing this out. This also should be fixed now. Please let us know if you still see the issue.
  15.  
  16.  
  17. The key error should also be fixed. Please be sure to follow the steps to reset the config.conf above. Please let us know if you are still able to see this issue. We are unable to reproduce it.
  18.  
  19.  
  20. Other feedback.
  21.  

·          

·         Braille display and , character. This should be fixed. Please let us know if the command messages are better now.

·          

·          

·         Keyboard layout feedback. We incorporated the suggestion given to us to translate keys 11 and 12. Does this address the issue you were facing? 

·          

  1.  
  2. While we have the code you pointed us to, we use the tab key to navigate the table. Therefore, we are unsure if processing alt+tab would be possible as we replace the _captureFunc. Further, this may create a confusing experience. We would like to retain the behavior we have now if it is an acceptable behavior as users get familiar with this new interaction technique.
  3.  

11.   

    1.  
    2. Sprites, by design, relies on the spatial arrangement of the keys on the keyboard. Consequently, we are unable to support Braille display key-based interactions at this point.
    3.  

12.   

  1.  

 

Best, Venkatesh

 


 

Hi,

Regarding working with files in user config folder, another example I can offer is StationPlaylist, specifically its config hub facility:

https://github.com/josephsl/stationplaylist

 

Cheers,

Joseph

 

From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of Wen Qiu
Sent: Thursday, June 10, 2021 8:10 PM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

Hi everyone! My name is Wen and I am Venkatesh’s teammate on the Sprites add-on development. 

 

Thank you James for pointing out the security issue, and we investigated other add-ons for ways to address it. Since we still want to keep the log file to be in the same folder as the NVDA config file, one example we have found is ReadFeeds, which obtains the config file path via globalVars.appArgs.configPath. In that case, instead of writing 

path = os.path.join(os.environ['APPDATA'], 'nvda\\sprites')

We will have something like:

path = os.path.join(globalVars.appArgs.configPath, 'sprites')

Could you please let us know if this solution would address the portable copy issue? If not, it would be great if you could suggest an alternative or point us to some add-on examples that safely write to a local file on a user’s computer.

 

And Rui, sorry to hear that the add-on is still not working for you. If you could answer the following questions and give us more details about your add-on usage, it would be greatly appreciated:

  1. How were you testing the add-on? Did you install a build of the add-on or copy the add-on folder into the scratchpad?
  2. Are there any exceptions shown in the NVDA logs? If not, it is possible that the exceptions have been intercepted by our logging mechanism, so it would be great if you can send us the content of your log file, which should be in the sprites folder in the same directory as the NVDA config file. You can also go to the NVDA preference menu, then Settings, then the Sprites submenu, and there should be a button that opens the folder where the log file is located.

 

Thank you so much again for your patience and feedback. 

 

Best wishes, Wen

 


From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> on behalf of Rui Fontes <rui.fontes@...>
Sent: Thursday, June 10, 2021 5:14 PM
To: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io>
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

Nop!

I am still with a lot of problems with this add-on..:

- Now, nothing is write on NVDA.ini;

-The grav key do not work and I can't replace it with none..

 

Rui Fontes

 

Às 19:59 de 10/06/2021, Joseph Lee escreveu:

Hi,

At least concerns I raised were resolved. I vote to pass user experience portion.

Anyone else have any comments? If not, I propose a green light (approved for distribution with add-on update key being “sprites”).

Cheers,

Joseph

 

From: nvda-addons@nvda-addons.groups.io <nvda-addons@nvda-addons.groups.io> On Behalf Of Venkatesh Potluri
Sent: Thursday, June 10, 2021 11:45 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] requesting review of the SPRITEs Table Navigation for Chrome addon

 

Thank you Cyrille and Rui for the pointers and feedback. We attempted to address the feedback and released 0.9.2 of the addon. Please note, if you are facing issues related to the first time use not showing up, please do the following before installing 0.9.2.

  1.  
  2. Uninstall Sprites.
  3.  
  4.  
  5. Restart NVDA.
  6.  
  7.  
  8. Open the developer console.
  9.  
  10.  
  11. Import config
  12.  
  13.  
  14. Config.conf[‘sprites’] = {}
  15.  
  16.  
  17. Config.conf.spec[‘sprites’] = {}
  18.  
  19.  
  20. Restart NVDA.
  21.  
  22.  
  23. Install 0.9.2 and restart NVDA.
  24.  

 

Here are more comments on the feedback that was given to us.

  1.  
  2. Following the above steps and installing 0.9.2 should fix the issue.
  3.  
  4.  
  5. The error should also be fixed. Please send us the log if this error still persists. We are unable to reproduce this issue, so we added additional print statements to get some information that should help us debug.
  6.  
  7.  
  8. We replaced the logging window that shows up on launching Chrome for the first time to open a new Chrome tab. We also have a button to go to the tutorial if the user wants to learn how to use Sprites. Please let us know if this is acceptable to pass the review.
  9.  
  10.  
  11. This should also be fixed now.
  12.  
  13.  
  14. Thank you for pointing this out. This also should be fixed now. Please let us know if you still see the issue.
  15.  
  16.  
  17. The key error should also be fixed. Please be sure to follow the steps to reset the config.conf above. Please let us know if you are still able to see this issue. We are unable to reproduce it.
  18.  
  19.  
  20. Other feedback.
  21.  

·          

·         Braille display and , character. This should be fixed. Please let us know if the command messages are better now.

·          

·          

·         Keyboard layout feedback. We incorporated the suggestion given to us to translate keys 11 and 12. Does this address the issue you were facing? 

·          

  1.  
  2. While we have the code you pointed us to, we use the tab key to navigate the table. Therefore, we are unsure if processing alt+tab would be possible as we replace the _captureFunc. Further, this may create a confusing experience. We would like to retain the behavior we have now if it is an acceptable behavior as users get familiar with this new interaction technique.
  3.  

11.   

    1.  
    2. Sprites, by design, relies on the spatial arrangement of the keys on the keyboard. Consequently, we are unable to support Braille display key-based interactions at this point.
    3.  

12.   

  1.  

 

Best, Venkatesh

 


Luke Davis
 

On Jun 10, Wen Qiu wrote:

We will have something like:
path = os.path.join(globalVars.appArgs.configPath, 'sprites')
It seems like it would be more useful to the user, not to have add-ons writing files and folders to the root of the config directory.
After all, each add-on already has its own dedicated directory.
For example, yours would be at:
path = os.path.join(globalVars.appArgs.configPath, "addons", "sprites")

I know that some, like the remote add-on, place files in the main config directory, but personally I consider this to not be a best practice.

Luke


James Scholes
 

I agree with Luke, store the logs under the root directory of your add-on. And the most important point of all... don't store the log path in your config as you're doing now. This will make it broken no matter which path you use, because I might move the config to another computer where the previously stored log path doesn't exist. Just calculate the logging path every time you initialise the logging system.

Here's how I get the add-on path to store data or retrieve something I've bundled. First, put this somewhere in your add-on:

def thisAddon():
for addon in addonHandler.getRunningAddons():
if addon.name == 'whatever-the-name-of-your-addon-is':
return addon
return None

Then, somewhere else, you can do:

addonPath = thisAddon().path

Keep in mind, I only use this in personal add-ons, not published ones. For a production-quality add-on, you would probably want to make two adjustments (at least):

1. use a constant for the add-on name, or get it from the manifest. Although maybe you can only get the manifest if you know the add-on name... Whatever. Just don't hard-code it, is the point.
2. Check if the return value of thisAddon is not None.

On a similar note, I don't see much error handling in your code either. You create directories, write files, all sorts of things that can fail without checking if they even worked.

Regards,

James Scholes

On 11/06/2021 at 12:06 am, Luke Davis wrote:
On Jun 10, Wen Qiu wrote:

We will have something like:
path = os.path.join(globalVars.appArgs.configPath, 'sprites')
It seems like it would be more useful to the user, not to have add-ons
writing files and folders to the root of the config directory.
After all, each add-on already has its own dedicated directory.
For example, yours would be at:
path = os.path.join(globalVars.appArgs.configPath, "addons", "sprites")

I know that some, like the remote add-on, place files in the main
config directory, but personally I consider this to not be a best
practice.

Luke