Topics

Python dependencies in add-ons


Andy B.
 

Hi,

 

It does work.

 

From . import shared

From . import dialogs

 

Never had a problem with it.

 

 

Sent from Mail for Windows 10

 

From: Rui Fontes
Sent: Tuesday, June 9, 2020 10:05 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Sorry, but to import a module present in the same folder of the .py file, the syntax is not:

from . import modul

 

Rui Fontes

 

Às 14:05 de 09/06/2020, Cyrille via groups.io escreveu:

Hi Andy

Some thoughts, I did not test by myself however.

Maybe you should insert the path containing the html library at the beginning of the path and not at the end to ensure that this version of html library is fetched:
sys.path.insert(0, addonNamePath)
import html.parse
del sys.path[0]

Moreover, it seems that doing as you wrote, you get twice the add-on's path in sys.path. One time added by NVDA to import your add-on and a second time in the code of your add-on. I do not know if putting twice the addonpath in sys.path may generate side effect or difficulties in debug regarding import. An alternative solution is to put your html library in a "lib" subfolder and add/remove this subfolder's path.

Again, I have not tested all this by myself; it is just thoughts that came to my mind when I read your message. Hope it can help.

Cheers,

Cyrille

 

 

Le 09/06/2020 à 14:22, Andy B. a écrit :

I did the following to see if this would work:

 

  • Copied the html library from python3/lib install path and put it in my addon install folder.
  • Added the lines below to the top of my addon __init__.py file:
  • import os
  • import sys
  • addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder path
  • sys.path.append(addonNamePath)
  • import html.parser
  • del sys.path[-1]

I either get module not found errors or syntax errors because relative imports aren’t allowed. Doing the same to the lxml library works as expected. However, it depends on html.parser which NVDA removed from the html library.

 

Sent from Mail for Windows 10

 

From: Shaun Everiss
Sent: Tuesday, June 9, 2020 5:53 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Well just remember guys that while this may not become that important in say another few years, nvda is used on a lot of systems.

Not everyone has the latest and greatest and not everyone can have the latest and greatest of systems depending where they are especially now.

I know quite a few with win7 and yes xp.

While nvda latest doesn't matter in this reguard, even if we take in to mind that at minimum your system will have 4gb of ram, 30mb could be a lot to someone.

And even if this is excluded, not everyone has an ssd.

And even if that is excluded also not everyone will have the best cpu.

I know at least a few using 3rd generation intel systems and older amd systems.

I myself maintain a 3700, a i530, a 7200, and a 4500 and I know that in the second hand systems part of my tech store they are still trading crappy 2nd generation systems.

Even though a laptop of reasonable design costs between 800 and 1000 bucks easily, some can't afford it and will use older tech.

They will use it till it dies.

And even if that is not a problem not everyone even in my country has fibre.

Some have wireless, lite fiber, some still use dialup.

So while thats not many 30mb may matter.

At first I was like just load it all in, but thats why the core package never has everything in it.

In fact nvda is designed for modules to be installed.

No one needs everyone of them, but everyone has a certain subset.

So for me I have enough for all the programs that use one of course, including the os, and a few extra utilities I like.

In the home network systems, I have enough for the systems to work as well as apps on those if I need but a lot less.

In the systems I administrate though which are not mine including family servers, the only modules I have are the modules I absolutely need.

Then again if you think extra stuff is needed then it is needed.

 

 

On 9/06/2020 9:30 pm, Sean wrote:

I know that the NVDA team wants to keep the size of NVDA to a minimum.
This is important for use as a portable.

I think the pure python code doesn't take up much size.
The .pyd files of this type of modules are very size large.
If there are problems with file sizes, this can be reduced with UPX.
After all, 20-30 MB of RAM is used more, but it is an efficient method.

I wish you good coding...
As the work emerges, great ideas will continue to come from here.

On 09/06/2020 11:28, Shubham Jain wrote:

Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

 


James Scholes
 

Using NVDA’s version of html and xml breaks the imported libraries
that require html.parser or xml.parser. On the other hand, using the original Python provided versions breaks NVDA.

Breaks NVDA in what way? If you include a full version of html and xml in your add-on, modify sys.path appropriately, import bs4 or whatever and then remove your path modifications, I don't understand why there would be a conflict? I understand that the setup is not ideal, but not why it would break NVDA.

Regards,

James Scholes

On 08/06/2020 at 1:41 pm, Andy B. wrote:
Hi,
This generally works. However, it has one major problem that NVDA won’t allow add-on authors to overcome. If a dependency requires modules/libraries that NVDA removed from the standard Python library, all of the included Python libraries that attempt to import such removed libraries will fail. For example, trying to import bs4, lxml, or cssutils will terminally fail because NVDA removed html.parser and xml.parser from the standard library while maintaining the rest of the html and xml modules. Attempting to reimport the original html or xml standard Python libraries into an add-on also fails because it will cause a namespace conflict with the NVDA provided html and xml libraries. As a result, either one can exist, but not both. I attempted this and was forced to choose from a few different options:
* Use NVDA’s version of html and xml with parsers removed.
* Use my version of html or xml and ignore NVDA’s version.
* Rename the original html or xml standard Python libraries and manage
3^rd party dependencies myself.
* Give up and consider it a lost cause.
I took the last option: give up because it is a lost cause. Using NVDA’s version of html and xml breaks the imported libraries that require html.parser or xml.parser. On the other hand, using the original Python provided versions breaks NVDA. Renaming the originals to something different seems like a ton of unwanted work because I would have to trudge through all the bs4/cssutils/lxml code and rename html.parser to something else, and after all of that, we can’t be sure it will work. If someone has a fix or workaround for this problem, I am all ears. It would bring a whole new level to my add-on.
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
*From: *Sean <mailto:s.tolstoyevski@...>
*Sent: *Monday, June 8, 2020 1:40 PM
*To: *nvda-addons@nvda-addons.groups.io <mailto:nvda-addons@nvda-addons.groups.io>
*Subject: *Re: [nvda-addons] Python dependencies in add-ons
example code:
addonNamePathxX = os.path.abspath(os.path.dirname(__file__)) ^ #addon folder path
sys.path.append(addonNamePathX)
import numpy
import opencv #example modules, Not available in NVDA.
del sys.path[-1]
......
#content code
On 08/06/2020 20:33, Sean via groups.io wrote:
Hi  Subham,
I think your work will be a revolutionary thing.
Because screen readers have been working with the same logic for 20
years.
Good luck.
There is no separate tool to manage the dependencies of NVDA addons.
In addon's __init__.py file, the path of the modules is added with
sys.path.append.
Addon  writers often use this.
One thing I want to add is;
All required modules should be in the same folder.
You can write C ++ codes to Python with a tool like Cython.
Or the code section written in C ++ can be compiled as a DLL.
In this way, we can use the functions of the DLL with Ctypes.
Ctypes is more commonly used in NVDA source.
On 08/06/2020 14:19, Shubham Jain wrote:
Hello!
As part of my GSoC project
<https://summerofcode.withgoogle.com/projects/#6039693356957696>, I
am writing an add-on that allows users to get descriptions of
images. To work, the ML models depend on some python libraries
like Numpy, Pillow, onnxruntime and OpenCv. My questions are:
* Is it possible to package these libraries in an add-on?
* Since I only require a few specific functions from these
libraries, is it possible to only package those parts into
the add-on?
Alternatively, the models could be converted to run using native
C++ by depending on the LibTorch library. Is it possible to
write add-ons in native C++?
*regards,*
*Shubham Jain*
--
Sean
* Email: seantolstoyevski@...
<mailto:seantolstoyevski@...>
* GitHub: SeanTolstoyevski <https://github.com/SeanTolstoyevski/> 👨‍🦯 I’m student and programmer. I write often Python, sometimes Go
and rarely C++.
--
Sean
* Email: seantolstoyevski@...
<mailto:seantolstoyevski@...>
* GitHub: SeanTolstoyevski <https://github.com/SeanTolstoyevski/> 👨‍🦯I’m student and programmer. I write often Python, sometimes Go and rarely C++.


Andy B.
 

Hi,

 

Importing libraries/modules in my addon aren’t a problem in general. When NVDA removed parser from the html library that it uses, it broke other external libraries someone could use in addons. Now we can’t use things like lxml, bs4, tinycss, webencodings, request, cssutils, etc because they depend on html.parser. When I attempt to import a second copy of html library that contains parser, NVDA will either load my copy or its own copy, but not both. It would help a ton if NVDA could include the parsers that came with the html/xml library. After all, the parsers only total around 30k in size. At least it would solve many of my problems.

 

 

 

Sent from Mail for Windows 10

 

From: Cyrille via groups.io
Sent: Tuesday, June 9, 2020 9:24 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Hi Andy

Some thoughts, I did not test by myself however.

Maybe you should insert the path containing the html library at the beginning of the path and not at the end to ensure that this version of html library is fetched:
sys.path.insert(0, addonNamePath)
import html.parse
del sys.path[0]

Moreover, it seems that doing as you wrote, you get twice the add-on's path in sys.path. One time added by NVDA to import your add-on and a second time in the code of your add-on. I do not know if putting twice the addonpath in sys.path may generate side effect or difficulties in debug regarding import. An alternative solution is to put your html library in a "lib" subfolder and add/remove this subfolder's path.

Again, I have not tested all this by myself; it is just thoughts that came to my mind when I read your message. Hope it can help.

Cheers,

Cyrille

 

 

Le 09/06/2020 à 14:22, Andy B. a écrit :

I did the following to see if this would work:

 

  • Copied the html library from python3/lib install path and put it in my addon install folder.
  • Added the lines below to the top of my addon __init__.py file:
  • import os
  • import sys
  • addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder path
  • sys.path.append(addonNamePath)
  • import html.parser
  • del sys.path[-1]

I either get module not found errors or syntax errors because relative imports aren’t allowed. Doing the same to the lxml library works as expected. However, it depends on html.parser which NVDA removed from the html library.

 

Sent from Mail for Windows 10

 

From: Shaun Everiss
Sent: Tuesday, June 9, 2020 5:53 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Well just remember guys that while this may not become that important in say another few years, nvda is used on a lot of systems.

Not everyone has the latest and greatest and not everyone can have the latest and greatest of systems depending where they are especially now.

I know quite a few with win7 and yes xp.

While nvda latest doesn't matter in this reguard, even if we take in to mind that at minimum your system will have 4gb of ram, 30mb could be a lot to someone.

And even if this is excluded, not everyone has an ssd.

And even if that is excluded also not everyone will have the best cpu.

I know at least a few using 3rd generation intel systems and older amd systems.

I myself maintain a 3700, a i530, a 7200, and a 4500 and I know that in the second hand systems part of my tech store they are still trading crappy 2nd generation systems.

Even though a laptop of reasonable design costs between 800 and 1000 bucks easily, some can't afford it and will use older tech.

They will use it till it dies.

And even if that is not a problem not everyone even in my country has fibre.

Some have wireless, lite fiber, some still use dialup.

So while thats not many 30mb may matter.

At first I was like just load it all in, but thats why the core package never has everything in it.

In fact nvda is designed for modules to be installed.

No one needs everyone of them, but everyone has a certain subset.

So for me I have enough for all the programs that use one of course, including the os, and a few extra utilities I like.

In the home network systems, I have enough for the systems to work as well as apps on those if I need but a lot less.

In the systems I administrate though which are not mine including family servers, the only modules I have are the modules I absolutely need.

Then again if you think extra stuff is needed then it is needed.

 

 

On 9/06/2020 9:30 pm, Sean wrote:

I know that the NVDA team wants to keep the size of NVDA to a minimum.
This is important for use as a portable.

I think the pure python code doesn't take up much size.
The .pyd files of this type of modules are very size large.
If there are problems with file sizes, this can be reduced with UPX.
After all, 20-30 MB of RAM is used more, but it is an efficient method.

I wish you good coding...
As the work emerges, great ideas will continue to come from here.

On 09/06/2020 11:28, Shubham Jain wrote:

Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain

--

Sean

·        Email: seantolstoyevski@...

·        GitHub: SeanTolstoyevski

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

 


Andy B.
 

At this point, ideal isn’t an option, and I am ready to accept that fact. There isn’t a problem with importing html/xml into my addon. However, attempting to import html.parser violates the relative imports rule set in Python 3, or least I believe it is a relative imports problem. When I copy the html library found in python3 install path/lib into my addon, then include the following code at the top of the __init__.py file, I get an error in the log saying ‘import error: no module by the name html.parser found’.

 

Sample code:

import os

import sys

addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder path

sys.path.append(addonNamePath)

import html.parser

del sys.path[-1]

 

 

 

Sent from Mail for Windows 10

 

From: James Scholes
Sent: Tuesday, June 9, 2020 11:19 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

> Using NVDA’s version of html and xml breaks the imported libraries

that require html.parser or xml.parser. On the other hand, using the

original Python provided versions breaks NVDA.

 

Breaks NVDA in what way?  If you include a full version of html and xml

in your add-on, modify sys.path appropriately, import bs4 or whatever

and then remove your path modifications, I don't understand why there

would be a conflict?  I understand that the setup is not ideal, but not

why it would break NVDA.

 

Regards,

 

James Scholes

 

On 08/06/2020 at 1:41 pm, Andy B. wrote:

> Hi,

>

> This generally works. However, it has one major problem that NVDA won’t

> allow add-on authors to overcome. If a dependency requires

> modules/libraries that NVDA removed from the standard Python library,

> all of the included Python libraries that attempt to import such removed

> libraries will fail. For example, trying to import bs4, lxml, or

> cssutils will terminally fail because NVDA removed html.parser and

> xml.parser from the standard library while maintaining the rest of the

> html and xml modules. Attempting to reimport the original html or xml

> standard Python libraries into an add-on also fails because it will

> cause a namespace conflict with the NVDA provided html and xml

> libraries. As a result, either one can exist, but not both. I attempted

> this and was forced to choose from a few different options:

>

>   * Use NVDA’s version of html and xml with parsers removed.

>   * Use my version of html or xml and ignore NVDA’s version.

>   * Rename the original html or xml standard Python libraries and manage

>     3^rd party dependencies myself.

>   * Give up and consider it a lost cause.

>

> I took the last option: give up because it is a lost cause. Using NVDA’s

> version of html and xml breaks the imported libraries that require

> html.parser or xml.parser. On the other hand, using the original Python

> provided versions breaks NVDA. Renaming the originals to something

> different seems like a ton of unwanted work because I would have to

> trudge through all the bs4/cssutils/lxml code and rename html.parser to

> something else, and after all of that, we can’t be sure it will work. If

> someone has a fix or workaround for this problem, I am all ears. It

> would bring a whole new level to my add-on.

>

> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for

> Windows 10

>

> *From: *Sean <mailto:s.tolstoyevski@...>

> *Sent: *Monday, June 8, 2020 1:40 PM

> *To: *nvda-addons@nvda-addons.groups.io

> <mailto:nvda-addons@nvda-addons.groups.io>

> *Subject: *Re: [nvda-addons] Python dependencies in add-ons

>

> example code:

>

> addonNamePathxX = os.path.abspath(os.path.dirname(__file__)) ^ #addon

> folder path

> sys.path.append(addonNamePathX)

>

> import numpy

>

> import opencv #example modules, Not available in NVDA.

>

> del sys.path[-1]

>

> ......

>

> #content code

>

> On 08/06/2020 20:33, Sean via groups.io wrote:

>

>     Hi  Subham,

>

>     I think your work will be a revolutionary thing.

>     Because screen readers have been working with the same logic for 20

>     years.

>     Good luck.

>

>     There is no separate tool to manage the dependencies of NVDA addons.

>

>     In addon's __init__.py file, the path of the modules is added with

>     sys.path.append.

>     Addon  writers often use this.

>

>     One thing I want to add is;

>     All required modules should be in the same folder.

>

>     You can write C ++ codes to Python with a tool like Cython.

>     Or the code section written in C ++ can be compiled as a DLL.

>     In this way, we can use the functions of the DLL with Ctypes.

>     Ctypes is more commonly used in NVDA source.

>

>     On 08/06/2020 14:19, Shubham Jain wrote:

>

>         Hello!

>

>         As part of my GSoC project

>         <https://summerofcode.withgoogle.com/projects/#6039693356957696>, I

>         am writing an add-on that allows users to get descriptions of

>         images. To work, the ML models depend on some python libraries

>         like Numpy, Pillow, onnxruntime and OpenCv. My questions are:

>

>           * Is it possible to package these libraries in an add-on?

>           * Since I only require a few specific functions from these

>             libraries, is it possible to only package those parts into

>             the add-on?

>

>

>         Alternatively, the models could be converted to run using native

>         C++ by depending on the LibTorch library. Is it possible to

>         write add-ons in native C++?

>

>         *regards,*

>         *Shubham Jain*

>

>     --

>

>

>         Sean

>

>       * Email: seantolstoyevski@...

>         <mailto:seantolstoyevski@...>

>       * GitHub: SeanTolstoyevski <https://github.com/SeanTolstoyevski/>

>

>     👨‍🦯 I’m student and programmer. I write often Python, sometimes Go

>     and rarely C++.

>

> --

>

>

>     Sean

>

>   * Email: seantolstoyevski@...

>     <mailto:seantolstoyevski@...>

>   * GitHub: SeanTolstoyevski <https://github.com/SeanTolstoyevski/>

>

> 👨‍🦯I’m student and programmer. I write often Python, sometimes Go and

> rarely C++.

>

>

 

 

 


Alberto Buffolino
 

Andy B., il 09/06/2020 17.29, ha scritto:
When I attempt to import a second copy of html library that contains parser, NVDA will either load my copy or its own copy, but not both.
Alberto:
Hi Andy,
I'm not sure, but have you tried to follow recomendation to put module folder under a "lib" directory in add-on root (where manifest.ini is located), instead of under globalPlugins?
It could avoid this problem, maybe...
Alberto


Andy B.
 

Hi,

 

I was able to import html.parser. Unfortunately, I receive an error: import error: no such module _markupbase. Further exploration of the topic concludes that _markupbase is a CPython module in the standard library. Unfortunately, NVDA also removed it as part of the size requirement issue. Thus, it appears that anything using markup of any sort will fail. Lxml, html.parser, bs4, cssutils, tinycss, etc. Any ideas how to get _markupbase in my addon?

 

 

Sent from Mail for Windows 10

 

From: Alberto Buffolino
Sent: Tuesday, June 9, 2020 11:44 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Andy B., il 09/06/2020 17.29, ha scritto:

> When I attempt to import a second

> copy of html library that contains parser, NVDA will either load my copy

> or its own copy, but not both.

Alberto:

Hi Andy,

I'm not sure, but have you tried to follow recomendation to put module

folder under a "lib" directory in add-on root (where manifest.ini is

located), instead of under globalPlugins?

It could avoid this problem, maybe...

Alberto

 

 

 


James Scholes
 

First, as others have suggested, you should be inserting any new paths at the beginning of sys.path, not appending them. Otherwise, whatever is built into NVDA will get priority.

You can copy _markupbase.py from any Python 3.x installation; it's located inside the lib folder.

Regards,

James Scholes

On 09/06/2020 at 10:37 am, Andy B. wrote:
At this point, ideal isn’t an option, and I am ready to accept that fact. There isn’t a problem with importing html/xml into my addon. However, attempting to import html.parser violates the relative imports rule set in Python 3, or least I believe it is a relative imports problem. When I copy the html library found in python3 install path/lib into my addon, then include the following code at the top of the __init__.py file, I get an error in the log saying ‘import error: no module by the name html.parser found’.
Sample code:
import os
import sys
addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder path
sys.path.append(addonNamePath)
import html.parser
del sys.path[-1]
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
*From: *James Scholes <mailto:james@...>
*Sent: *Tuesday, June 9, 2020 11:19 AM
*To: *nvda-addons@nvda-addons.groups.io <mailto:nvda-addons@nvda-addons.groups.io>
*Subject: *Re: [nvda-addons] Python dependencies in add-ons

> Using NVDA’s version of html and xml breaks the imported libraries
that require html.parser or xml.parser. On the other hand, using the
original Python provided versions breaks NVDA.
Breaks NVDA in what way?  If you include a full version of html and xml
in your add-on, modify sys.path appropriately, import bs4 or whatever
and then remove your path modifications, I don't understand why there
would be a conflict?  I understand that the setup is not ideal, but not
why it would break NVDA.
Regards,
James Scholes
On 08/06/2020 at 1:41 pm, Andy B. wrote:

> Hi,

>

> This generally works. However, it has one major problem that NVDA won’t

> allow add-on authors to overcome. If a dependency requires

> modules/libraries that NVDA removed from the standard Python library,

> all of the included Python libraries that attempt to import such removed

> libraries will fail. For example, trying to import bs4, lxml, or

> cssutils will terminally fail because NVDA removed html.parser and

> xml.parser from the standard library while maintaining the rest of the

> html and xml modules. Attempting to reimport the original html or xml

> standard Python libraries into an add-on also fails because it will

> cause a namespace conflict with the NVDA provided html and xml

> libraries. As a result, either one can exist, but not both. I attempted

> this and was forced to choose from a few different options:

>

>   * Use NVDA’s version of html and xml with parsers removed.

>   * Use my version of html or xml and ignore NVDA’s version.

>   * Rename the original html or xml standard Python libraries and manage

>     3^rd party dependencies myself.

>   * Give up and consider it a lost cause.

>

> I took the last option: give up because it is a lost cause. Using NVDA’s

> version of html and xml breaks the imported libraries that require

> html.parser or xml.parser. On the other hand, using the original Python

> provided versions breaks NVDA. Renaming the originals to something

> different seems like a ton of unwanted work because I would have to

> trudge through all the bs4/cssutils/lxml code and rename html.parser to

> something else, and after all of that, we can’t be sure it will work. If

> someone has a fix or workaround for this problem, I am all ears. It

> would bring a whole new level to my add-on.

>

> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for

> Windows 10

>

> *From: *Sean <mailto:s.tolstoyevski@...>

> *Sent: *Monday, June 8, 2020 1:40 PM

> *To: *nvda-addons@nvda-addons.groups.io

> <mailto:nvda-addons@nvda-addons.groups.io>

> *Subject: *Re: [nvda-addons] Python dependencies in add-ons

>

> example code:

>

> addonNamePathxX = os.path.abspath(os.path.dirname(__file__)) ^ #addon

> folder path

> sys.path.append(addonNamePathX)

>

> import numpy

>

> import opencv #example modules, Not available in NVDA.

>

> del sys.path[-1]

>

> ......

>

> #content code

>

> On 08/06/2020 20:33, Sean via groups.io wrote:

>

>     Hi  Subham,

>

>     I think your work will be a revolutionary thing.

>     Because screen readers have been working with the same logic for 20

>     years.

>     Good luck.

>

>     There is no separate tool to manage the dependencies of NVDA addons.

>

>     In addon's __init__.py file, the path of the modules is added with

>     sys.path.append.

>     Addon  writers often use this.

>

>     One thing I want to add is;

>     All required modules should be in the same folder.

>

>     You can write C ++ codes to Python with a tool like Cython.

>     Or the code section written in C ++ can be compiled as a DLL.

>     In this way, we can use the functions of the DLL with Ctypes.

>     Ctypes is more commonly used in NVDA source.

>

>     On 08/06/2020 14:19, Shubham Jain wrote:

>

>         Hello!

>

>         As part of my GSoC project

>
<https://summerofcode.withgoogle.com/projects/#6039693356957696>, I

>         am writing an add-on that allows users to get descriptions of

>         images. To work, the ML models depend on some python libraries

>         like Numpy, Pillow, onnxruntime and OpenCv. My questions are:

>

>           * Is it possible to package these libraries in an add-on?

>           * Since I only require a few specific functions from these

>             libraries, is it possible to only package those parts into

>             the add-on?

>

>

>         Alternatively, the models could be converted to run using native

>         C++ by depending on the LibTorch library. Is it possible to

>         write add-ons in native C++?

>

>         *regards,*

>         *Shubham Jain*

>

>     --

>

>

>         Sean

>

>       * Email: seantolstoyevski@...

>         <mailto:seantolstoyevski@...>

>       * GitHub: SeanTolstoyevski <https://github.com/SeanTolstoyevski/>

>

>     👨‍🦯 I’m student and programmer. I write often Python, sometimes Go

>     and rarely C++.

>

> --

>

>

>     Sean

>

>   * Email: seantolstoyevski@...

>     <mailto:seantolstoyevski@...>

>   * GitHub: SeanTolstoyevski <https://github.com/SeanTolstoyevski/>

>

> 👨‍🦯I’m student and programmer. I write often Python, sometimes Go and

> rarely C++.

>

>


Andy B.
 

Hi,

 

After sorting out the problem, I came up with the following that works to this point.

 

  • Grab the addon’s path from addon.path property.
  • Append ‘\lib’ to the addon.path property.
  • insert the path above into the first position sys.path property
  • Make any needed imports such as html.parser, bs4, cssutils, etc.
  • Remove the sys.path[0] entry.
  • Continue coding as normal.

 

The added benefit is the ability to start an NVDA Python terminal and still have access to the custom 3rd party libraries. I could still access bs4 and html.parser with import statements, even when the path is removed. Will do more testing later tonight/tomorrow though.

 

Sent from Mail for Windows 10

 

From: James Scholes
Sent: Tuesday, June 9, 2020 1:45 PM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

First, as others have suggested, you should be inserting any new paths

at the beginning of sys.path, not appending them.  Otherwise, whatever

is built into NVDA will get priority.

 

You can copy _markupbase.py from any Python 3.x installation; it's

located inside the lib folder.

 

Regards,

 

James Scholes

 

On 09/06/2020 at 10:37 am, Andy B. wrote:

> At this point, ideal isn’t an option, and I am ready to accept that

> fact. There isn’t a problem with importing html/xml into my addon.

> However, attempting to import html.parser violates the relative imports

> rule set in Python 3, or least I believe it is a relative imports

> problem. When I copy the html library found in python3 install path/lib

> into my addon, then include the following code at the top of the

> __init__.py file, I get an error in the log saying ‘import error: no

> module by the name html.parser found’.

>

> Sample code:

>

> import os

>

> import sys

>

> addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder

> path

>

> sys.path.append(addonNamePath)

>

> import html.parser

>

> del sys.path[-1]

>

> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for

> Windows 10

>

> *From: *James Scholes <mailto:james@...>

> *Sent: *Tuesday, June 9, 2020 11:19 AM

> *To: *nvda-addons@nvda-addons.groups.io

> <mailto:nvda-addons@nvda-addons.groups.io>

> *Subject: *Re: [nvda-addons] Python dependencies in add-ons

>

>  > Using NVDA’s version of html and xml breaks the imported libraries

>

> that require html.parser or xml.parser. On the other hand, using the

>

> original Python provided versions breaks NVDA.

>

> Breaks NVDA in what way?  If you include a full version of html and xml

>

> in your add-on, modify sys.path appropriately, import bs4 or whatever

>

> and then remove your path modifications, I don't understand why there

>

> would be a conflict?  I understand that the setup is not ideal, but not

>

> why it would break NVDA.

>

> Regards,

>

> James Scholes

>

> On 08/06/2020 at 1:41 pm, Andy B. wrote:

>

>  > Hi,

>

>  >

>

>  > This generally works. However, it has one major problem that NVDA won’t

>

>  > allow add-on authors to overcome. If a dependency requires

>

>  > modules/libraries that NVDA removed from the standard Python library,

>

>  > all of the included Python libraries that attempt to import such removed

>

>  > libraries will fail. For example, trying to import bs4, lxml, or

>

>  > cssutils will terminally fail because NVDA removed html.parser and

>

>  > xml.parser from the standard library while maintaining the rest of the

>

>  > html and xml modules. Attempting to reimport the original html or xml

>

>  > standard Python libraries into an add-on also fails because it will

>

>  > cause a namespace conflict with the NVDA provided html and xml

>

>  > libraries. As a result, either one can exist, but not both. I attempted

>

>  > this and was forced to choose from a few different options:

>

>  >

>

>  >   * Use NVDA’s version of html and xml with parsers removed.

>

>  >   * Use my version of html or xml and ignore NVDA’s version.

>

>  >   * Rename the original html or xml standard Python libraries and manage

>

>  >     3^rd party dependencies myself.

>

>  >   * Give up and consider it a lost cause.

>

>  >

>

>  > I took the last option: give up because it is a lost cause. Using NVDA’s

>

>  > version of html and xml breaks the imported libraries that require

>

>  > html.parser or xml.parser. On the other hand, using the original Python

>

>  > provided versions breaks NVDA. Renaming the originals to something

>

>  > different seems like a ton of unwanted work because I would have to

>

>  > trudge through all the bs4/cssutils/lxml code and rename html.parser to

>

>  > something else, and after all of that, we can’t be sure it will work. If

>

>  > someone has a fix or workaround for this problem, I am all ears. It

>

>  > would bring a whole new level to my add-on.

>

>  >

>

>  > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for

>

>  > Windows 10

>

>  >

>

>  > *From: *Sean <mailto:s.tolstoyevski@...>

>

>  > *Sent: *Monday, June 8, 2020 1:40 PM

>

>  > *To: *nvda-addons@nvda-addons.groups.io

>

>  > <mailto:nvda-addons@nvda-addons.groups.io>

>

>  > *Subject: *Re: [nvda-addons] Python dependencies in add-ons

>

>  >

>

>  > example code:

>

>  >

>

>  > addonNamePathxX = os.path.abspath(os.path.dirname(__file__)) ^ #addon

>

>  > folder path

>

>  > sys.path.append(addonNamePathX)

>

>  >

>

>  > import numpy

>

>  >

>

>  > import opencv #example modules, Not available in NVDA.

>

>  >

>

>  > del sys.path[-1]

>

>  >

>

>  > ......

>

>  >

>

>  > #content code

>

>  >

>

>  > On 08/06/2020 20:33, Sean via groups.io wrote:

>

>  >

>

>  >     Hi  Subham,

>

>  >

>

>  >     I think your work will be a revolutionary thing.

>

>  >     Because screen readers have been working with the same logic for 20

>

>  >     years.

>

>  >     Good luck.

>

>  >

>

>  >     There is no separate tool to manage the dependencies of NVDA addons.

>

>  >

>

>  >     In addon's __init__.py file, the path of the modules is added with

>

>  >     sys.path.append.

>

>  >     Addon  writers often use this.

>

>  >

>

>  >     One thing I want to add is;

>

>  >     All required modules should be in the same folder.

>

>  >

>

>  >     You can write C ++ codes to Python with a tool like Cython.

>

>  >     Or the code section written in C ++ can be compiled as a DLL.

>

>  >     In this way, we can use the functions of the DLL with Ctypes.

>

>  >     Ctypes is more commonly used in NVDA source.

>

>  >

>

>  >     On 08/06/2020 14:19, Shubham Jain wrote:

>

>  >

>

>  >         Hello!

>

>  >

>

>  >         As part of my GSoC project

>

>  >        

> <https://summerofcode.withgoogle.com/projects/#6039693356957696>, I

>

>  >         am writing an add-on that allows users to get descriptions of

>

>  >         images. To work, the ML models depend on some python libraries

>

>  >         like Numpy, Pillow, onnxruntime and OpenCv. My questions are:

>

>  >

>

>  >           * Is it possible to package these libraries in an add-on?

>

>  >           * Since I only require a few specific functions from these

>

>  >             libraries, is it possible to only package those parts into

>

>  >             the add-on?

>

>  >

>

>  >

>

>  >         Alternatively, the models could be converted to run using native

>

>  >         C++ by depending on the LibTorch library. Is it possible to

>

>  >         write add-ons in native C++?

>

>  >

>

>  >         *regards,*

>

>  >         *Shubham Jain*

>

>  >

>

>  >     --

>

>  >

>

>  >

>

>  >         Sean

>

>  >

>

>  >       * Email: seantolstoyevski@...

>

>  >         <mailto:seantolstoyevski@...>

>

>  >       * GitHub: SeanTolstoyevski <https://github.com/SeanTolstoyevski/>

>

>  >

>

>  >     👨‍🦯 I’m student and programmer. I write often Python, sometimes Go

>

>  >     and rarely C++.

>

>  >

>

>  > --

>

>  >

>

>  >

>

>  >     Sean

>

>  >

>

>  >   * Email: seantolstoyevski@...

>

>  >     <mailto:seantolstoyevski@...>

>

>  >   * GitHub: SeanTolstoyevski <https://github.com/SeanTolstoyevski/>

>

>  >

>

>  > 👨‍🦯I’m student and programmer. I write often Python, sometimes Go and

>

>  > rarely C++.

>

>  >

>

>  >

>

>