Topics

Python dependencies in add-ons

Shubham Jain
 

Hello!

As part of my GSoC project, I am writing an add-on that allows users to get descriptions of images. To work, the ML models depend on some python libraries like Numpy, Pillow, onnxruntime and OpenCv. My questions are:
  • Is it possible to package these libraries in an add-on?
  • Since I only require a few specific functions from these libraries, is it possible to only package those parts into the add-on?

Alternatively, the models could be converted to run using native C++ by depending on the LibTorch library. Is it possible to write add-ons in native C++?

regards,
Shubham Jain

Christopher Pross
 

Hey,

I had the same question for my add-on regarding the webcam face-guding add-on. It uses deep-learned open-cv networks from open-cv. you need for open-cv numpy. So I had solved this.

But only with building myself open-cv with an python 32-bit version, which is also conpatible with nvda and with getting a pre-binary from numpy, unpack it and copy the numpy folder in the folder were the addon is. Then you have to at temporaly add the root folder of the add-on to the python path with sys.path.
The self-builded version of open-cv contained a .pyd file, together with an python loader from open-cv you can load easyly open-cv from nvda-addons.

I hope yo could follow me.


all the best,

christopher

Am 08.06.2020 um 13:19 schrieb Shubham Jain:

Hello!

As part of my GSoC project, I am writing an add-on that allows users to get descriptions of images. To work, the ML models depend on some python libraries like Numpy, Pillow, onnxruntime and OpenCv. My questions are:
  • Is it possible to package these libraries in an add-on?
  • Since I only require a few specific functions from these libraries, is it possible to only package those parts into the add-on?

Alternatively, the models could be converted to run using native C++ by depending on the LibTorch library. Is it possible to write add-ons in native C++?

regards,
Shubham Jain

Noelia Ruiz
 

Hi, for the question about c++, as far as I know this is not possible.
You need at least certain Python files to store classes, as described
in the developer guide.
I'm very interested on your add-on. Hope you can share it with us and,
if you want, send it here for review so we can include it in the
add-ons website according with the future add-ons workflow when NV
Access can work on it, as planned by them.

Thanks for this project.

2020-06-08 17:06 GMT+02:00, Christopher Pross <chpross42@...>:

Hey,

I had the same question for my add-on regarding the webcam face-guding
add-on. It uses deep-learned open-cv networks from open-cv. you need for
open-cv numpy. So I had solved this.

But only with building myself open-cv with an python 32-bit version,
which is also conpatible with nvda and with getting a pre-binary from
numpy, unpack it and copy the numpy folder in the folder were the addon
is. Then you have to at temporaly add the root folder of the add-on to
the python path with sys.path.
The self-builded version of open-cv contained a .pyd file, together with
an python loader from open-cv you can load easyly open-cv from nvda-addons.

I hope yo could follow me.


all the best,

christopher

Am 08.06.2020 um 13:19 schrieb Shubham Jain:
Hello!

As part of my GSoC project
<https://summerofcode.withgoogle.com/projects/#6039693356957696>, I am
writing an add-on that allows users to get descriptions of images. To
work, the ML models depend on some python libraries like Numpy,
Pillow, onnxruntime and OpenCv. My questions are:

* Is it possible to package these libraries in an add-on?
* Since I only require a few specific functions from these
libraries, is it possible to only package those parts into the
add-on?


Alternatively, the models could be converted to run using native C++
by depending on the LibTorch library. Is it possible to write add-ons
in native C++?

*regards,*
*Shubham Jain*


Shubham Jain
 

Hello Christopher!

Thank you for your reply. I did not understand your solution completely but I believe I will when I try it. It would be incredibly helpful if you could share the code of your add-on (which, by the way, is a very good idea!). I hope it is okay to message you privately if I run into some issues while trying your solution.

regards,
Shubham Jain

Shubham Jain
 

Hello Noelia,

Perhaps it would be possible to run the C++ code from the Python files necessary to make the add-on work.

Thank you for your interest in my project! I definitely plan on sharing it with the community. Your feedback and review are invaluable to this project and to me as a first-time open-source developer. More importantly, I wish that this project is useful and comfortable to it's users and it would be very difficult to achieve those goals without inputs from the community.

reagrds,
Shubham Jain

Sean
 

Hi  Subham,

I think your work will be a revolutionary thing.
Because screen readers have been working with the same logic for 20 years.
Good luck.

There is no separate tool to manage the dependencies of NVDA addons.

In addon's __init__.py file, the path of the modules is added with sys.path.append.
Addon  writers often use this.

One thing I want to add is;
All required modules should be in the same folder.

You can write C ++ codes to Python with a tool like Cython.
Or the code section written in C ++ can be compiled as a DLL.
In this way, we can use the functions of the DLL with Ctypes.
Ctypes is more commonly used in NVDA source.

On 08/06/2020 14:19, Shubham Jain wrote:
Hello!

As part of my GSoC project, I am writing an add-on that allows users to get descriptions of images. To work, the ML models depend on some python libraries like Numpy, Pillow, onnxruntime and OpenCv. My questions are:
  • Is it possible to package these libraries in an add-on?
  • Since I only require a few specific functions from these libraries, is it possible to only package those parts into the add-on?

Alternatively, the models could be converted to run using native C++ by depending on the LibTorch library. Is it possible to write add-ons in native C++?

regards,
Shubham Jain
--

Sean

👨‍🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

Sean
 

example code:

addonNamePathxX = os.path.abspath(os.path.dirname(__file__)) ^ #addon folder path
sys.path.append(addonNamePathX)

import numpy

import opencv #example modules, Not available in NVDA.

del sys.path[-1]


.......

#content code


On 08/06/2020 20:33, Sean via groups.io wrote:

Hi  Subham,

I think your work will be a revolutionary thing.
Because screen readers have been working with the same logic for 20 years.
Good luck.

There is no separate tool to manage the dependencies of NVDA addons.

In addon's __init__.py file, the path of the modules is added with sys.path.append.
Addon  writers often use this.

One thing I want to add is;
All required modules should be in the same folder.

You can write C ++ codes to Python with a tool like Cython.
Or the code section written in C ++ can be compiled as a DLL.
In this way, we can use the functions of the DLL with Ctypes.
Ctypes is more commonly used in NVDA source.

On 08/06/2020 14:19, Shubham Jain wrote:
Hello!

As part of my GSoC project, I am writing an add-on that allows users to get descriptions of images. To work, the ML models depend on some python libraries like Numpy, Pillow, onnxruntime and OpenCv. My questions are:
  • Is it possible to package these libraries in an add-on?
  • Since I only require a few specific functions from these libraries, is it possible to only package those parts into the add-on?

Alternatively, the models could be converted to run using native C++ by depending on the LibTorch library. Is it possible to write add-ons in native C++?

regards,
Shubham Jain
--

Sean

👨‍🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

--

Sean

👨‍🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

Christopher Pross
 

Hey,

of curse you can write me private, if you want.

And in a few days I am finishing my add-on and you could see it on github.


all the best,

Christopher


Am 08.06.2020 um 19:24 schrieb Shubham Jain:

Hello Noelia,

Perhaps it would be possible to run the C++ code from the Python files necessary to make the add-on work.

Thank you for your interest in my project! I definitely plan on sharing it with the community. Your feedback and review are invaluable to this project and to me as a first-time open-source developer. More importantly, I wish that this project is useful and comfortable to it's users and it would be very difficult to achieve those goals without inputs from the community.

reagrds,
Shubham Jain

Shubham Jain
 

Hi Sean!

Your words are very uplifting, thank you very much!
I think compiling the code to a windows DLL is a great idea! I do not know much about Cython so I will look into that as an option.
Thank you for taking the time to share an actual code example, it is greatly appreciated.

regards,
Shubham Jain

Shubham Jain
 

Hi Christopher,

Thank you. Looking forward to your add-on!

regards,
Shubham Jain

Andy B.
 

Hi,

 

This generally works. However, it has one major problem that NVDA won’t allow add-on authors to overcome. If a dependency requires modules/libraries that NVDA removed from the standard Python library, all of the included Python libraries that attempt to import such removed libraries will fail. For example, trying to import bs4, lxml, or cssutils will terminally fail because NVDA removed html.parser and xml.parser from the standard library while maintaining the rest of the html and xml modules. Attempting to reimport the original html or xml standard Python libraries into an add-on also fails because it will cause a namespace conflict with the NVDA provided html and xml libraries. As a result, either one can exist, but not both. I attempted this and was forced to choose from a few different options:

 

  • Use NVDA’s version of html and xml with parsers removed.
  • Use my version of html or xml and ignore NVDA’s version.
  • Rename the original html or xml standard Python libraries and manage 3rd party dependencies myself.
  • Give up and consider it a lost cause.

I took the last option: give up because it is a lost cause. Using NVDA’s version of html and xml breaks the imported libraries that require html.parser or xml.parser. On the other hand, using the original Python provided versions breaks NVDA. Renaming the originals to something different seems like a ton of unwanted work because I would have to trudge through all the bs4/cssutils/lxml code and rename html.parser to something else, and after all of that, we can’t be sure it will work. If someone has a fix or workaround for this problem, I am all ears. It would bring a whole new level to my add-on.

 

 

Sent from Mail for Windows 10

 

From: Sean
Sent: Monday, June 8, 2020 1:40 PM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

example code:

addonNamePathxX = os.path.abspath(os.path.dirname(__file__)) ^ #addon folder path
sys.path.append(addonNamePathX)

import numpy

import opencv #example modules, Not available in NVDA.

del sys.path[-1]

 

......

#content code

 

On 08/06/2020 20:33, Sean via groups.io wrote:

Hi  Subham,

I think your work will be a revolutionary thing.
Because screen readers have been working with the same logic for 20 years.
Good luck.

There is no separate tool to manage the dependencies of NVDA addons.

In addon's __init__.py file, the path of the modules is added with sys.path.append.
Addon  writers often use this.

One thing I want to add is;
All required modules should be in the same folder.

You can write C ++ codes to Python with a tool like Cython.
Or the code section written in C ++ can be compiled as a DLL.
In this way, we can use the functions of the DLL with Ctypes.
Ctypes is more commonly used in NVDA source.

On 08/06/2020 14:19, Shubham Jain wrote:

Hello!

As part of my GSoC project, I am writing an add-on that allows users to get descriptions of images. To work, the ML models depend on some python libraries like Numpy, Pillow, onnxruntime and OpenCv. My questions are:

  • Is it possible to package these libraries in an add-on?
  • Since I only require a few specific functions from these libraries, is it possible to only package those parts into the add-on?


Alternatively, the models could be converted to run using native C++ by depending on the LibTorch library. Is it possible to write add-ons in native C++?

regards,
Shubham Jain

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

Sean
 

Hi,
I know that there are not some modules that Python developers use a lot among the modules that NVDA packages. (see. library.zip)

For example, I was trying to write a code sharing addon.
I thought urllib3 was the best option for me.
But urllib3 wanted a module called hmac.
hmac is one of Python's main modules.

If I have such a problem, I copy the .py file if the module consists of a single file from pythonPath\lib.
Example of these modules: hmac.py, tempfile.py

Then I create a folder for these modules.
Example: "hmac" folder for Hmac.
Then I change the name of the hmac.py file  module to "__init__.py".
hmac\__init__.py

I add the path of the modules to Python with sys.path.append.

If one of the dependencies is a module consisting of more than one file, it is enough to copy that module folder.
Example: sqlite3

Although the processes seem very long, once completed, most of the problems are solved.

Finally, some main modules may need .pyd files. Sqlite3 is an example.
I have never encountered this type of error.
I do not know that NVDA can correctly detect .pyd files in this method.
What I wrote can be very complicated. I'm trying to  enhance my English :) .
If you still have an unresolved point in your mind, I can share a sample zip file.

On 08/06/2020 21:41, Andy B. wrote:

Hi,

 

This generally works. However, it has one major problem that NVDA won’t allow add-on authors to overcome. If a dependency requires modules/libraries that NVDA removed from the standard Python library, all of the included Python libraries that attempt to import such removed libraries will fail. For example, trying to import bs4, lxml, or cssutils will terminally fail because NVDA removed html.parser and xml.parser from the standard library while maintaining the rest of the html and xml modules. Attempting to reimport the original html or xml standard Python libraries into an add-on also fails because it will cause a namespace conflict with the NVDA provided html and xml libraries. As a result, either one can exist, but not both. I attempted this and was forced to choose from a few different options:

 

  • Use NVDA’s version of html and xml with parsers removed.
  • Use my version of html or xml and ignore NVDA’s version.
  • Rename the original html or xml standard Python libraries and manage 3rd party dependencies myself.
  • Give up and consider it a lost cause.

I took the last option: give up because it is a lost cause. Using NVDA’s version of html and xml breaks the imported libraries that require html.parser or xml.parser. On the other hand, using the original Python provided versions breaks NVDA. Renaming the originals to something different seems like a ton of unwanted work because I would have to trudge through all the bs4/cssutils/lxml code and rename html.parser to something else, and after all of that, we can’t be sure it will work. If someone has a fix or workaround for this problem, I am all ears. It would bring a whole new level to my add-on.

 

 

Sent from Mail for Windows 10

 

From: Sean
Sent: Monday, June 8, 2020 1:40 PM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

example code:

addonNamePathxX = os.path.abspath(os.path.dirname(__file__)) ^ #addon folder path
sys.path.append(addonNamePathX)

import numpy

import opencv #example modules, Not available in NVDA.

del sys.path[-1]

 

......

#content code

 

On 08/06/2020 20:33, Sean via groups.io wrote:

Hi  Subham,

I think your work will be a revolutionary thing.
Because screen readers have been working with the same logic for 20 years.
Good luck.

There is no separate tool to manage the dependencies of NVDA addons.

In addon's __init__.py file, the path of the modules is added with sys.path.append.
Addon  writers often use this.

One thing I want to add is;
All required modules should be in the same folder.

You can write C ++ codes to Python with a tool like Cython.
Or the code section written in C ++ can be compiled as a DLL.
In this way, we can use the functions of the DLL with Ctypes.
Ctypes is more commonly used in NVDA source.

On 08/06/2020 14:19, Shubham Jain wrote:

Hello!

As part of my GSoC project, I am writing an add-on that allows users to get descriptions of images. To work, the ML models depend on some python libraries like Numpy, Pillow, onnxruntime and OpenCv. My questions are:

  • Is it possible to package these libraries in an add-on?
  • Since I only require a few specific functions from these libraries, is it possible to only package those parts into the add-on?


Alternatively, the models could be converted to run using native C++ by depending on the LibTorch library. Is it possible to write add-ons in native C++?

regards,
Shubham Jain

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

--

Sean

👨‍🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

Andy B.
 

A sample zip file would be great.



On Jun 8, 2020, at 9:17 PM, Sean <s.tolstoyevski@...> wrote:



Hi,
I know that there are not some modules that Python developers use a lot among the modules that NVDA packages. (see. library.zip)

For example, I was trying to write a code sharing addon.
I thought urllib3 was the best option for me.
But urllib3 wanted a module called hmac.
hmac is one of Python's main modules.

If I have such a problem, I copy the .py file if the module consists of a single file from pythonPath\lib.
Example of these modules: hmac.py, tempfile.py

Then I create a folder for these modules.
Example: "hmac" folder for Hmac.
Then I change the name of the hmac.py file  module to "__init__.py".
hmac\__init__.py

I add the path of the modules to Python with sys.path.append.

If one of the dependencies is a module consisting of more than one file, it is enough to copy that module folder.
Example: sqlite3

Although the processes seem very long, once completed, most of the problems are solved.

Finally, some main modules may need .pyd files. Sqlite3 is an example.
I have never encountered this type of error.
I do not know that NVDA can correctly detect .pyd files in this method.
What I wrote can be very complicated. I'm trying to  enhance my English :) .
If you still have an unresolved point in your mind, I can share a sample zip file.

On 08/06/2020 21:41, Andy B. wrote:

Hi,

 

This generally works. However, it has one major problem that NVDA won’t allow add-on authors to overcome. If a dependency requires modules/libraries that NVDA removed from the standard Python library, all of the included Python libraries that attempt to import such removed libraries will fail. For example, trying to import bs4, lxml, or cssutils will terminally fail because NVDA removed html.parser and xml.parser from the standard library while maintaining the rest of the html and xml modules. Attempting to reimport the original html or xml standard Python libraries into an add-on also fails because it will cause a namespace conflict with the NVDA provided html and xml libraries. As a result, either one can exist, but not both. I attempted this and was forced to choose from a few different options:

 

  • Use NVDA’s version of html and xml with parsers removed.
  • Use my version of html or xml and ignore NVDA’s version.
  • Rename the original html or xml standard Python libraries and manage 3rd party dependencies myself.
  • Give up and consider it a lost cause.

I took the last option: give up because it is a lost cause. Using NVDA’s version of html and xml breaks the imported libraries that require html.parser or xml.parser. On the other hand, using the original Python provided versions breaks NVDA. Renaming the originals to something different seems like a ton of unwanted work because I would have to trudge through all the bs4/cssutils/lxml code and rename html.parser to something else, and after all of that, we can’t be sure it will work. If someone has a fix or workaround for this problem, I am all ears. It would bring a whole new level to my add-on.

 

 

Sent from Mail for Windows 10

 

From: Sean
Sent: Monday, June 8, 2020 1:40 PM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

example code:

addonNamePathxX = os.path.abspath(os.path.dirname(__file__)) ^ #addon folder path
sys.path.append(addonNamePathX)

import numpy

import opencv #example modules, Not available in NVDA.

del sys.path[-1]

 

......

#content code

 

On 08/06/2020 20:33, Sean via groups.io wrote:

Hi  Subham,

I think your work will be a revolutionary thing.
Because screen readers have been working with the same logic for 20 years.
Good luck.

There is no separate tool to manage the dependencies of NVDA addons.

In addon's __init__.py file, the path of the modules is added with sys.path.append.
Addon  writers often use this.

One thing I want to add is;
All required modules should be in the same folder.

You can write C ++ codes to Python with a tool like Cython.
Or the code section written in C ++ can be compiled as a DLL.
In this way, we can use the functions of the DLL with Ctypes.
Ctypes is more commonly used in NVDA source.

On 08/06/2020 14:19, Shubham Jain wrote:

Hello!

As part of my GSoC project, I am writing an add-on that allows users to get descriptions of images. To work, the ML models depend on some python libraries like Numpy, Pillow, onnxruntime and OpenCv. My questions are:

  • Is it possible to package these libraries in an add-on?
  • Since I only require a few specific functions from these libraries, is it possible to only package those parts into the add-on?


Alternatively, the models could be converted to run using native C++ by depending on the LibTorch library. Is it possible to write add-ons in native C++?

regards,
Shubham Jain

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

--

Sean

👨‍🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

Shubham Jain
 

Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain

Sean
 

I know that the NVDA team wants to keep the size of NVDA to a minimum.
This is important for use as a portable.

I think the pure python code doesn't take up much size.
The .pyd files of this type of modules are very size large.
If there are problems with file sizes, this can be reduced with UPX.
After all, 20-30 MB of RAM is used more, but it is an efficient method.

I wish you good coding...
As the work emerges, great ideas will continue to come from here.

On 09/06/2020 11:28, Shubham Jain wrote:
Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain
--

Sean

👨‍🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

Well just remember guys that while this may not become that important in say another few years, nvda is used on a lot of systems.

Not everyone has the latest and greatest and not everyone can have the latest and greatest of systems depending where they are especially now.

I know quite a few with win7 and yes xp.

While nvda latest doesn't matter in this reguard, even if we take in to mind that at minimum your system will have 4gb of ram, 30mb could be a lot to someone.

And even if this is excluded, not everyone has an ssd.

And even if that is excluded also not everyone will have the best cpu.

I know at least a few using 3rd generation intel systems and older amd systems.

I myself maintain a 3700, a i530, a 7200, and a 4500 and I know that in the second hand systems part of my tech store they are still trading crappy 2nd generation systems.

Even though a laptop of reasonable design costs between 800 and 1000 bucks easily, some can't afford it and will use older tech.

They will use it till it dies.

And even if that is not a problem not everyone even in my country has fibre.

Some have wireless, lite fiber, some still use dialup.

So while thats not many 30mb may matter.

At first I was like just load it all in, but thats why the core package never has everything in it.

In fact nvda is designed for modules to be installed.

No one needs everyone of them, but everyone has a certain subset.

So for me I have enough for all the programs that use one of course, including the os, and a few extra utilities I like.

In the home network systems, I have enough for the systems to work as well as apps on those if I need but a lot less.

In the systems I administrate though which are not mine including family servers, the only modules I have are the modules I absolutely need.

Then again if you think extra stuff is needed then it is needed.



On 9/06/2020 9:30 pm, Sean wrote:

I know that the NVDA team wants to keep the size of NVDA to a minimum.
This is important for use as a portable.

I think the pure python code doesn't take up much size.
The .pyd files of this type of modules are very size large.
If there are problems with file sizes, this can be reduced with UPX.
After all, 20-30 MB of RAM is used more, but it is an efficient method.

I wish you good coding...
As the work emerges, great ideas will continue to come from here.

On 09/06/2020 11:28, Shubham Jain wrote:
Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain
--

Sean

👨‍🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

Andy B.
 

I did the following to see if this would work:

 

  • Copied the html library from python3/lib install path and put it in my addon install folder.
  • Added the lines below to the top of my addon __init__.py file:
  • import os
  • import sys
  • addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder path
  • sys.path.append(addonNamePath)
  • import html.parser
  • del sys.path[-1]

I either get module not found errors or syntax errors because relative imports aren’t allowed. Doing the same to the lxml library works as expected. However, it depends on html.parser which NVDA removed from the html library.

 

Sent from Mail for Windows 10

 

From: Shaun Everiss
Sent: Tuesday, June 9, 2020 5:53 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Well just remember guys that while this may not become that important in say another few years, nvda is used on a lot of systems.

Not everyone has the latest and greatest and not everyone can have the latest and greatest of systems depending where they are especially now.

I know quite a few with win7 and yes xp.

While nvda latest doesn't matter in this reguard, even if we take in to mind that at minimum your system will have 4gb of ram, 30mb could be a lot to someone.

And even if this is excluded, not everyone has an ssd.

And even if that is excluded also not everyone will have the best cpu.

I know at least a few using 3rd generation intel systems and older amd systems.

I myself maintain a 3700, a i530, a 7200, and a 4500 and I know that in the second hand systems part of my tech store they are still trading crappy 2nd generation systems.

Even though a laptop of reasonable design costs between 800 and 1000 bucks easily, some can't afford it and will use older tech.

They will use it till it dies.

And even if that is not a problem not everyone even in my country has fibre.

Some have wireless, lite fiber, some still use dialup.

So while thats not many 30mb may matter.

At first I was like just load it all in, but thats why the core package never has everything in it.

In fact nvda is designed for modules to be installed.

No one needs everyone of them, but everyone has a certain subset.

So for me I have enough for all the programs that use one of course, including the os, and a few extra utilities I like.

In the home network systems, I have enough for the systems to work as well as apps on those if I need but a lot less.

In the systems I administrate though which are not mine including family servers, the only modules I have are the modules I absolutely need.

Then again if you think extra stuff is needed then it is needed.

 

 

On 9/06/2020 9:30 pm, Sean wrote:

I know that the NVDA team wants to keep the size of NVDA to a minimum.
This is important for use as a portable.

I think the pure python code doesn't take up much size.
The .pyd files of this type of modules are very size large.
If there are problems with file sizes, this can be reduced with UPX.
After all, 20-30 MB of RAM is used more, but it is an efficient method.

I wish you good coding...
As the work emerges, great ideas will continue to come from here.

On 09/06/2020 11:28, Shubham Jain wrote:

Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

Cyrille
 

Hi Andy

Some thoughts, I did not test by myself however.

Maybe you should insert the path containing the html library at the beginning of the path and not at the end to ensure that this version of html library is fetched:
sys.path.insert(0, addonNamePath)
import html.parse
del sys.path[0]

Moreover, it seems that doing as you wrote, you get twice the add-on's path in sys.path. One time added by NVDA to import your add-on and a second time in the code of your add-on. I do not know if putting twice the addonpath in sys.path may generate side effect or difficulties in debug regarding import. An alternative solution is to put your html library in a "lib" subfolder and add/remove this subfolder's path.

Again, I have not tested all this by myself; it is just thoughts that came to my mind when I read your message. Hope it can help.

Cheers,

Cyrille



Le 09/06/2020 à 14:22, Andy B. a écrit :

I did the following to see if this would work:

 

  • Copied the html library from python3/lib install path and put it in my addon install folder.
  • Added the lines below to the top of my addon __init__.py file:
  • import os
  • import sys
  • addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder path
  • sys.path.append(addonNamePath)
  • import html.parser
  • del sys.path[-1]

I either get module not found errors or syntax errors because relative imports aren’t allowed. Doing the same to the lxml library works as expected. However, it depends on html.parser which NVDA removed from the html library.

 

Sent from Mail for Windows 10

 

From: Shaun Everiss
Sent: Tuesday, June 9, 2020 5:53 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Well just remember guys that while this may not become that important in say another few years, nvda is used on a lot of systems.

Not everyone has the latest and greatest and not everyone can have the latest and greatest of systems depending where they are especially now.

I know quite a few with win7 and yes xp.

While nvda latest doesn't matter in this reguard, even if we take in to mind that at minimum your system will have 4gb of ram, 30mb could be a lot to someone.

And even if this is excluded, not everyone has an ssd.

And even if that is excluded also not everyone will have the best cpu.

I know at least a few using 3rd generation intel systems and older amd systems.

I myself maintain a 3700, a i530, a 7200, and a 4500 and I know that in the second hand systems part of my tech store they are still trading crappy 2nd generation systems.

Even though a laptop of reasonable design costs between 800 and 1000 bucks easily, some can't afford it and will use older tech.

They will use it till it dies.

And even if that is not a problem not everyone even in my country has fibre.

Some have wireless, lite fiber, some still use dialup.

So while thats not many 30mb may matter.

At first I was like just load it all in, but thats why the core package never has everything in it.

In fact nvda is designed for modules to be installed.

No one needs everyone of them, but everyone has a certain subset.

So for me I have enough for all the programs that use one of course, including the os, and a few extra utilities I like.

In the home network systems, I have enough for the systems to work as well as apps on those if I need but a lot less.

In the systems I administrate though which are not mine including family servers, the only modules I have are the modules I absolutely need.

Then again if you think extra stuff is needed then it is needed.

 

 

On 9/06/2020 9:30 pm, Sean wrote:

I know that the NVDA team wants to keep the size of NVDA to a minimum.
This is important for use as a portable.

I think the pure python code doesn't take up much size.
The .pyd files of this type of modules are very size large.
If there are problems with file sizes, this can be reduced with UPX.
After all, 20-30 MB of RAM is used more, but it is an efficient method.

I wish you good coding...
As the work emerges, great ideas will continue to come from here.

On 09/06/2020 11:28, Shubham Jain wrote:

Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

Sean
 

This seems to be an interesting problem.

Just I think:
You might need to do something like "myhtml" in the name of the html module folder.
But this will destroy all requirements.

I think this problem is related to Python's module search tree structure.
And as far as I know, the folder you're in is the top priority search path.

On 09/06/2020 15:22, Andy B. wrote:

I did the following to see if this would work:

 

  • Copied the html library from python3/lib install path and put it in my addon install folder.
  • Added the lines below to the top of my addon __init__.py file:
  • import os
  • import sys
  • addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder path
  • sys.path.append(addonNamePath)
  • import html.parser
  • del sys.path[-1]

I either get module not found errors or syntax errors because relative imports aren’t allowed. Doing the same to the lxml library works as expected. However, it depends on html.parser which NVDA removed from the html library.

 

Sent from Mail for Windows 10

 

From: Shaun Everiss
Sent: Tuesday, June 9, 2020 5:53 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Well just remember guys that while this may not become that important in say another few years, nvda is used on a lot of systems.

Not everyone has the latest and greatest and not everyone can have the latest and greatest of systems depending where they are especially now.

I know quite a few with win7 and yes xp.

While nvda latest doesn't matter in this reguard, even if we take in to mind that at minimum your system will have 4gb of ram, 30mb could be a lot to someone.

And even if this is excluded, not everyone has an ssd.

And even if that is excluded also not everyone will have the best cpu.

I know at least a few using 3rd generation intel systems and older amd systems.

I myself maintain a 3700, a i530, a 7200, and a 4500 and I know that in the second hand systems part of my tech store they are still trading crappy 2nd generation systems.

Even though a laptop of reasonable design costs between 800 and 1000 bucks easily, some can't afford it and will use older tech.

They will use it till it dies.

And even if that is not a problem not everyone even in my country has fibre.

Some have wireless, lite fiber, some still use dialup.

So while thats not many 30mb may matter.

At first I was like just load it all in, but thats why the core package never has everything in it.

In fact nvda is designed for modules to be installed.

No one needs everyone of them, but everyone has a certain subset.

So for me I have enough for all the programs that use one of course, including the os, and a few extra utilities I like.

In the home network systems, I have enough for the systems to work as well as apps on those if I need but a lot less.

In the systems I administrate though which are not mine including family servers, the only modules I have are the modules I absolutely need.

Then again if you think extra stuff is needed then it is needed.

 

 

On 9/06/2020 9:30 pm, Sean wrote:

I know that the NVDA team wants to keep the size of NVDA to a minimum.
This is important for use as a portable.

I think the pure python code doesn't take up much size.
The .pyd files of this type of modules are very size large.
If there are problems with file sizes, this can be reduced with UPX.
After all, 20-30 MB of RAM is used more, but it is an efficient method.

I wish you good coding...
As the work emerges, great ideas will continue to come from here.

On 09/06/2020 11:28, Shubham Jain wrote:

Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

 

--

Sean

👨‍🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.

Rui Fontes
 

Sorry, but to import a module present in the same folder of the .py file, the syntax is not:

from . import modul


Rui Fontes


Às 14:05 de 09/06/2020, Cyrille via groups.io escreveu:

Hi Andy

Some thoughts, I did not test by myself however.

Maybe you should insert the path containing the html library at the beginning of the path and not at the end to ensure that this version of html library is fetched:
sys.path.insert(0, addonNamePath)
import html.parse
del sys.path[0]

Moreover, it seems that doing as you wrote, you get twice the add-on's path in sys.path. One time added by NVDA to import your add-on and a second time in the code of your add-on. I do not know if putting twice the addonpath in sys.path may generate side effect or difficulties in debug regarding import. An alternative solution is to put your html library in a "lib" subfolder and add/remove this subfolder's path.

Again, I have not tested all this by myself; it is just thoughts that came to my mind when I read your message. Hope it can help.

Cheers,

Cyrille



Le 09/06/2020 à 14:22, Andy B. a écrit :

I did the following to see if this would work:

 

  • Copied the html library from python3/lib install path and put it in my addon install folder.
  • Added the lines below to the top of my addon __init__.py file:
  • import os
  • import sys
  • addonNamePath = os.path.abspath(os.path.dirname(__file__)) #addon folder path
  • sys.path.append(addonNamePath)
  • import html.parser
  • del sys.path[-1]

I either get module not found errors or syntax errors because relative imports aren’t allowed. Doing the same to the lxml library works as expected. However, it depends on html.parser which NVDA removed from the html library.

 

Sent from Mail for Windows 10

 

From: Shaun Everiss
Sent: Tuesday, June 9, 2020 5:53 AM
To: nvda-addons@nvda-addons.groups.io
Subject: Re: [nvda-addons] Python dependencies in add-ons

 

Well just remember guys that while this may not become that important in say another few years, nvda is used on a lot of systems.

Not everyone has the latest and greatest and not everyone can have the latest and greatest of systems depending where they are especially now.

I know quite a few with win7 and yes xp.

While nvda latest doesn't matter in this reguard, even if we take in to mind that at minimum your system will have 4gb of ram, 30mb could be a lot to someone.

And even if this is excluded, not everyone has an ssd.

And even if that is excluded also not everyone will have the best cpu.

I know at least a few using 3rd generation intel systems and older amd systems.

I myself maintain a 3700, a i530, a 7200, and a 4500 and I know that in the second hand systems part of my tech store they are still trading crappy 2nd generation systems.

Even though a laptop of reasonable design costs between 800 and 1000 bucks easily, some can't afford it and will use older tech.

They will use it till it dies.

And even if that is not a problem not everyone even in my country has fibre.

Some have wireless, lite fiber, some still use dialup.

So while thats not many 30mb may matter.

At first I was like just load it all in, but thats why the core package never has everything in it.

In fact nvda is designed for modules to be installed.

No one needs everyone of them, but everyone has a certain subset.

So for me I have enough for all the programs that use one of course, including the os, and a few extra utilities I like.

In the home network systems, I have enough for the systems to work as well as apps on those if I need but a lot less.

In the systems I administrate though which are not mine including family servers, the only modules I have are the modules I absolutely need.

Then again if you think extra stuff is needed then it is needed.

 

 

On 9/06/2020 9:30 pm, Sean wrote:

I know that the NVDA team wants to keep the size of NVDA to a minimum.
This is important for use as a portable.

I think the pure python code doesn't take up much size.
The .pyd files of this type of modules are very size large.
If there are problems with file sizes, this can be reduced with UPX.
After all, 20-30 MB of RAM is used more, but it is an efficient method.

I wish you good coding...
As the work emerges, great ideas will continue to come from here.

On 09/06/2020 11:28, Shubham Jain wrote:

Hi Sean!

This seems like a fairly simple solution! The weights of the model already take up a lot of space (~150-200 Mb) and some of these libraries are ~50 Mb in size so I might need to get rid of all the unnecessary parts of the library to reduce space. Since I only require little functionality from these libraries, it would help with that too.

Your English is great and your explanation was very clear! Many thanks.

regards,
Shubham Jain

--

Sean

👨🦯 I’m student and programmer. I write often Python, sometimes Go and rarely C++.