X Tutup
The Wayback Machine - https://web.archive.org/web/20200916181137/https://github.com/nodegit/nodegit/issues/1774
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make nodegit context-aware for compatibility with Electron 9 and beyond #1774

Open
Livven opened this issue May 24, 2020 · 9 comments · May be fixed by #1795
Open

Make nodegit context-aware for compatibility with Electron 9 and beyond #1774

Livven opened this issue May 24, 2020 · 9 comments · May be fixed by #1795

Comments

@Livven
Copy link

@Livven Livven commented May 24, 2020

Electron 9 changed the default value of app.allowRendererProcessReuse to true, which breaks nodegit. Setting it to false manually fixes the issue, but this will only be possible until Electron 11.

Are there any plans to fix this? Seems like #1298 might be related. Also see electron/electron#18397 for more information.

System information

  • node version: v12.16.3
  • npm or yarn version: 6.14.4
  • OS/version/architecture: win32 10.0.18363 x64
  • Applicable nodegit version: 0.26.5
node -v
npm -v # (or yarn -v)
node -e "console.log(process.platform)"
node -e "console.log(require('os').release())"
node -e "console.log(console.log(process.arch))"
@oyed
Copy link

@oyed oyed commented Jun 6, 2020

This definitely needs to be looked in to, if not prioritized, for the future of nodegit in Electron environments given how fast Electron releases are rolling out. With RC for Electron 10 already out, it's only a matter of (short) time before 11 is the norm.

@maxkorp
Copy link
Collaborator

@maxkorp maxkorp commented Jun 9, 2020

Is there a preference for context-awareness over n-api compat?

@implausible
Copy link
Member

@implausible implausible commented Jun 9, 2020

Axosoft is aware that we need to update this shortly. We'll be putting 2 engineers on this after our next GitKraken release. We want to hit this before Electron 11 is in beta. GitKraken essentially needs this to happen no matter what, so it will be prioritized.

As for general context-awareness vs n-api compat, definitely prefer switching to n-api compat, that seems like a better long term. There's also a ton of utility in n-api that should simplify a lot of extra work we've had to do (thread safe callbacks, I'm looking at you).

Sit tight, it will arrive in time for Electron 11.

@Livven
Copy link
Author

@Livven Livven commented Jun 9, 2020

Awesome, good to hear that!

@maxkorp To be honest I have no idea what the difference and generally the context here is, I just used the terminology I saw in the warning messages :p

@implausible
Copy link
Member

@implausible implausible commented Jul 30, 2020

An update: I'm starting work on this officially. I am currently working on making sure that I'm not going to back myself into a corner by moving to N-API. There are some concerns I have with moving toward N-API that have to deal with the differences between ObjectWrap in N-API and in NAN and how we use them in NodeGit. In particular the inheritance model for https://github.com/nodegit/nodegit/blob/master/generate/templates/manual/include/nodegit_wrapper.h has me a bit worried about initializing classes using https://github.com/nodejs/node-addon-api/blob/master/doc/object_wrap.md. It almost seems incompatible to me because of the DefineClass macro. This is possibly my own shortcoming that I don't necessarily know how to make these compatible with each other...

The other concerning thing is to do with our use of an internal thread pool. We built a thread pool #1019 here to solve thread pool saturation when sharing with the standard node pool; if we moved to N-API i suppose that this would mean that users would need to consider increasing UV_THREADPOOL_SIZE to prevent saturation... It also could be resolved by having NodeGit itself run in a worker thread (a benefit of context-aware native modules)... but that would actually be a holistic nightmare for devs of GitKraken and I assume current users of nodegit for the amount of refactor necessary to move nodegit off of the main loop and onto a worker...

It seems like between some of those problems, Context Aware API is probably not only a shorter change, but less destructive downstream. I'm not a fan of NAN and raw v8, and NAPI is a lot cleaner imo, but we're not even going to get the benefits of ABI compatibility because of our reliance on libuv for a separate thread pool and the OpenSSL symbols that ship with Node.

All of this is to say, we're still going to be context aware by the time Electron 11 lands, but I think I'm going to change direction and try to update the library to be context-aware instead.

I think some of my major goals here are:

  • Fix async resources across async workers and callbacks
  • Ensure worker threads are 100% supported by the library.
@maxkorp
Copy link
Collaborator

@maxkorp maxkorp commented Jul 31, 2020

After doing some reading up, I think that's a fair evaluation and a good direction to head

@implausible
Copy link
Member

@implausible implausible commented Jul 31, 2020

Making some progress with modeling the threadpool updates. Ran into a snag with Node's cleanup callback. Looks to be solved here nodejs/node#34567. I'm going to continue development with the thread pool with the assumption that this will land downstream in Node sometime soon. If that's in betore I finish great! If not, then I will end up shutting off worker support until it does land.

@implausible
Copy link
Member

@implausible implausible commented Jul 31, 2020

Hm, this opens up the door for the next real issue, since a libgit2 execute thread can be active during shutdown, that libgit2 function could be trying to make callbacks back into javascript. In those cases, we need to be sending back the cancellation flag back to libgit2 on any callbacks that the user has defined. For callbacks that don't have cancellation, we just need to ignore them. I think in general, we want the libgit2 functions to either finish to completion or cancel when they require more javascript interation.

This is all predicated on what the Node devs have explained, which is that once cleanup of a worker thread has started, no more attempts should be made to communicate back to the javascript thread. At that point we should wait for completion of all execute threads while actively cancelling operations via callbacks when we see them come through. Any callback batons that get created should forcefully close if they do not have results ready from the JS thread.

@implausible
Copy link
Member

@implausible implausible commented Sep 11, 2020

That was a wild ride, but we think we got worker threads and async resources fully functional in #1795

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

4 participants
You can’t perform that action at this time.
X Tutup