It appears that there are two versions of this extension - the .gmez version (with 5 files included), and the one used in the .gmz demo (with 2 files included). It works in the demo, and if I export the 2 files version from it and import it to my project, it is working. But the game crashes on game_restart. No errors, nothing, it just freezes and shuts down. Even the simplest project, even your demo, if you add game_restart() on button push. The big version (with 5 files) does not crashes, but I also can't make it work. Just importing the extension and putting "window_set_cursor_sprite(spr_test, 0)" at the beginning does it for me with the small version and does nothing with the big one (the rest of the project is the same). So is there a way to make a small version to not crash on game_restart, or am I doing something wrong with the big version? 

Firstly, I would strongly discourage against using game_restart(), especially in GMS1 - it will leak every single dynamic thing that you have made (be it data structures, dynamically loaded assets, etc.), will not reset global variables (meaning that if you’re trying to access a not-yet-set global, you’ll get the value from before game_restart()), and otherwise introduce undefined behaviour. For extensions, it runs their “init” code (which is supposed to only run on game start) the second time, which requires workarounds just for that purpose.

Then, you are saying “5 files” - are you manually extracting the GMEZ instead of adding the extension via Extensions - Import Extension? I cannot reproduce this on my end

I am importing the extension through the context menu, like you mentioned. By "files" I mean subcategories in "Extensions -> window_set_cursor". The version from the demo has only "window_set_cursor.dll" and "window_set_cursor.gml" and is working. If I export this extension and the import it to another project it will work. If I to import the extension that came as the separate download (that contains "window_set_cursor.dll, window_set_cursor.gml, window_set_cursor_dll.gml, window_set_cursor.js and window_set_cursor_else.gml"), it does not work.
But I see that game_restart() is a bad way to restart my game (I use surfaces a lot and they will probably stay in memory), so I will have to redo this section, and because of that I will avoid the bug with the extension. Thank you for the detailed answer!