GPU memory leak?

[Invaliderror]

In Task Manager, the amount of “Dedicated GPU Memory” reported as allocated to SpaceDesk keeps growing at a rate of about 100MB per screen update on the remote display. In the attached window capture, SpaceDesk has 19.4TB of hypothetical memory assigned to it so far, which is almost 1000X my total system memory. If I wiggle my mouse on the remote display, the allocation goes up by about 1GB/s.

I have to periodically restart SpaceDesk to forcibly free all of that ghost memory allocation since it will eventually cause my system to crash from running out of memory, presumably from tracking all of the meta-data for the stray allocations.

Anyone else seeing GPU dedicated memory creep up like this?

Attachments:

You must be logged in to view attached files.

[spacedesk Lea]

Hi @invaliderror,

Thank you very much for reporting this issue.
We are currently investigating and trying to reproduce this issue.

For further analysis, could you please send us the diagnostic info of your primary machine?
Just open the spacedesk Driver Console -> Diagnostic -> Save All Information button.

PS: Saving all information will take a few seconds, then please send us the whole folder which contains all the diagnostic logs collected.
Please right-click the folder -> Send to -> Compressed (zipped) folder, then attach the *.zip file in your next reply.

[Invaliderror]

I have tried “Save All Information” three times and the designated output folder remains empty even after installing DebugView, running debug mode for the few seconds it takes to get 20GB of “Dedicated video memory” by wiggling a window on the remote display and turning it back off before clicking the button.

If it is a systemic problem, it shouldn’t take more than a few seconds to duplicate:
1- connect a remote display
2- open Task Manager in detail view showing dedicated GPU memory
3- wiggle a window on the remote display
4- watch the dedicated GPU memory rise by ~1GB/s for as long as the window is being wiggled on the remote display

The first time my computer mysteriously crashed from running out of memory was about a year ago. Whatever the issue might be, it isn’t a recent development.

I found DebugView’s log where SpaceView dumped its unpacked copy. No activity in there beyond the initial display connection chatter.

Since it is a VRAM-related issue, it is likely worth mentioning that I am using a GTX1050.

Attachments:

You must be logged in to view attached files.

[spacedesk Lea]

Hi @invaliderror,

Thanks for the feedback.

We were able to reproduce the increasing dedicated GPU memory (but no system crash yet..).
We will continue analyzing it and get back to you as soon as we have news regarding this issue.

In the meantime, please run dxdiag.exe, then click “Save All Information” then send us the dxdiag.txt of your primary machine.

[Invaliderror]

Getting to the crash takes a few weeks of actively using the remote screen a few hours per day without restarting SpaceDesk. The 17TB in my screen cap was about a week after updating to 0.46. Between the timing and the tendency of most computer things aligning with powers of two, I’m guessing the out-of-memory system collapse happens at either 32TB or 64TB on my setup. The system runs out of physical memory tracking virtual memory that gets allocated on every screen update but never actually used.

Technically, the system didn’t quite crash back then. The OS terminated processes left and right to free up memory and put the system in an unusable state where attempting to do anything resulted in out-of-memory errors until enough of whatever was still running got OoM-crashed on shutdown to free up memory for the remainder to shut down normally.

Attachments:

You must be logged in to view attached files.

[spacedesk Lea]

Hi @invaliderror,

After further analysis, we believe that this GPU dedicated memory counter in task manager’s Details tab is reporting a wrong value and a known issue in Windows 10/11.
Reference: https://docs.microsoft.com/en-us/troubleshoot/windows-client/performance/gpu-process-memory-counters-report-wrong-value
Please check the correct GPU memory usage, you can monitor it in the task manager’s Performance tab.

Maybe your system is crashing for a different reason and not because of the GPU dedicate memory leak.
Can you tell us more about the system crash?
Did it happen while spacedesk viewer is connected? Did it crash immediately upon connect?
Did you notice any unusual behavior on your primary machine before the system crash?
To further analyze this issue, could you please send us the dump file of the system crash you observed before.
Dump files (*.dmp) are located in C:\Windows\minidump folder.

Looking forward to your feedback.

[Invaliderror]

When I don’t use SpaceDesk, my PC runs fine 3-4 months at a time between manual reboots for system and driver updates.

I don’t have a minidump folder as the system never outright crashed. It ran out of memory which caused programs to crash and the system to become unusable but still running. It was the first time I have seen an “out of memory” error in 10+ years.

I just went through my Event Logs, found a huge blob of service failures on 2022-04-02 which must be when the quasi-crash happened and mixed in there, I have a Resource Exhaustion Detector event stating the following:
Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: spacedeskService.exe (6624) consumed 15600336896 bytes, Wow.exe (23148) consumed 7649738752 bytes, and soffice.bin (10900) consumed 1279406080 bytes.

At the very least, something definitely caused virtual memory allocation to go completely nuts. I rarely play WoW for more than 3-4h at a time and it somehow got to 7TB of virtual memory too.

[Author]

Posts