Yes, display performance does heavily depend on the network indeed. Highest performance is achieved by cabled networks (e.g. USB, Ethernet). Performance on wireless networks can significantly be improved bypassing routers (configuring primary PC or secondary device as Access Point).
For further more detailed information about the display performance, please check Instruction Manual chapter “Performance Tuning” https://manual.spacedesk.net/PERFORMANCETUNING.html