delegated encoding mode

[totaam]

Building on #370 and #426: we want to be able to do the encoding on the host (proxy server) using NVENC and let the actual xpra server sitting behind it just throw RGB pixels at it.

How this is going to work:

• the proxy server tells the real xpra server it will handle video encoding via a new capability attribute ("encoding.proxy_video_encoder = { ... } "?):
  • how many encoding contexts it will accept
  • what size requirements it has (min/max sizes, mask for non-even sizes, etc..)
• the real server then creates a "proxy video encoder" spec, which functions just like a real one, except that its characteristics are populated from the values above
• when this new encoder is requested to encode frames, it will just call the regular RGB encoder and add a tag to the metadata ("proxy_video"?) We will want the ability to choose whether to use compression or not from the real server to the proxy (virtual machines running on the same host as the proxy will probably not want the overhead, but servers connected over ethernet will)
• when the proxy receives such draw packets, it will create a video encoding context and compress the frame
• the proxy needs to watch for lost-window packets, and close the contexts if necessary

Potential problems:

• when running multiple virtual machines on the proxy server: large guest to host memory transfers will become a problem: virtio-net is good, but not that good. A 1080p screen at 25fps sent as RGBX will cost 200MB/s (and we do have to copy it more than once!), even a powerful server can't take too many of those
• creating an encoding context takes time: either we allow draw packets to be sent out of sync (and verify the window hasn't gone already by the time our packet is ready to send - what about resizes and moves?...), or we will suffer latency spikes. alternatively, we could create the context in advance, but we don't support context resizing yet... see nvenc improvements: YUV444P mode and bandwidth auto tuning #466
• latency perceived by the server: when the server receives the draw ACK packets, it will look like the client is taking a long time to process draw packets compared to other packets which go straight through the proxy - hopefully this won't cause us problems
• batch delay calculations: if we manage to send far more RGB frames to the proxy than it is capable of encoding and sending, the lack of ACK packets should smooth things out. I think.
• out of sync issues: just because we tell the real server how many video contexts it is allowed to create does not mean it will always honour it (for whatever reason), or that we won't have failures to create contexts proxy side. We could overcommit contexts too. So, we probably need to fallback to software encoding for these, hopefully rare, cases. I wouldn't want to have to add a message back to the real server to do realtime context tuning... but we may have to.

Longer term:

• load balancing: if the proxy server has a pair of K1 cards in it, we want to load balance
• overcommit and load distribution: a session that only does 2 fps should not monopolize an encoding context, we could give the encoding context to a more demanding session

[totaam]

2014-01-27 14:15:23: antoine uploaded file proxy-encode-v1.patch (14.4 KiB)

PoC proxy encoding delegation preparatory work

[totaam]

2014-01-27 15:27:04: antoine uploaded file proxy-encode-v2.patch (18.6 KiB)

PoC proxy encoding delegation: sends BGRX pixels with video proxy tag

[totaam]

• r5286 (see detailed commit message) adds basic support for proxy encoding, tested with x264.
• r5275 ensures we can send RGB pixels uncompressed when setting speed to 100%, no matter how many pixels there are.
• r5276 and r5285 increase the network performance when dealing with large packets (which is the case for uncompressed RGB)
• r5287 + r5288 tidy things up

What still needs to be done:

• dealing with an encoding thread to reduce latency
• dealing with speed and quality changes - which are normally handled out-of-band...
• limit the number of contexts

[totaam]

• fixes in r5300, r5307
• r5308 honours the quality and speed settings (with x264 encoding)
• r5348 adds an encoding thread for better latency
• r5375 adds a proxy unix domain socket so we can get "xpra info" from the proxy process
• r5478 better logging
• r5483 lots of improvements and fixes (see commit message)
• the load balancing and NVENC issues are now here: CUDA and NVENC load balancing #520

It is working surprisingly well!

[totaam]

Ready for testing (with and without #520):

• how many users can we run using this setup?
• what is the first bottleneck? (profile it?)

[totaam]

2014-06-10 06:18:58: smo commented

Closing this for now until we do some further testing. I'll reopen and comment when I have time to do more.