File Transfer speeds with Godot

Kemb0

I was recently following the high level multiplayer page on the Godot site and thought it'd be an interesting experiment to see if I could make something that could transfer files between devices on my network.

I got it working with the eNet stuff but boy was it slooooow. I then gave Websocket a go which seemed noticeably faster but still only gave about 40 Mbps on a network that should theoretically handle 1000. I was sending bursts of 255 x 64k packets, waiting on a return confirmation of arrival before requesting the next burst. I'm new to this so no idea if I can just send the whole lot in one go, but it seemed wise to send it in chunks. I was able to send a 1.2gb file correctly in about 5 minutes

Does anyone here know much about this side of things with Godot? Should I be able to get faster file transfer speeds or is Godot causing a bottleneck? Would TCP/UDP servers be any different? Or does the eNet stuff from Godot simply use them anyway? Could I create multiple "servers" using different ports and send parts of the file to each of them simultaneously to force it to use more bandwidth?

Thought this would be a fun project as I'm forever moving files around different devices and Windows sucks at it, forever "forgetting" or refusing to send files to other computers, not to mention trying to work with Macs as well. I know there'll be software out there to do this but I'd quite like to be in control of what it does and how it works, not to mention it's just a practical fun way to learn multiplayer.

Bimbam

I have not dabbled at all with Godot Multiplayer or Enet specifically, so will go for some general answers that may be worth investigating:

64k for a packet seems huge to me, far exceeding a regular jumbo frame (which is ~9k but to be fair my knowledge could be out of date) and may be causing packet loss (should show on router). I would suggest writing a dynamic function that starts at a min size of ~68b and then jumps up/down in decreasing increments while monitoring speed (or loss if possible) to try and get to the max stable packet size. If by packet your not actually referring to a TCP/IP packet but rather just a chunk of data (as really managing packets and the MTU should be abstracted), this is also to a degree expected as what I just described above is never fully completing. What I mean is in the time it takes to transfer a data chunk of 64k, the network/IO stack has not gotten close to fully optimising the transmission speed and effectively restarts the process for the next transmission (in addition to IO overhead each time). Not only can you see this in windows in general when handling hundreds of small file transfers, but even apps like WinSCP will exhibit the same behavior. The solution here would be to pre-compress/batch data transfers and decompress/re-order on receipt. There is a cost to this as well, so really it should be thresholded at a certain point where time to compress < time to burst send. Not sure how I would go about calculating this threshold dynamically which may be why WinSCP doesn't do this out of the box.
UDP would be faster at the cost of reliable receipt. It's probably not an option for anything outside of multicasting video streams which can handle the dataloss or adding a CRC check at the end to confirm full transfer completed (maybe fine on LAN but mileage will vary).
Re. working with OSX, Apple willfully or otherwise broke their SMB implementation (an open 'standard' that they decided they would not implement as standard -.-) years ago which results in it being slower/less reliable than it is on Windows or Linux making connecting OSX to both platforms a headache. Miraculously their AFP protocol works fine when transferring from Apple to Apple devices -.-. My personal eye-rolling aside, if you CAN use AFP when working with Macs, I would recommend it.
File transfers are only as fast as their slowest connection. If for example, you're transferring to a RAID5 WD Green NAS, 40MB/s sustained throughput is about what you could expect on a good day simply because the drive speed can't keep up. And that's assuming nobody else is reading/writing from it. If this is direct machine to machine and you have confirmed both machines are linked at 1Gbps full duplex with SSDs, yehnah, won't be this.
Last point for now, in bygone times Ethernet autoneg could cause one link to downsync to 100mbps half-duplex and cause collision storms. This is a monumental headache, but also not something I have seen in years (especially not on home hardware). However, the takeaway of checking your routing device for packet collisions/loss is still worth investigating but leads down its own rabbitholes.

Kemb0

Thanks BImbam for your thorough reply. I hadn't considered that smaller packets might be preferable. I'll do some experimenting there as you suggest and report back later (sadly hols over now so back to working evenings). I am already appending an index to each packet with the intention of using it to rebuild in the correct order later, however so far they are all coming in in order. I'll try that with UDP too and see how it compares. And yes I was referring to a chunk of data when I say packet.

Bimbam

I hadn't considered that smaller packets might be preferable. & And yes I was referring to a chunk of data when I say packet.

It is a little important to distinguish that data you are sending from A->B as chunks != to a TCP/IP packet, so may have confused my explanation a bit. For data you the user are sending from A->B, bigger is better.

A single 10mb file will transfer faster than 1000*10kb files as it results in less overhead caused by per file operations that go on in the background (things like opening a file, closing a file, AV scanning the new file, packaging the file into packets to send across a network etc. etc.).

'So what is a packet' you ask. Well, I am not going to try to butcher an explanation of the OSI Model, but the ;tldr is a 'packet is the data you are sending, broken up into chunks automatically that are suitable for your network based on a specific max packet size (MTU, usually ~1500bytes but can be up to ~9000bytes), where each packet has additional information prepended to it that explains to various parts of your OS, and the receiving OS, and the network hardware in between, what to do with said data. This means things like where it's going, where it's from, whether your expecting confirmation of receipt, if so, how to confirm receipt (CRC Checksum) and so on. The vast majority of this you wouldn't normally need to know, as it is abstracted by the Software your using, the Operating System and Networking hardware.

'I get file operations per file would add overhead, but why would multiple small files be less efficient for packets?' you ask. Well when the OS breaks your payload into the requisite packets there is almost always some remainder. Take this example (that ignores all the overhead stuff mentioned above for the sake of simplicity): You send 64000 bytes of data. This is split in packets of size 1500, resulting in 42.6 packets, or 43 packets. You're sending this 255 times, resulting in 10965 packets Now you send 1 file that is 16320000bytes (255 times 64000). This is split in packets of size 1500, resulting in 10880 packets, thus more efficient.

'So why is UDP faster' you ask. While TCP waits to receive an acknowledgement that data has been received successfully, UDP does not. This means not only does the header information for the packet not need to include things like a CRC checksum (so more space for your data per packet), it also doesn't have to wait for the acknowledgement the previous packet was received. While in most cases this will mean UDP executes faster, TCP does also handle congestion intelligently and will back off if the receiver can't take the heat, while UDP kinda does the opposite and just floods it with information, so in some scenarios it CAN be slower. In general, TCP is what you want unless your broadcasting/multicasting.

When I read your post, I initially assumed you were actually manually altering the packet size to in essence override the default behaviour of the TCP/IP stack. If you WERE doing this, what I said before applies that you would need to monitor the network for packet drops to find out what your max transmission size is and set this dynamically. This IS a valid thing people used to do back in the day to eek out small performance gains on janky internet connections and manually override their MTU settings. However, what I described is loosely what Path-MTU Discovery does anyway so doubt it needs to ever be done manually anymore.

Sorry if I've just ended up confusing you, I think I confused myself more than anything else (I mostly just wanted to make sure I still understood this crap >.<)

newmodels

The FPS limit might have slowed than your transfer.

ChapatizRetro

I don't know if that can help you but using HTTP class on godot, you need to modify speed download by using : client.download_chunk_size = 65536 (default is at 4 KiB on HTML5 export)

Kemb0

Hopefully to clarify Bimbam, when sending data I'm in essence doing this, currently with the WebSocketClient.

//setup _client = WebSocketClient.new() //when sending data loop this bit below 255 (arbitrary number) times then wait for server to acknowledge before sending another 255 etc if not loadedFile.eof_reached(): &ensp;&ensp;packet = loadedFile.get_buffer(65530) &ensp;&ensp;//do some stuff to add a unique index in to the packet &ensp;&ensp;_client.get_peer(1).put_packet(packet)

Incidentally if anyone knows, I'm trying to test out UDPServer but after first calling:

server := UDPServer.new()

and subsequently:

server.poll()

I'm getting an error:

Invalid call. Nonexistent function 'poll' in base 'UDPServer'.

Yet the poll() method is what the Godot documentation says to call.

https://docs.godotengine.org/en/stable/classes/class_udpserver.html

Kemb0

Just as an update, I tried out the TCP connection and was able to complete a 1Gb file transfer in around 10seconds,so seems like that's worked a treat.

Calinou

It's expected that ENet results in slow file transfers since it only uses UDP (an unreliable protocol at heart). UDP was never designed for fast file transfers in the first place, and you should be happy if you can reach speeds above 1 MB/s using UDP :)