Limit on size of incoming OSC message - 4098 bytes


#1

Hi, when receiving OSC messages, they seem to be getting truncated at 1550 characters in length. When getting the same OSC sender to send somewhere else (in this case a Max patch), the whole of the message is correctly received.

Cheers


#2

There is a limit on UDP packet sizes that is much lower than the theoretical maximum of 64 KB.

All hosts must be prepared to accept datagrams of up to 576 octets (whether they arrive whole or in fragments). It is recommended that hosts only send datagrams larger than 576 octets if they have assurance that the destination is prepared to accept the larger datagrams.

There is no foolproof way to get the maximum size, it could be limited by OS settings, network hardware or various other things.

More info here: OSC blobs are lost above certain size


#3

Hi - thx. Yes, I understand this at the lower level, but the OSC code in JUCE should surely handle this and just pass on the completed OSC message once all packets have been received?

At the end of the day, Max is handling these OSC messages fine so I would expect JUCE to do the same.


#4

Odd. Are you using the juce OSC sender? Fragmentation should only happen at the IP level. The UDP packet should either arrive or not. Can you see the complete data in the raw data received by juce. Is it a parsing issue?


#5

No, this is from a different source, but as said, these message are being received fine in Max, but are being truncated at 1550 bytes in JUCE. The OSC is well formed and can be received in its entirety in Max. Reducing the length of the sent message so that ‘s’ below is less that 1550 bytes allows content to be processed correctly.

An exception is being thrown here:

    String readString()
    {
        checkBytesAvailable (4, "OSC input stream exhausted while reading string");

        auto posBegin = (size_t) getPosition();
        auto s = input.readString();
        auto posEnd = (size_t) getPosition();

        if (static_cast<const char*> (getData()) [posEnd - 1] != '\0')
            throw OSCFormatError ("OSC input stream exhausted before finding null terminator of string");

        size_t bytesRead = posEnd - posBegin;
        readPaddingZeros (bytesRead);

        return s;
    }

#6

I do not have any experience with OSC, but can you verify that the null terminator actually exists in the packet? Max may be forgiving in it’s OSC parsing and work without the null terminator. while JUCE seems to be enforcing the spec, which indicates that OSC-strings need to be null terminated. One way of verifying this is using something like WireShark to look at the packet outside of your application. If the terminator is in the packet, then it does seem like a JUCE bug, otherwise the sender is sending a malformed packet.


#7

Yes, the terminator is in the message, but not in this packet. The message is about 3500 bytes in length, but as said, the result of:

auto s = input.readString();

is 1550 bytes. So it’s correct in the fact that it’s missing the null terminator, but that’s because the null terminator is presumably in the next packet.

If I reduce the length of the message to be less than 1550 bytes then the null terminator is correctly found and the message is processed.


#8

FYI, this code has been running fine for a couple of years - only hit a problem in the last couple of days with a message much longer than is usually sent.


#9

Looks likely that JUCE isn’t reassembling the multipacket OSC packets, as that 1550 is near the 1500 mentioned in this post:
http://opensoundcontrol.org/topic/247


#10

The sender is not sending a malformed packet - as explained a couple of times, these messages are processed fine in Max. The sender is a Python sender and the library is commonly used.


#11

yes, that would seem to fit!


#12

Can you post the raw data of the packet? What is the size of input.getTotalLength()?


#13

The great thing about JUCE is you can look at the source code… :slight_smile: and taking a quick peek in juce_OSCReceiver.cpp it’s pretty clear that is simply receives data from the socket and passes it on to be parsed. OSCReceiver::run handles reading the socket, which then calls OSCReceiver::handleBuffer, which passes it into an OSCInputStream for parsing…


#14

You could throw some debug output into OSCReceiver::run to further diagnose/verify.


#15

Off out now but will investigate further


#16

I ran into a similar problem some time ago. I was sending big OSC blobs between a JUCE-based sender and a JUCE-based receiver and the packets received were reported to be invalid. It turned out that the packets received were truncated too. However, capturing the packets with Wireshark (which is a great tool for OSC debugging by the way) showed valid non truncated UDP Packets. I put some breakpoints in OSCReceiver::run and found out that however the underlying socket delivered truncated raw data.

In the end I gave up digging deeper at which low level point it fails after having read the post G-Mon linked above as it points out that everything above 567 bytes doesn’t necessarily have to work as it is above the limit the UDP standard gurantees to work.


#17

I looked into this a bit, I can reproduce the issue that you are having, but I see it at 4098 bytes, not 1550.

The issue is:

        enum { oscBufferSize = 4098 };
        char buffer[oscBufferSize];
        auto bytesRead = (size_t) socket->read (buffer, (int) sizeof (buffer), false);

Meaning it won’t read the entire messages if it is over 4 KB. The remaining data will get discarded. Not sure why you are seeing issues at 1550 bytes rather than 4KB, are you on latest juce?

I created test program to reproduce issue here: https://github.com/FigBug/juce_bugs/blob/master/OscRecv/Source/MainComponent.h


#18

Hi, nice find - I’ve checked again and you’re right - it’s hitting the 4098 limit - the message contains several fields and it’s the final field which is getting truncated when it hits the 4098 limit (or 1550 bytes of that field).


#19

Verified that increasing this buffer size fixes the issue.


#20

any comments from JUCE devs on whether they’re going to address this?