Network DatagramSocket proposal

General JUCE discussion
1

Since some time now, i modify the juce.socket class every time i download a new version.

It enables me to find the IP adress of the person sending the UDP package.

When using a lot of sockets, and even broadcasting, this function is a must:

int DatagramSocket::readWithIP (void* destBuffer, const int maxBytesToRead, const bool blockUntilSpecifiedAmountHasArrived, String  *ipaddress);

This way, i'm able to verify the sender of the packet, and even filter out myself.

 

This is the implementation (for Mac):

int DatagramSocket::readWithIP (void* destBuffer, const int maxBytesToRead, const bool blockUntilSpecifiedAmountHasArrived, String  *ipaddress)

{

    return connected ? SocketHelpers::readSocketIP (handle, destBuffer, maxBytesToRead,

                                                    connected, blockUntilSpecifiedAmountHasArrived, ipaddress)

    : -1;

}

and:

static int readSocketIP (const SocketHandle handle,

                             void* const destBuffer, const int maxBytesToRead,

                             bool volatile& connected,

                             const bool blockUntilSpecifiedAmountHasArrived,

                             String *ipaddress = nullptr

                             ) noexcept

    {

        int bytesRead = 0;

        

        while (bytesRead < maxBytesToRead)
        {
            int bytesThisTime;
#if JUCE_WINDOWS
            bytesThisTime = recv (handle, static_cast<char*> (destBuffer) + bytesRead, maxBytesToRead - bytesRead, 0);
#else

            struct sockaddr_in saClient;
            socklen_t nLen = sizeof(sockaddr);
            bytesThisTime = recvfrom(handle, static_cast<char*> (destBuffer) + bytesRead, maxBytesToRead - bytesRead, 0, (sockaddr *)&saClient, &nLen);

            if(ipaddress != nullptr){
                ipaddress->append(String(inet_ntoa((in_addr)saClient.sin_addr)), 15);
            }
#endif
            if (bytesThisTime <= 0 || ! connected)
            {
                if (bytesRead == 0)
                    bytesRead = -1;
                    break;

            }
            bytesRead += bytesThisTime;

            if (! blockUntilSpecifiedAmountHasArrived)
                break;
        }

        return bytesRead;

    }

I'm not sure how this works out on different platforms...

2

I've added something equivalent to this now, and also in the process tidied up some internal socket code. I don't actually have a sensible test app for the DatagramSocket, so would appreciate you trying my changes to see if they work for you!

3

Jules, could you please also add the possibility to retrieve the source port ? In a UPnP device implementation it is vital to know which source addr + port to send the reply to.

4

And while I'm at requests, please also add a possibility to send (write) a message via an already bound port, f.i. nominally a datagram socket will be bound to a port given by the OS, and I would like to be able to send a message using that socket (which connects to the need to be able to get the source port on the receiving side).

Today I use sendto (socket API) directly with the handle given by DatagramSocket::getRawSocketHandle .

Maybe via an added DatagramSocket::write(const void* sourceBuffer, const int numBytesToWrite, const String& remoteHostName, const int remotePort) ?

5

Getting the source port of a message is definitely a useful feature and we will add this functionality to Juce.

However, I don't quite understand the last request about binding to a particular port. You say that you need to use the socket API to directly call sendto. However, looking at the current implementation of write (juce_Socket.cpp:590) you have complete control over ALL the parameters of sendto, i.e. by calling connect or bindToPort (or both) before calling write. As UDP packets are completely stateless I think it should not matter that a call to connect (juce_Socket.cpp:523) will close and re-open the underlying native socket. The remote should have no way of sensing this, right?

6

No, not the state, but consider me have an UDP listener with port Y. I want to be able to send queries to a multicast port so that receivers can respond back, to port Y. 

This is currently impossible, since the re-opening of the socket will give me a new port X, at the same time as messing up my listener (note that a connect re-opens the socket, but does no bind). 

With my write function proposal, it will be possible to reuse the already available socket so that the source port will be correct.

 

7

Sorry for the late reply. I've re-written the DatagramSocket class to more closely reflect the underlying OS calls and done some general clean-up of the socket classes. Now there is no such thing as a "connected" state for a Datagram socket. I've also added a sample chat app "ChatterBot" in the example folder which shows how to have seperate reader and writer threads. I've pushed it to my private repo for now:

https://github.com/hogliux/JUCE

It would be great if you could have a look and tell me your thoughts before I push this to the official repo.

 

8

Thanks Fabian, looks quite interesting! Will test and get back to you!

9

Ok, tested and I think it works nicely with a slimmer/neater API . I've added functionality to be able to receive multicast messages:

juce_Socket.h (DatagramSocket):


    /** Join a multicast group
    
        @returns true if it succeeds.
    */
    bool joinMulticast(const String& multicastIPAddress);
    /** Leave a multicast group
    
        @returns true if it succeeds.
    */
    bool leaveMulticast(const String& multicastIPAddress);

juce_Socket.cpp (SocketHelpers):


    static bool multicast (int handle, const String& multicastIPAddress, bool join) noexcept
    {
        struct ip_mreq mreq;
        zerostruct (mreq);
        mreq.imr_multiaddr.s_addr = inet_addr( multicastIPAddress.toUTF8() );
        mreq.imr_interface.s_addr = INADDR_ANY;
        if (setsockopt(handle, IPPROTO_IP, join ? IP_ADD_MEMBERSHIP : IP_DROP_MEMBERSHIP, (const char*)&mreq, sizeof(mreq)) == 0)
            return true;
        return false;
    }

and


bool DatagramSocket::joinMulticast(const String& multicastIPAddress)
{
    if (!isBound)
        return false;
    return SocketHelpers::multicast(handle, multicastIPAddress, true);
}
bool DatagramSocket::leaveMulticast(const String& multicastIPAddress)
{
    if (!isBound)
        return false;
    return SocketHelpers::multicast(handle, multicastIPAddress, false);
}
10

OK this is looking really good. Just for completeness, does it also make sense to implement multicast for TCP aka StreamingSockets? I don’t have much experience with multicast.

11

No. Since streaming sockets is per definition 1 to 1, and multicast is 1 to many, it only makes sense in DatagramSocket.

12

I've noticed a problem: When I send a message, the sender IP is 127.0.0.1. This will be a problem on the network, since the receiver will try to reply to itself. Also, I see that I can only see multicast messages sent by localhost. Strange, I got this working earlier, I'll see what changes I did then.

 

13

Ok, I think I got it to work now. I needed to add the IP_MULTICAST_LOOP option when joining the multicast. Now I get all multicasts on the network, and I have a client application correctly requesting/receiving data. 

Alas, there's no option to upload a ZIP file here, and I didn't checkout your code via git, so can you send me your email and I can send you the juce_Socket code with my additions.

 

14

I'd like to do some testing on the multicast before pushing it. I've been doing some testing with my own code and see quite different behaviour regarding multicast between linux, windows and mac. I'll push the socket changes without the multicast for now so that people don't start writing code based on the old DatagramSocket interface. Once I've had some more time for testing I'll include multicast support

15

Ok, I too have witnessed strange behavior on Windows 7, which I think I've tracked down to the INADDR_ANY given to imr_interface when joining the multicast.

The docs state that the OS will determine the interface on which to join the multicast group, and it seems that sometimes Windows chooses localhost interface, and sometimes the physical ethernet interface leading to the application intermittently non-working.

If I specifically give the physical ethernet interface to imr_interface, multicast receiving works all the time on that interface.

So probably, the thing to do is either to use IPAdress::findAllAdresses and do a muticast join on all those interfaces for the socket, or to add the interface address as a parameter to joinMulticast method. 

See if you can get consistent behaviour over OSs using this method.

16

One other thing that I've seen is that on Windows 7 at least, the multicast interface used can be random, sometime localhost, sometimes the physical ethernet interface. In the former case it is quite obvious that no machines on the ethernet network will get the multicast message...

One way to get around this is to explicitly specify through setsockopt(IP_PROTO, IP_MULTICAST_IF, ...) which of the interfaces to use for multicast (and not let the OS resolve the interface).

17

Hmm it might be an idea then to do some sort of lazy binding once the destination addr is known. It should be quite simple to work out the which interface to use once the user supplies an address. Does the setsockopt(IP_PROTO, IP_MULTICAST_IF, ...) call need to come before or after binding the socket?

18

As I understand the docs, you don't need to bind to use IP_MULTICAST_IF option, however, when I've tried it I still can get the message sent on local interface. With the Intel UPnP device sniffer tool, the multicast messages seem to be sent on all interfaces, so they've managed somehow...

Edit: This page seems to suggest the above, and a way to send multicast on all interfaces (Linux solution) http://atastypixel.com/blog/the-making-of-talkie-multi-interface-broadcasting-and-multicast/

 

19

Ok. On Windows at least it is necessary to bind the socket to a specific interface, i.e. not INADDR_ANY, as there does not seem to be any guarantees as to which interface will be used otherwise.

So DatagramSocket::bindToPort becomes:


    //==============================================================================
    /** Binds the socket to the specified local port, and optional interface.
        The localPortNumber is the port on which to bind this socket. If this value is 0,
        the port number is assigned by the operating system.
        The interfaceAddress is the specific interface on which to bind this socket. If this value is omitted
        the interface is chosen by the operating system.
        @returns    true on success; false may indicate that another socket is already bound
                    on the same port
    */
    bool bindToPort (int localPortNumber, const String& interfaceAddress = String());

and SocketHelpers::bindSocketToPort :


   static bool bindSocketToPort (const SocketHandle handle, const int port, const String& address) noexcept
    {
        if (handle <= 0 || port < 0)
            return false;
        struct sockaddr_in servTmpAddr;
        zerostruct (servTmpAddr); // (can't use "= { 0 }" on this object because it's typedef'ed as a C struct)
        servTmpAddr.sin_family = PF_INET;
        if (address.isNotEmpty())
        {
            servTmpAddr.sin_addr.s_addr = inet_addr (address.toUTF8());
        }
        else
        {
            servTmpAddr.sin_addr.s_addr = INADDR_ANY;
        }
        servTmpAddr.sin_port = htons ((uint16) port);
        return bind (handle, (struct sockaddr*) &servTmpAddr, sizeof (struct sockaddr_in)) >= 0;
    }

This way I can setup a socket for each interface (found by IPAddress::findAllAddresses), and issue a multicast request on all interfaces.

Now it works rock solid.

So probably, there's no need for IP_MULTICAST_IF using this method (and besides, IP_MULTICAST_IF doesn't seem to work as expected on Windows)

Edit: Just to recap why this is necessary. In UPnP the receiver of the multicast responds to the sender by way of source addr/port (in the IP packet), this is why it is necessary to bind to a specific interface, so that multicasts get sent correctly (and be responded to correctly).

 

20

This looks very nice, and I actually need these features now. Will this be part of a new release anytime soon, or is it best to grab it from the Git at the moment? Is the new network functionality considered stable at the moment?