Remote Procedure Calls

Remote procedure calls are one of few existing ways of communicating between processes. While threads can use mutexes (as all threads of same process have access to same memory and therefore to same mutexes), threads of different processes can't all access the same memory regions all the time.

Remote procedure calling has, as pretty much any IPC mechanism two ends. The callee end and the caller end. Callee end is the end which ends up being called as a result of RPC and we will call it "server" in further text. Caller is the one, who will perform the call, once it decides it is right time to do so. We will call it "client" for short.

High level design of RPC mechanism

From high level perspective, RPC interface works in a way similar to C++ objects. Server process can declare one or more services, which are composed of methods. Methods are callable via the instance of the service.

Hierarchy of RPC components

From the server perspective, the service instance is a fully defined structure, which holds service data and contains pointer to the list of callable service methods.

From the caller perspective, the service instance is an opaque pointer. This pointer, once obtained, can be used to call service methods. Internally, the object pointed to contains link to service method table. If caller uses this pointer to call service method, this call gets resolved on compile time. If method with requested name does not exist in target service, the compilation fails.

Despite the fact, that the called method is resolved during the compile time, the call itself is performed based on the service status during the runtime. Compile time resolution makes sure that target method actually exists within referenced service and composes kernel call to perform RPC call.

Later, during the runtime, kernel resolves service method address and passes control to the provided address. At no time, the calling process has access to service object, despite it's ability to call service methods. Runtime resolution is done by the kernel.

RPC interface

RPC interface is a structure, which lists pointer-to-function kind of members. This is a typical way of writing "C classes". There are no special requirements for internal organization of this structure except of typical best practices in writing reusable interfaces.

#include <cmrx/rpc/interface.h>
 
struct RPC_interface {
    void (*method_1)(INSTANCE(this), int arg1);
    int (*method_2)(INSTANCE(this), int arg1, int arg2);
    void * (*method_3)(INSTANCE(this) /* no arguments */);
};

There are limitations given by RPC mechanism in CMRX though. Any method of RPC interface has to have 1 up to 5 arguments. First arguments must be pointer to instance. This semantics is same as used in Python methods. Remaining four arguments can be of any scalar type. Method can return any scalar type, but does not need to return anything.

RPC service

RPC service holds the data, RPC service provides access to. It is plain C structure with only one requirement. First member of this struct must be a pointer to the RPC service, which provides access to this object and this member must be called vtable. Amount, types, structure and order of remaining items is completely up to application developer. Even removal or reordering of items within structure won't compromise binary compatibility, as long as your service can cope with it.

struct RPC_service {
    const struct RPC_interface vtable;
    long long service_data;
};

Becoming a RPC server

In order for some application to define a RPC server, it has to do quite a few things:

Declare interfaces

Very first step is to define the interface. Interface is a structure visible both to the server and the client, which defines what actions some type of service supports. Interface is purely procedural and does not support publishing of data members. Interface has a form of structure containing pointer-to-function members. Interfaces shall be as abstract as possible, creating contracts between the caller and the callee rather than describe manipulation methods specific to one single interface. This facilitates interface reuse. If your service has possible actions similar to some other service, then both services shall implement the same interface. If they do then these services are interchangeable. Caller can call an instance of both without actually knowing what specific service it is calling or that they are in fact different.

This step can be skipped if there is suitable interface available somewhere in your project. You can reuse it rather than defining your own interface.

Declare services

After interfaces were defined (or adopted), the structure of the service itself has to be provided. This is typically a structure, which holds all the state of the service. This structure must contain a link to the interface, this service implements.

In CMRX, the internals of the structure are never accessible outside the owning process. So in practice, it is possible to provide two declarations of the service: One, fully specified with all the data members. This one is used internally while writing the service implementation. Another, containing only the reference to interface implemented may be provided for callers.

Above is not a must, even if caller knows the internal structure of the interface, it is not possible to access it. Not even in read-only manner.

Define service implementation

Next, it is necessary to provide implementation of all functions declared in the interface this service implements. Once these functions are implemented, they can be put together into interface implementation table. This is an instance of interface, where function pointers point to actual functions just created.

#include <cmrx/rpc/implementation.h>
IMPLEMENTATION_OF(struct RPC_service, struct RPC_interface);
 
static void service_method_1(INSTANCE(this), int arg1)
{
    /* ..... */
}
 
static int service_method_2(INSTANCE(this), int arg1, int arg2)
{
    /* ..... */
}
 
static void * service_method_3(INSTANCE(this))
{
    /* ..... */
}
 
VTABLE struct RPC_interface service_vtable = {
    service_method_1,
    service_method_2,
    service_method_3
};
 
struct RPC_service service = {
    &service_vtable,
    0x42
};

If any interface provides some function, then it is mandatory to implement it. No pointer in interface can point to NULL. Doing so will crash any caller which attempts to call such interface method.

Instances of interfaces have to be marked by special keyword VTABLE. This keyword serves the purpose of putting such variable into VTABLE "region" of the process. Only those interface implementations residing in VTABLE regions can be called. Net effects of this limitation are two:

1. Only RPC services, which were intentionally provided as such, can be used to perform legitimate RPC call. No random data put together which resemble pointer-to-function array can be used as a RPC service.
2. Presence of VTABLE address in some particular process' VTABLE region established relationship between RPC service and owning process. This is also important from memory protection perspective, as it allows kernel to determine, how to set up memory protection during the RPC call.

Calling RPC service

Once RPC service has been established having its own implementation it is possible to perform calls of this service. Service instance lives in the address space of the process, which instantiated it. It is not possible to place the service instance into some globally-accessible place as no such space exists. It is not even needed. All that any potential caller needs is the address of the service. Call of this service can then be performed using the following code:

int retval = rpc_call(service_ptr, method_2, 0x42, 0xF00F);

This call will perform the call of method_2 from within service using CMRX RPC mechanism.

Advantages of RPC mechanism

Why mechanism so complicated?

Despite looking complicated, this mechanism has a few advantages over directly calling functions, or calling functions via pointer-to-function table indirection. If we first consider the simplest case of calling specific service functions via their names directly, such as:

int retval = service_method_2(service_ptr, 0x42, 0xF00F);

Then there is an obvious difference, that in the latter case, you are explicitly stating the method to call. While using the RPC mechanism, you are only stating its name. So the service is able to provide whatever specific function that implements the requested method for this specific kind of service. Calling client and called server are tightly bound to perform one specific action.

Another way of calling service methods in a polymorphic way is to use "object oriented C" approach and call the method like:

int retval = service_ptr->vtable->method_2(service_ptr, 0x42, 0xF00F);

This approach is much closer to what RPC call actually does with a few little, yet important differences. Even if it looks like the actual function executed when method_2 is called can be changed, it isn't absolutely true. There is still rather high degree of binding between the interface and the service here. The function, whose address is used to initialize the vtable structure must match the type and count of arguments in the vtable type declaration. This will force you to use method with the following prototype:

int (*method_2_t)(RPC_service * this, int arg1, int arg2);

This works as long as you have one service and one interface. There is direct relationship between RPC_interface structure whose members type their first argument as RPC_service *. If you attempt to create an interface usable with multiple services, you will face type incompatibility problems. Either you'll have to typecast functions during the initialization of RPC_interface instance, or use some generic type for this, such as void * this. Both will clutter the code with typecasts and will throw part of type safety as typecasts will be explicit and can contain typos or other mistakes.

Another problem of both approaches is, that the function which ends up being called one way or another must reside in the same process as the caller. You can't call function from within another process as it will crash in the moment it tries to access it's own process' memory.

RPC call mechanism deals with both problems. Syntactic sugar around the macro INSTANCE() causes that whenever an interface is declared, its this pointer is not typed. It effectively behaves as void * this. On the other side, whenever you provide an implementation of the interface, you have to provide a service - interface pair. When you do this, the check, if the intended service actually provides intended interface will happen. This way, typos are avoided. Attempt to implement an interface for service it isn't providing it will cause build error. Another effect is that use of macro INSTANCE() during the implementation of the interface will use the correct type expected by this specific implementation.

From the client's perspective, the actual type of service data structure is not important. Client handles structure instance as an opaque pointer during runtime. It never accesses it as it resides in other process' memory. All the preparatory work to implement the RPC call is done during the compile time. Here, the type checking is done, so that the type and amount of arguments to the call is checked. Compilation will fail if there is a mismatch. Compilation will also fail if there is an attempt to call non-existent RPC method.

Then, during the runtime, when RPC call is performed, CMRX kernel will make necessary adjustments of the runtime environment to allow the called method to access the memory region of owning process.