Messaging with Redis and C# using ServiceStack

Leave a comment

In my previous post I wrote about subscribing and publishing to Redis message queues in node.js using the node_redis library.

The proposed architecture of the Kraken Office system ( see selecting technologies for Kraken Office for details ) uses Redis as a message broker to link the various parts of the system together. The previous post shows how to do this from the node.js web socket server but now we need to access the message queues in Redis from C# as the application and data access layer of the system will be written using the .NET framework.

We will be using the ServiceStack.Redis client library to access Redis from .NET.

Using ServiceStack.Redis

ServiceStack.Redis is a client .NET library that enables you to access the Redis no-SQL data store. It has functionality for using Redis as a data store and also for accessing the subscriber / publisher messaging functionality.

To use ServiceStack.Redis simply download the lastest code base and compile it using Visual Studio. Once compiled you will have access to the binaries used to interface with Redis.

Firstly you will need to start a Redis service instance (see subscribing and publishing to Redis message queues in node.js for details).

Open or create a .NET project using C# and add references to the following binaries generated during compilation of the ServiceStack.Redis project:

  1. ServiceStack.Common.dll
  2. ServiceStack.Interfaces.dll
  3. ServiceStack.Redis.dll
  4. ServiceStack.Redis.Text.dll

Once you have added the references you can then create a Redis client object in C# as following:

var client = new RedisClient("localhost", 6379);

Publishing a message to Redis is now as simple as:

var client = new RedisClient("localhost", 6379);
client.PublishMessage("node_layer", "message_body");

This publishes the message body to the specified message channel in Redis, in this case ‘node_layer‘. The url and port specified when constructing the client should point to your Redis server instance.

Subscribing to receive messages from a particular channels also uses the same Redis client object. To subscribe to a channel and start receiving messages the following code can be used:

var client = new RedisClient("localhost", 6379);
IRedisSubscription redisSubscription = null;

using(_redisSubscription = client.CreateSubscription())
{
     _redisSubscription.OnMessage += (channel, message) =>
     {
          // handle message as appropriate
     };
}

_redisSubscription.SubscribeToChannels(new string[] { "application_layer" });
<pre>

You can easily subscribe to multiple channels by specifying additional channel names in the array parameter of the SubscribeToChannels method.

Note that the SubscribeToChannels method is blocking so your process will block on this method call and messages to the OnMessage event will be fired on a different thread.

Conclusions

Accessing Redis’ messaging facilities in .NET can be achieved easily by using the ServiceStack.Redis client library.

The message broker and client library have a major limitation, however, in that they work on a broadcast basis. As a result, multiple clients listening to a particular channel will all receive every message.

This raises issues when trying to use the Redis message broker for creating scalable systems where extra clients can be added to provide additional processing resource. For this type of scalability a round-robin system of message is needed, where messages are balanced across subscribers.

Currently ServiceStack.Redis does not provide a round-robin message distribution option. As I like the Redis product and have found message queue integration easy from both .NET and node.js I have decided to write my own message proxy service to provide round-robin messaging facilities. This will be the topic of my next post.

Until then – happy coding!

Advertisements

Messaging with Redis and Node.js Using Node_Redis Client

1 Comment

Now that I have connectivity between client HTML 5 pages and a node.js web socket server (see creating an HTML web socket client for details), the next step in the process is to integrate a message broker to distribute messages to the scalable middleware.

In my architecture post I covered the proposed architecture of the Kraken Office system. In order to create a scalable, distributed system I am using messaging to break dependencies between components and to enable scalability of the middleware components. I have chosen the Redis data store technology as the message broker for the system, partially as Redis will also be used for data storage and also because it is extremely fast.

I had initially selected the Redback library to assist in integrating my Node.js server with Redis. On closer inspection, however, it turns out that this library is suitable for Redis persistence requirements but does not provide integration with the Redis publisher / subscriber messaging feature. I therefore looked for an alternative integration library that did provide access to the messaging features and eventually selected the node_redis project, which includes all of the features necessary for sending and receiving messages with Redis in a node.js context.

Installing And Starting Redis On Window

As I am working with Windows the first step is to download, install and start the Redis server. Redis is written for the Unix platform although several Windows ports of the Redis project exist. After looking around I eventually used the Windows downloads available here.

Redis doesn’t actually require any installation. In order to start the service you simply need to navigate to the download location and then double-click the redis-server.exe file. This starts the Redis service in a console window. Projects exist on the web for hosting Redis within a Windows Service – a quick search on the web will find them – although initially at least I am content to run Redis within a console window as it allows me to quickly and easily view the state of the server and any errors.

Redis can be configured by changing the setting in the redis.conf file. I won’t go through the various settings in here now other than to say that this is where the port settings can be found, enabling you to run multiple instances of Redis on different ports.

Installing The Node_Redis Library

Installing the Node_Redis library, enabling connectivity between Node.js and Redis, is straightforward. The library can be installed using the NPM package manager. Simply open a command window and use the CD command to navigate to the root installation of your node.js installation. Once you have done this you then use the following NPM command to install the library:

npm install redis

If you execute your node.js javascript driver files from the installation directory (usually Program Files) then the Node_Redis library is ready for use. If you execute your node.js scripts from a different location you will need to copy the node_modules folder from the installation directory to the directory in which your javascript node.js driver files reside. If you do not do this node.js will not be able to locate the correct library files.

Sending Messages To Redis From Node.js

Before using the Node_Redis library in your own node.js javascript driver files you will need to add a require statement to the top of your javascript file. This should be done as follows:

var redis = require("redis");

This ensures that node.js can locate the correct library files.

The next thing is to create an instance of the Redis client as follows:

var REDIS_URL = 'localhost';
var REDIS_PORT = 6379;

var redisClient = redis.createClient( REDIS_PORT, REDIS_URL );

Obviously the Url and Port variables will need to be set to those of your own Redis server. You can find the port on which your Redis server is running by referring to the Redis start-up trace information displayed in the console window.

To send a message to the broker you can now simply call:

redisClient.publish( 'channel_name','message_body' );

The ability to specify a channel enables multiple messaging channels to be open at any one time. A subscriber can the decide from which channel they are interested in receiving messages.

A real-world example would be:

redisClient.publish( "application_layer", "{client logon request message}" );

This shows how easy it is to publish messages to a Redis server from node.js.

Building on our echo web socket server described in setting up a web socket service with node.js on windows, below is code for an amended server which doesn’t echo messages back to the client but instead sends the message to a channel in Redis.

In order to run this web socket server in node.js you should:

  1.  Save the code to a file with a .js extension in the root of your node.js installation
  2. Open a command window and use the CD command to navigate to the node.js installation directory
  3. Use the node <filename> command to start the node.js server
#!/usr/bin/env nodevar WebSocketServer = require('websocket').server;
var http = require('http');
var redis = require("redis");

var REDIS_URL = 'localhost';
var REDIS_PORT = 6379;
// create a redis connection
try
{
 var redisClient = redis.createClient( REDIS_PORT, REDIS_URL );
}
catch (err)
{
 console.log( "ERROR => Cannot connect to Redis message broker: URL => " + REDIS_URL + "; Port => " + REDIS_PORT );
 console.log(err);
}

var server = http.createServer(function(request, response) {
 console.log((new Date()) + ' Received request for ' + request.url);
 response.writeHead(404);
 response.end();
});
server.listen(8080, function() {
 console.log((new Date()) + ' Server is listening on port 8080');
});

wsServer = new WebSocketServer({
 httpServer: server,
 // You should not use autoAcceptConnections for production
 // applications, as it defeats all standard cross-origin protection
 // facilities built into the protocol and the browser. You should
 // *always* verify the connection's origin and decide whether or not
 // to accept it.
 autoAcceptConnections: false
});

function originIsAllowed(origin) {
 // put logic here to detect whether the specified origin is allowed.
 console.log(origin);
 return true;
}

wsServer.on('request', function(request) {
 if (!originIsAllowed(request.origin)) {
 // Make sure we only accept requests from an allowed origin
 request.reject();
 console.log((new Date()) + ' Connection from origin ' + request.origin + ' rejected.');
 return;
 }

var connection = request.accept('kraken-protocol', request.origin);
 console.log((new Date()) + ' Connection accepted.');

connection.on('message', function(message) {

 if (message.type === 'utf8') {
 console.log('Received Message: ' + message.utf8Data);

// post the message to the redis message broker
 var channelName = 'application';
 redisClient.publish(channelName, message.utf8Data);

}
 });
 connection.on('close', function(reasonCode, description) {
 console.log((new Date()) + ' Peer ' + connection.remoteAddress + ' disconnected.');
 });
});
Subscribing To Messages From Redis In Node.js

Receiving messages from Redis in node.js is also straightforward.

Firstly you need to subscribe to a particular channel and then handle the message received event. The code below shows a simple example of how this works:

var redis = require("redis")

// create client on required port
var REDIS_URL = 'localhost';
var REDIS_PORT = 6379;
var redisClient = redis.createClient( REDIS_PORT, REDIS_URL );
client.on("message", function (channel, message) {

// message received - output to console window
 console.log("client channel => " + channel + "; message => " + message + ";");

}
});
// subscribe to receive messages from a particular channel
client.subscribe("application_channel");

If you need to handle messages based upon channel you will need to check this in the message event.

Conclusions

Using Redis as a message broker can help remove tight coupling in systems by removing direct communications between components. Using node.js enables a client to pass a message directly to a web socket server and then to distribute this via Redis to any number of subscribed middleware components, enabling scalable systems to be created.

The next step is to look into receiving messages from Redis in my .NET middleware components so this will be the focus of my next blog.

In the meantime – happy coding!