My last, long, post was concerned with the overall architecture of the Kraken Office project. I am now at a point where I am settled enough about the initial architecture of a duplex, message based web application that I can start selecting technologies for the project.
Most of my career has involved developing software for the Microsoft platform ( ASP, VB6, VB Script, ASP.NET, C#, SQL, Silverlight etc.) so I am naturally going to approach the problem with .NET as the technology for the development of the majority of the middleware, That is not to say that I won’t consider alternative technologies but this is to be my starting point.
The client facing user interface will be a web application. This is to maximise the audience for Kraken Office.
Having recent experience with Silverlight and the MVVM methodology, this would be a logical choice. I am becoming increasing concerned, however, about Silverlight’s usage as a internet facing technology due to its reliance on a plug-in. The fact that there is no plug-in support for most mobile devices is especially disconcerting. Another issue is Microsoft’s lack of vocal support for the technology which is currently worrying the entire Microsoft development community (see Microsoft developers horrified).
I want Kraken Office to appeal to as large an audience as possible and I don’t think this goal would be advanced by setting system pre-requisites or by excluding the mobile market.
The obvious alternative for creating a web facing interface with rich LOB features is HTML 5. While this is a relatively new technology and not yet universally supported across all browsers, the development community has generally welcomed it and take up has been good. Microsoft appear to have invested heavily in its usage in Windows 8, although the full details are yet to emerge.
It is important to remember, however, that the end product of the development process is a useful product with the biggest audience possible. If HTML 5 is the best way of achieving this goal then I feel this is the approach to take.
While HTML 5 may be the technology I also need to decide on a development environment. For me this is pretty easy decision – I will use the Microsoft MVC 3 platform for the website framework and will work within Visual Studio with which I have a lot of experience.
The last main issue is communication with the application server. I have used duplex communication via Duplex WCF services, but these utilise a long poll mechanism rather than a real push mechanism. Web sockets are the new HTML 5 technology for duplex communication and offer support for real duplex communication between server and client. Web sockets currently have limited browser support but are currently being implemented in all major new browser releases. For further information on web sockets refer to this interesting comparison of the pros and cons of long polling vs web sockets
Unfortunately web sockets are not yet supported in Microsoft’s WCF platform, although support is on the way at some stage (WCF WebSocket preview).
In view of the lack of WCF support my first thought was to write a web socket server in C#. I did some initial prototyping and developed a working web sockets server but found the solution verbose and complex. I also have reservations about network based coding in general as I have always found it one of the most difficult types of component to get stable and reliable. In addition to this I want to re-use as many existing frameworks and toolkits as possible, especially when it comes to project infrastructure, so that I can concentrate on developing business functionality.
I came across a new project called XSockets which has only just been released at the time of writing and which looks like it could offer WebSockets hosted in .NET. It was a brand new release, however, with very little documentation so has been discounted for now.
In conclusion the following sums up my initial web platform technology choices:
- HTML 5 developed in Visual Studio with MVC 3
- Node.js hosted web socket server using WebSocket-Node plugin
My initial system designs for Kraken Office, which can be found at Designing A Scalable, Resilient System, assumed that I would be creating a .NET based web sockets server. With my decision to implement the web socket server with node.js, this will now not be the case. What is clear, however, is that the node.js server will be the bridge between the client web application and the middleware. In order to ensure that the system remains scalable any communication between node.js and the middleware must use a message broker of some description.
While investigating message brokers, Enterprise Service Buses (ESBs) and in-memory data stores I came across a range of options. ESBs seem too heavy-weight for the system in mind, although at a later date such power may be required. Currently, however, I don’t want to spend all of my precious time resources on trying to configure a system that is not adding value to the project.
One of the most recommended, and fast, in-memory data caches is Redis. A Windows port of the project is available here. In addition to being immensely fast, it provides support for publisher / subscriber queues. Redis is open source and would enable me to kill two birds with one stone; to provide both in-memory data storage and message broker requirements in one system. This would minimise the amount of new technology I need to learn and also the amount of software installation and configuration.
Using Redis and node.js will necessitate a change to the architecture of the system. The diagram below documents v2 of the architecture.
The system now uses a scalable node.js layer as the initial application layer. This layer will manage web socket instances for the client web applications. The node.js layer will relay messages to the rest of the system via the Redis messaging layer. A .NET application layer will receive messages from the node.js layer and will decode and act upon the messages. If a message requires data interaction then the application layer will again submit this to the Redis messaging layer, in a different message channel. The data layer (another .NET based layer) will subscribe to data channel messages in Redis and will action any relevant messages. Data will be returned from the data layer via the Redis messaging layer back to the application layer and then onto the node.js layer.
This should prove to be a scalable and resilient system.
In conclusion the following sums up my initial middleware technology choices:
- .NET middleware layers (application and data) written in Visual Studio using C#
- Messaging provided by Redis and accessed by node.js and .NET clients using Redback and ServiceStack respectively
- Fast in-memory data caching using Redis
While data cached in Redis can be persisted to disk using a variety of configurable persistence strategies, I do not consider it suitable for a permanent data storage solution for two main reasons:
- Persisting data on a regular basis to disk, which would be necessary for data integrity, will degrade the performance of Redis
- The Kraken Office system will need to archive certain types of data but retain the ability to retrieve the data on demand, for which Redis would be unsuitable
As a result I am going to stick with my decision to persist data to a RDBMS for permanent storage, in addition to retaining data in Redis for real-time application usage. As I am most familiar with Microsoft SQL Server I will stick with this as an initial database, although I will investigate MySQL in the near future due to its Open Source status.