Sunday, October 24, 2010

Fleshing out a Remote User Interface in C#


Following my previous post where I threw some ideas out about how to architect a remote WPF user interface, I have now had the chance to flesh out more of the details. I will attempt to dig a little deeper into the implementation without getting bogged down with the details in this post. 

.NET Remoting is one solution to the general problem of creating client-side proxy objects that transparently represent their server-side instances. It is however not an ideal solution for rhobust systems because of all the gotchas associated with the service interface boundaries not being explicit.
We dont need such a generalized or complex solution as .Net Remoting because we are dealing with only one family of classes that conform to a distict pattern. We are therefore able to limit the communication/event propagation protocol to support only the functionality required for remoting View-Model objects that a WPF view could bind to.
These are the limitations of View-Model classes that we will conform to:
  • View-Model classes are data-only classes that implement INotifyPropertyChanged in order to inform the view when properties (or collections) change state. 
  • The only functional behaviour associated with these classes is achieved by providing properties that implement the ICommand interface.
  • All instances of objects are instantiated by the model, never by the view
Lets have a look at our very simple (but very functional) service interface then:


    [ServiceContract]
    public interface IProxyViewModelService
    {
        [OperationContract]
        Instance GetViewModel(string sessionId);

        [OperationContract]
        List<ViewModelUpdate> GetUpdates(string sessionId);

        [OperationContract]
        void SendUpdates(string sessionId, List<PropertyUpdate> updates);
       
        [OperationContract]
        void SendCommand(
string sessionId, CommandInvocation commandInvocation);
    }


You will notice that the client will poll for updates using the GetUpdates() request. In my experience it is much simpler to implement and get the permissions and routing to work out of the box. With a little attention to implementation detail we can even get it to be just as responsive and bandwidth-savvy as a callback service.  But that will have to be another post.

In this interface I have defined some classes that may need clarification:

Instance => The base class for all Server Instances that get sent over the wire.
ValueInstance => A value-type instance that holds an immutable object like a string or int
ObjectInstance =>A fully serialized object that exists on the server.
CollectionInstance => A fully serialized collection of either Value, or Object instances.
ReferenceInstance => Contains an Id that uniquely identifies each (object or collection) instance on both client and server.
PropertyUpdate => Holds the objectId of the owning object, propertyName, new and old Instance values.
CollectionUpdate => Holds the objectId of the owning collection, plus similar properties to the NotifyCollectionChangedEventArgs that WindowsBase provides.
ViewModelUpdate => the base class for all updates.

The server must be concerned with:
  • Serializing the current state of the view model object so it can be re-created on the client
  • Provide notification of property-updates and collection-modifications to all registered clients
  • Applying updates to properties from the clients
  • Keeping track of registrations
The client must be concerned with:
  • Reconstructing view model objects from the serialized form as provided by the server
  • Applying collection modifications and property updates as they are propagated from the server
  • Notifying the server of changes to properties that the client makes
  • Mocking ICommand interfaces to invoke service calls
 Here is a diagram showing an overview of how things are wired up.


I must explain that although the ViewModel class should conform strictly to the requirements I specified at the beginning of the article, there is one more requirement that I must mention. It must have a parameterless constructor so that the proxy object can be created on the client side. This poses a problem:  Often our server side objects have dependencies (on database repositories and the like) that should be passed in on the constructor. To support this scenario there needs to be some mechanism for the implementation of the View Model to inherit from a base class. It is the base class that defines the interface that the View requires, and it is the base class that will be de-serialized on the client.

We can see from the diagram above that the view can be programmed independently of whether it is a remote interface or not, except for the knowledge about whether the session/connection is still alive. A small consideration when trying to provide a WPF  UI to a Windows Service by any other means!

Tuesday, September 14, 2010

Remote User Interfaces in WPF

Recently I have been doing some windows service applications and have therefore needed to separate the UI into an additional application.

I would like to still take advantage of WPF and the view-model/model-view pattern for my remote interface. I need some kind of proxy object that is instantiated by the service and is duplicated on the UI process.
The UI needs to be able to show updates that have occurred on the service, and the service needs to be able to respond to updates from the UI.

So I've been thinking about how we can do that in a way that is fairly transparent to both the UI and service. I still want the UI to use data-binding, and the ICommand interface for invoking actions. I want to be able to share one well defined view-model between both the service and UI, and not have to do any over-the-wire specific logic in either the service or UI.

The goal is to minimize any work related to making the UI remote. Creating the UI and presenter/view-model should be almost as easy as working with a view-model in the same process. I say "almost" because it there are specific concerns related to working remotely that must be considered, but that concern should NOT overlap with the concern of the view-model/model-view pattern.
The question is "how can I distil this concern into a framework or set of base classes so it is no longer cross-cutting"

Option 1: The Remoting approach
With careful fine-tuning of lease parameters, you should be able to create a fairly reliable view-model derived from MarshalByRefObject, complete with two-way update notifications. The beauty of Remoting is that the proxy objects are all handled for you.
The problem is that because it is sooo transparent, a communication exception can happen just about on any call on the proxy object. Its very difficult to handle the exceptions, and even more difficult knowing how to recover. You also don't have any control of how chatty the events get. Configuring firewalls can be a problem too.
If the UI is only used for short periods of time, and the occasional restart-because-of-an-exception is bearable, Remoting may be an acceptable solution.

Option 2: The WCF approach
There are several hurdles to overcome:

Two-way communication.
This is needed to send notifications both ways. Subscribing to a callback service or event service is one option, but if you have ever tried to use this technique you will find it an absolute PITA to get working reliably. The other option is to use polling. With polling the question still remains: How often do I poll to make sure I:
  1. keep the user interface responsive, and
  2. not waste cpu time and network bandwidth when there is nothing to update.
I would suggest a polling strategy that removes this concern. The client polls as quickly as it is able to, but the server will block for a pre-arranged time if there are no messages. At any point during that timeout period a message arrives for delivery, the server will send the response immediately at that time. In this way all events that occur on the server will be delivered to the client instantaneously, even though polls are only occurring every 10 seconds!

How to create the proxy view-model
When it comes to creating a proxy object that reflects the state of the "real" object that exists on the server, it is always easier to proxy an interface. At least the top-level DataContext should be an interface, and then it would be helpful if everything the view-model referenced in it's interface was either another interface, or at least be serializable. These are pretty much the same constraints as the Remoting scenario, except using interfaces instead of MarshalByRefObject. Although the Wpf xaml engine does not require a well-defined interface (I mean interface in the broadest sense of the word), we will have to put in a little more work to make sure only what is important to the view gets duplicated over the wire.

Here is where I do something you may not be expecting. The view-model interface will NOT be the service interface.
I propose that the service interface be a generic and completely reusable interface that contains methods for replicating the viewModel over the wire, for notifying of property/collection changes, and for invoking commands.

We create a completely generic reusable service oriented library that is able to, by reflection
  1. Build up a tree of service objects that perfectly describes the state of the entire view-model so that it can be transferred over the wire and replicated at the UI end.
  2. Identify each object by a key (it's path in the tree?) so both server and client can notify of changes to that object
  3. Watch every watchable object in the view-model and queue the state changes so it can be picked up when the UI polls.
  4. Keep track of who the live clients are, with timeouts/ max queue size.
And from the UI (client) end:
  1. Dynamically construct a view-model from the tree it gets from the server upon registration.
  2. Transfer method calls on view-model interfaces into an equivalent service call to be invoked on the "real" model ( e.g. SaveCommand , RoutedCommand)
  3. Translate property changes and communicate them to the server as{objectId,propertyName,newValue}
  4. Poll for property changes (in background thread), and apply them to the replicated-model (in the dispatcher thread)
The UI will be aware of the remote nature of the view-model to the extent that it should be able to respond to the server being unavailable, in which case it must re-obtain the entire view-model upon successful reconnection. This behaviour could also be in the library, easily implemented by using a base class, or helper method.

Using this library it should be possible to split existing WPF applications into UI and service with the minimal amount of effort (assuming there is already a good separation in the code between UI and business logic).

Example
Consider a log-viewing application where you view the latest 500 log messages of a service.
The view-model looks something like:
interface ILogViewModel
{
ObservableCollection LogMessages{get;}
}
Here is a typical sequence of events:
  1. The client will recieve a tree representation of the entire viewModel (with all collection items) when it registers itself. It might use it's IP address as it's registration id.
  2. The client will poll (using it's registration id), and the server will block and wait for something to happen to the viewModel (namely the LogMessages collection getting modified)
  3. As soon as a log message comes in the server will reply with a status update (in this case it is a collectionChange{item inserted at 0} for the object identified "ILogViewModel\LogMessages")
  4. The client inserts the LogMessage according to the update it received
  5. The Wpf binding ensures that the view is updated also.

Tidy.

Monday, June 7, 2010

Should programming languages be extensible?


Introduction:

Modern object-oriented programming languages provide, at their most basic level abstraction (through interfaces), encapsulation (through private class members) , programming by difference (through inheritance/decoration), and composition (of data through containers and of logic through declaring sequencial operations) . These 4 concepts give us the power to divide a complex problem into more bite-sized chunks, providing knowledge barriers in order to hide information unrelated to the bite-sized piece the developer is working on at the time.

Modern languages also inherit some very important but non-object-oriented abstractions too. This is because some abstractions we would like to create do not naturally fit into the object-oriented model where data and logic are encapsulated together. Some examples are :
  • Purely functional abstractions (like mathematical operators)
  • Pure data abstractions (like DTO's)
  • Memory Management
  • Threading issues like deferred execution, locking, asynchronous calls
  • Aspect oriented programming / cross-cutting concerns
So we find that along with the pure object oriented syntax are many other language features that may (or may not) be useful depending on the domain. Language features like overloading, reflection, lambda expressions, checked exceptions, design by contract, dynamic types, memory management, and extension methods to name a few. No one language can incoorperate all these features because some of them are mutually exclusive, or specific to a narrow domain (like telephony, or web services for example). Some features will provide more constraints on how we are able to do things. Others add more syntax that makes the language less natural looking. Some features are focused on prototyping/productivity while others focus on reliability.

The more language features we include, the more complex the language becomes. This is why most popular languages only implement the features that increase productivity, and leave out the ones that create more reliable code. Besides, some of the abstractions can be massaged into an object oriented framework or library. The only problem with using the OO counterparts is:
1) The syntax does not have a natural language type flow
2) There is no compile time checking
3) They often rely heavily on reflection.. a feature which typically has costly performance overheads

This article explores the concept of giving the developer the ability to pick and choose language features, or even create new language features specific to their domain.

The way we do things currently

From the application developers point of view, the language he uses should be everything he needs to express the requirements in a non-ambiguous way. This is achieved by providing language layers, of which only the bottom layer is the actual programming language. Each layer can function as an extension of the layer below it, or it could hide the details of the layer below it. The former case is similar to inheritance, the later similar to encapsulation.

Here is an example of a typical web services application and it's layers
Application wiring (often represented with a diagram, or as a IOC-configuration file or web.config)
Features (The part that most closely represents the requirements specification)
Reuseable framework and library componentsDomain specific classes (encapsulate the business objects)
Compiler language

The way of the future?

I would like to suggest another layer just above the "Compiler language". I think the compiler language should be very simple and support very few features. Just above this layer should be a set of features that are part of the compilation process, providing language elements that are executed and verified at compile time.

In the same way we include namespaces in a class header file we should be able to import just the language features that are needed . This could be defined at the scope of the application, or more finely scoped to the namespace/class level. If we were able to do this, then the base language would be able to support any number of arbitrary language features, even ones that are mutually exclusive of each other.

One thing is for sure. If we had the power to create our own language features, we wouldn't be waiting for microsoft to bring out their next release of .Net before we start using generics(v2.0), link(v3.5), or dynamic types(v4.0)! It would also be very nice to not have to develop your modules in Java just because you want checked exceptions, or Eiffel just because you want design-by-contract.

So what are the roadblocks to providing this kind of extensibility? We have been creating applications that are reliable and extendable by using good OO techniques and test driven development for years. We can add modules in small units and verify the application works as expected. Is it too hard to apply the same principles to compiler design?

I'm looking for some feedback on that question.

The base-level-compiler would need to provide only a few language features, plus hookins for the feature extensions. The compiler could be re-compiled and retested with each new feature added, or it could provide a plugin type architecture.

Concrete examples of things you could do with an extensible compiler:

  • Language features to support state patterns, and then on top of that syntax like: until (some event) do {...}
  • Being able to extend the metadata of a class without modifying the class's definition.
  • Many frameworks and toolkits that use reflection , dynamic code generation, or code weaving can be re-written to do their work at compile time.

Roadblocks in C#

It seems that in C# many language features are hacks where the compiler needs explicit info on class definitions in order to do lots of "special case" compilation. For example [AttributeUsage] and [ThreadStatic] attributes. The compiler has specific knowledge of these attributes, which shows a dependency of the compiler on a compiled MSIL library??!! Other examples are :
  • foreach/yield -> depends on IEnumerable
  • using -> depends on IDisposable
  • try/catch -> depends on Exception
  • linq -> depends on IQueryable
  • dynamic -> depends on DynamicObject
I wonder if the reason the language is so hard to extend is because there are so many bad dependencies? It doesn't seem right that compiler code should be able to interpret the OO code in the layer above it! I guess it's similar to a domain object reading the web.config file to find it's configuration. If there is a using keyword, there should also be a dispose keyword, and you should be able to mark a method in a class as the dispose method. That way there is no dependency on the OO layer.

Writing code for processors with many cores

The code we write is not supposed to be a line by line count of what we want the cpu to do. It is merely a way of describing our business rules / logic in a way that is non-ambiguous.

How our code gets converted into machine code, (or an IL for that matter) is the job of the compiler / interpreter.

With multiple cpu's we may need to address an extra concern : distributing load across multiple processors. For any loop or repetitive action there may be several ways to achieve optimal performance depending on the data we may be working on.

People talk of using extra markup to define to the compiler which optimization scheme (if any) should be used. I say this is the wrong approach. It adds complexity to the piece of logic we are trying to describe, causing the programmer to hold too many concerns in their head at the same time.

The code we write should be only what is necessary to define the routine and no more. That is the best abstraction for code. No leaks please. Optimization should be a concern you address later in an orthogonal view, or using an external tool which extrapolates the routines that qualify for optimization, and then allows the user to choose which model to use.

Its a bit like one person can design the schema of the database, and another can add and remove indexes in order to address performance issues. (Its one separation of concerns that rdbms databases do well. Databases suck at providing other abstractions like inheritance and interfaces. Views are a great way to create a sub-set of functionality. I think the problem is that the database should be defined as the combination of all possible views rather than the views being a subset of the whole database. Its kind of like micro-kernel vs monolithic kernel. In the end the database administrator really needs to understand every table, rule, column and view if he is to
make any change to the schema of the database. Nasty. )

Better yet, (going back to the multi-processor optimization) you could have the application keep statistics and choose the best optimization at run time based on the data/conditions at the time. Anyway, at least we have the option to do all this if we separate the concern!

I suppose this approach can also be taken for locking on variables.
Identifying the possibility of race conditions,and dead locks... shouldn't all this be possible just by interogating the source? If a human can do it, a computer program could do it even easier, and with a lot more accuracy. Why then do we even bother to think about obtaining locks on sync objects? Why cant the compiler work out where its needed and let us FOCUS ON THE LOGIC/ BUSINESS RULES!

Whenever I hear "don't forget to do this.... don't forget to do that..." I think "We could be doing this much better!"

Tuesday, March 9, 2010

Inversable Functions

How many times have we written pairs of functions like this:

string urlEncode(string plaintext){ .. contents... }

string urlDecode(string encodedtext){ .. contents... }

Have you ever noticed that the code in the second function is just doing the inverse operations of the first , but in the reverse order?

That is redundant code in my mind. And redundant code has more opportunity to introduce bugs because I may update the first function and forget to update the second , or update the second incorrectly.

We should therefor have a general programming construct whereby we can mark certain functions as inversable. A function is automatically inversable if it is itself an ordered collection of inversable functions/operations.

In this way we should be able to use an inverse operator like so:

string url = urlEncode("52%");

Assert.AreEqual("52%",^urlEncode(url));

Or something like that anyway... Notice the ^ operator for doing the inverse.

Or maybe we could declare the name of the inverse function in the definition:

string urlEncode(string plaintext) : inverse urlDecode
{ .. contents... }

which generates a comiple error if the function is not actually inversable. Then we can call

Assert.AreEqual("52%",urlDecode(url));

Saturday, July 18, 2009

Introduction

I'm Andrew, a developer working predominantly with C#.

I've only really been in the game for about 3 yrs now, and I have a lot to learn, but I'm starting to feel like there's a lot we can do to improve the way we program.

I'm not a fantastic researcher, I'm more of "an ideas man" (think The Castle). There are lots of times I am looking for a solution to really common programming problem, and I just cant seem to find any solution that solves it in a really abstract and complete way.

The solutions we have (ie languages,toolkits,frameworks) provide useful abstractions, but are more often than not leaky, and nearly always incomplete. I say an incomplete abstraction nearly always increases the amount of knowledge required to understand a section of code.

In the mathematics world we have a beautiful abstraction called a matrix. If we can turn a set of simultaneous linear equations into a matrix, it represents a very useful and complete abstraction that we can do all sorts of simple operations on. Like find an inverse, or an eigen-vector. It takes some time to learn the abstraction, but by golly once you learn it you never have to look beyond the surface to see whats actually happening underneath.

As I've been working with C# 3.5, NHibernate, CAB (Composite Application Block) , ASP.Net, and the Microsoft MVC framework, I've been noticing some real holes, and being a person who is not content to just complain about things, I've decided I need to put some ideas out there on where the existing solutions don't cut the mustard, and some suggestions as to what we could do about it.

I hope that by putting out my ideas, and getting lots of criticism, that our communal understanding would be refined, and we will come up with some really useful stuff that might actually change the way we program!