var interval = Observable.Interval(TimeSpan.FromMilliseconds(500)); interval.Subscribe(i => Console.WriteLine("First: {0}", i)); interval.Subscribe(i => Console.WriteLine("Second: {0}", i));

At first glance it might look like I’m setting up two listeners to a single interval pulse, but what’s actually happening is that each time I call Subscribe, I’ve created a new timer to tick values. Imagine how bad this could be if instead of an interval I was, say, sending a message across the network and waiting for a response.

There are a few ways to solve this problem, but only one of them is actually correct. The first thing most people latch on to in Intellisense is Publish(). Now that method looks useful. So I might try:

var interval = Observable.Interval(TimeSpan.FromMilliseconds(500)).Publish(); interval.Subscribe(i => Console.WriteLine("First: {0}", i)); interval.Subscribe(i => Console.WriteLine("Second: {0}", i));

Now I get nothing at all. Great advertising there.

So what is actually happening here? Well, for one you might notice that interval is no longer IObservable<long> but is now an IConnectableObservable<long>, which extends IObservable with a single method: IDisposable Connect().

As it turns out, Publish is simply a convenience method for Multicast that supplies the parameter for you. Specifically, calling stream.Publish() is exactly the same as calling stream.Multicast(new Subject<T>()). Wait, a subject?

What Multicast does is create a concrete implementation of IConnectableObservable<T> to wrap the subject we give it and forwards the Subscribe method of the IConnectableObservable<T> to Subject<T>.Subscribe, so it looks something like this:

You might have noticed that the input doesn’t go anywhere. That’s exactly why our simple call to Publish() earlier didn’t produce any results at all – IConnectableObservable<T> hadn’t been fully wired up yet. To do that, we need to make a call to Connect(), which will subscribe our input into our subject.

Connect() returns to us an IDisposable which we can use to cut off the input again. Keep in mind the downstream observers have no idea any of this is happening. When we disconnect, OnCompleted will not be fired.

Getting back to my example, the correct code looks like this:

var interval = Observable.Interval(TimeSpan.FromMilliseconds(500)).Publish(); interval.Subscribe(i => Console.WriteLine("First: {0}", i)); interval.Subscribe(i => Console.WriteLine("Second: {0}", i)); var connection = interval.Connect(); // Later connection.Dispose();

It is very important to make sure all of your subscribers are setup before you call Connect(). You can think of Publish (or, really, Multicast) like a valve on a pipe. You want to be sure you have all your pipes together and sealed before you open it up, otherwise you’ll have a mess.

A problem that comes up fairly often is what do you do when you do not control the number or timing of subscriptions? For instance, if I have a stream of USD/EUR market rates, there’s no need for me to keep that stream open if nobody is listening, but if someone is, I’d like to share that connection, rather than create a new one.

This is where RefCount() comes in. RefCount() takes an IConnectableObservable<T> and returns an IObservable<Tg>, but with a twist. When RefCount gets its first subscriber, it automatically calls Connect() for you and keeps the connection open as long as anyone is listening; once the last subscriber disconnects, it will call Dispose on its connection token.

So now you might be wondering why I didn’t use RefCount() in my so-called “correct” implementation. I wouldn’t have had to call either Connect() or Dispose, and less is more, right? All that is true, but it omits the cost of safety. Once I dispose my connection, my source no longer has an object reference to my object, which allows the GC to do what it does best. Often, these streams start to make their way outside of my class, which can create a long dependency chain of object references. That’s fine, but if I dispose an object in the middle, I want to make sure that that object is now ready for collection, and if I RefCount(), I simply can’t make that assertion, because I’d have to ensure every downstream subscriber had also disposed.

Another scenario that comes up is how to keep a record of things you’ve already received. For instance, I might make a call to find all tweets with the hashtag “#RxNet” with live updates. If I subscribe second observer, I might expect that all the previously found data to be sent again without making a new request to the server. Fortunately, we have Replay() for this. It literally has 15 overloads, which cover just about every permutation of windowing by count and/or time, and supplying an optional scheduler and/or selector. The parameterless call, however, just remembers everything. Replay is just like Publish in the sense that it also forwards a call to Multicast, but this time with a ReplaySubject<T>.

Now the temptation is to combine Replay() and RefCount to make caches of things for subscribers when they are needed. Lets look at my Twitter example.

tweetService.FindByHashTag("RxNet").Replay().RefCount()

When the first observer subscribes, FindByHashTag will make a call (I assume this is deferred) to the network and start streaming data, and all is well. When the second observer subscribes, he gets all the previously found data and updates. Great! Now, let’s say both unsubscribe and a third observer then subscribes. He’s going to get all that previous data, and then the deferred call in FindByHashTag is going to be re-triggered and provide results that we might have already received from the replay cache! Instead, we should implement a caching solution that actually does what we expect and wrap it in an Observable.Create, or if we expect only fresh tweets, use Publish().RefCount() instead.

To take some shavings off of the top top of the iceberg that is why I use Powershell, I’d like to share a recent experience of Powershell delivering for me.

Setup for Failure

As a co-orgranizer for the New York .NET Meetup I was tasked with getting the list of attendees off of meetup.com and over to the lovely people at Microsoft who kindly give us space to meet. Now, you might think there’d be some nifty “export attendee list to CSV” function the meetup.com website, but like a lot of software, it doesn’t do half the things you think it should and this is one of those things. Usually my colleague David assembles the list, but this particular month he was out on vacation. He did, however, point me over to a GitHub repository of a tool that would do the extraction.

Following his advice, I grabbed the repository and brought up the C#/WinForms solution in Visual Studio. Looking at the project structure, I was a bit stunned at the scale of it all. The author had divided his concerns very well into UI, data access, the core infrastructure, and, of course, unit testing. I thought that was pretty peculiar considering all I wanted to do was get a CSV of names and such off of a website. Far be it for me to criticize another developer’s fastidiousness. Maybe it also launched space shuttles; you never know.

In what I can only now describe as naivety, I hit F5 and expected something to happen.

I was rewarded with 76 errors.

Right off the bat, I realized that the author had botched a commit and forgot a bunch of libraries. I was able to find NHibernate fairly easily with Nuget, but had no luck with “Rhino.Commons.NHibernate”. I tried to remove the dependency problem, but didn’t have much luck. And the whole time I was wondering why the hell you needed all these libraries to extract a damn CSV from the internet.

The Problem

Rather than throw more time after the problem, I decided to forge out on my own. Really, how hard could it be to

1. Get an XML doc from the internet
2. Extract the useful data
3. Perform some heuristics on the names
4. Dump a CSV file

Being a long-time C# programmer, my knee-jerk reaction was to build a solution in that technology. Forgoing a GUI to spend as little time as possible in building this, I’d probably build a single file design that ostensibly could consist of a single method. And if that were the case, why not script it?

So if I was going to write a script, what to use? I could write a JavaScript and run it on node.js, but it’s lacking proper CSV utilities and I’d have to run it on something other than my main Windows box. Not to mention I don’t particularly writing in JavaScript, so I’d probably write it in CoffeeScript and have to compile it, etc, etc.

I briefly considered writing an F# script, but I suspect only about ten people would know what on earth it was, and, at the end of the day, I would like to share my script to others.

The Solution

In the end, I concluded what I had known already: Powershell was the tool to use. It had excellent support for dealing with XML (via accelerators) and, as a real scripting language, had no pomp and circumstance.

Here’s the script I ended up writing:

function Get-MeetupRsvps([string]$eventId, [string]$apiKey) {$nameWord = "[\w-']{2,}" $regex = "^(?'first'$nameWord) ((\w\.?|($nameWord )+) )?(?'last'$nameWord)|(?'last'$nameWord), ?(?'first'$nameWord)( \w\.|( $nameWord)+)?$" function Get-AttendeeInfo {process {$matches = $null $answer = $_.answers.answers_item if(-not ($_.name -match $regex)) { $answer -match $regex | Out-Null }return New-Object PSObject -Property @{ 'FirstName' = $matches.first 'LastName' = $matches.last 'RSVPName' = $_.name 'RSVPAnswer' = $answer 'RSVPGuests' = $_.guests }} }$xml = [Xml](New-Object Net.WebClient).DownloadString("https://api.meetup.com/rsvps.xml?event_id=$eventId`&key=$apiKey") $xml.SelectNodes('/results/items/item[response="yes"]') `| Get-AttendeeInfo `| select FirstName, LastName, RSVPName, RSVPAnswer, RSVPGuests }

To dump this to a CSV file is then really easy:

Get-MeetupRsvps -EventId 1234 -ApiKey 'MyApiKey' | Export-Csv -Path rsvps.csv -NoTypeInformation

And because of this design, it’s really extensible. Potentially, instead of exporting to a CSV, you could pipe the information into another processor that would remove anyone named “Thorsten.” Actually, that would look like this:

Get-MeetupRsvps -EventId 1234 -ApiKey 'MyApiKey' `| ? { %_.FirstName -ne 'Thorsten' } ` | Export-Csv -Path rsvps.csv -NoTypeInformation

It’d be pretty difficult to do that if I’d written a C# executable – you’d have to go into Excel to do that. Or write a Powershell script. Just saying.

Here’s the real kicker: I spent all of five minutes writing my Powershell script and then spent minutes tweaking my regex to identify as many names as possible. I didn’t need to recompile, just F5 again in Poweshell ISE, which you have installed already if you’re on Windows 7. Since I left the Export-Csv part off during debugging, I could just read the console output and see what I got.

When I was happy with my output, it was dead simple to distribute: throw it in a GitHub Gist and move on with my life. If you decide to use it, all you need is Powershell installed (again, are you on Windows 7?) and the ability to copy and paste. No libraries. No worries. If you don’t like my regex, it couldn’t be easier to figure out how to replace it. If you want more fields, it’s easy to see where they should be added.

It’s really just that easy.

Introducing the Streams Pattern

First we need to create a “stream provider”. The purpose of this class is to establish a single place that “owns” the stream. I like to use the stream provider to establish default values and behavior:

public static class StreamNames {private const string Prefix = "Streams_"; public const string Accounts = Prefix + "Accounts"; public const string SelectedAccount = Prefix + "SelectedAccount"; }// No need to export this type. An instance will be // shared between the two child exports. public class AccountStreamProvider {[ImportingConstructor] public AccountStreamProvider(IAccountService accountService) {// Defer the query for the list of accounts until the first subscription var publishedAccounts = Observable .Defer(() => Observable.Return(accountService.GetAccounts())) .Publish(); publishedAccounts.Connect(); Accounts = publishedAccounts; SelectedAccount = new ReplaySubject(1); // Take the first account and set it as the initially selected account. Accounts .Take(1) .Select(Enumerable.FirstOrDefault) .Subscribe(SelectedAccount.OnNext); }[Export(StreamNames.Accounts)] public IObservable<IEnumerable<Account>> Accounts { get; set; }[Export(StreamNames.SelectedAccount)] [Export(StreamNames.SelectedAccount, typeof(IObservable<Account>)] public ISubject<Account> SelectedAccount { get;set; }}

Already the benefits of RX over the EventAggregator are showing.

Now we just need to get a reference to our exports in a relevant view model:

public static class Extensions {public static void DisposeWith(this IDisposable source, CompositeDisposable disposables) {disposables.Add(source); }} [Export, PartCreationPolicty(CreationPolicy.NotShared)] public class AccountViewModel : BindableBase, IDisposable {private readonly Streams _streams; private readonly CompositeDisposable _disposables; [Export] public class Streams {[Import(StreamNames.Accounts)] public IObservable<IEnumerable<Account>> Accounts { get; set; }[Import(StreamNames.SelectedAccount)] public ISubject<Account> SelectedAccount { get; set; }} [ImportingConstructor] public AccountViewModel(Streams streams) {_streams = streams; _disposables = new CompositeDisposable(); _streams .Accounts .Subscribe(a => Accounts = a) .DisposeWith(_disposables); _streams .SelectedAccount .Subscribe(a => SelectedAccount = a) .DisposeWith(_disposables); }private IEnumerable<Account> _accounts; public IEnumerable<Account> Accounts {get { return _accounts; }private set { SetProperty(ref _accounts, value, "Accounts"); }} private Account _selectedAccount; public Account SelectedAccount {get { return _selectedAcccount; }private set { SetProperty(ref _selectedAccount, value, "SelectedAccount", OnSelectedAccountChanged); }} // This method is only called when _selectedAccount // actually changes, so there's no indirect recursion. private void OnSelectedAccountChanged() {_streams.SelectedAccount.OnNext(_selectedAccount); }public void Dispose() {_disposables.Dispose(); }}

By the way, I’m using BinableBase from my INPC template.

Introducing the Config Pattern

If you’ve been using RX for a while, you might have noticed that I forgot to put my setters on the dispatcher. Sure, I could rely on automatic dispatching to fix that problem, but if my queries got any more complicated It’d be better for me to take care of the dispatch myself.

Of course, the problem with ObserveOnDispatcher is that it makes testing a huge pain. Fortunately, we can use MEF to get around that problem, too.

public class AccountViewModel : BindableBase, IDisposable {private readonly Streams _streams; private readonly Config _config; private readonly CompositeDisposable _disposables; [Export] public class Streams {[Import(StreamNames.Accounts)] public IObservable<IEnumerable<Account>> Accounts { get; set; }[Import(StreamNames.SelectedAccount)] public ISubject<Account> SelectedAccount { get; set; }} [Export] public class Config {public virtual IScheduler DispatcherScheduler { get { return Scheduler.Dispatcher; } } }[ImportingConstructor] public AccountViewModel(Streams streams, Config config) {_streams = streams; _config = config; _disposables = new CompositeDisposable(); _streams .Accounts .ObserveOn(_config.DispatcherScheduler) .Subscribe(a => Accounts = a ).DisposeWith(_disposables); _streams .SelectedAccount .ObserveOn(_config.DispatcherScheduler) .Subscribe(a => SelectedAccount = a) .DisposeWith(_disposables); }// ... }

Since the DispatcherScheduler property is virtual, it’s easy to mock it out. Using Moq, all you need to do is:

Mock.Of<AccountViewModel.Config>(m => m.DispatcherScheduler == Scheduler.Immediate)

So one of the things that I’ve been missing in my BindableBase class is property coercion, a la dependency properties. It’s a pretty smart idea; you can keep values in a valid state without pushing change notification. Unfortunately, there are some problems that crop up pretty quickly in INotifyPropertyChanged based view models.

Consider a Foo property that refuses to set a value less than zero.

private int _foo; public int Foo {get { return _foo; }set {if(_foo >= 0) SetProperty(ref _foo, value, "Foo"); }}

That looks like pretty good coercion, but the problem is that your binding and your property are now out of sync. That is, the binding told your object to set a value and assumed it did; you gave it no notification to the contrary. So while your object retains the old value, the bound TextBox will blissfully report -23.

A more brute force option would raise PropertyChanged in the event of this kind of coercion, but that breaks the code contract for INotifyPropertyChanged in that nothing changed. So how do we get around this problem?

If you take a look at the invocation list of your PropertyChanged event with some bound variables, you’ll notice that there’s a PropertyChangedEventManager hooked up.

Considering that the other item in the list is my default delegate, this object must be responsible for communicating my events to the binding system

Of course, the next thing to do is fire up Reflector and take a look at what’s there.

public class PropertyChangedEventManager : WeakEventManager {// Fields private WeakEventManager.ListenerList _proposedAllListenersList; private static readonly string AllListenersKey; // Methods static PropertyChangedEventManager(); private PropertyChangedEventManager(); public static void AddListener(INotifyPropertyChanged source, IWeakEventListener listener, string propertyName); private void OnPropertyChanged(object sender, PropertyChangedEventArgs args); private void PrivateAddListener(INotifyPropertyChanged source, IWeakEventListener listener, string propertyName); private void PrivateRemoveListener(INotifyPropertyChanged source, IWeakEventListener listener, string propertyName); protected override bool Purge(object source, object data, bool purgeAll); public static void RemoveListener(INotifyPropertyChanged source, IWeakEventListener listener, string propertyName); protected override void StartListening(object source); protected override void StopListening(object source); // Properties private static PropertyChangedEventManager CurrentManager { get; }}

Basically, AddListener access the private CurrentManager singleton and subscribes the given listener to OnPropertyChanged. Fortunately for us, there’s a flaw in th the implementation of OnPropertyChanged. What it does is it gets the list of listeners based on the sender and raises their event with the given sender and args. The problem here is that it doesn’t verify that the object raising the event is actually the sender! That is to say, we should be able to send a fake PropertyChanged event through another object acting as a surrogate. All we need to do is add that object to the PropertyChangedEvenManager’s list and start impersonating.

To that end, I added this private class to BindableBase to lazily add the INotifyPropertyChanged proxy object to the PropertyChangedEventManager. Since we’re not actually interested in the events, I made a stub implementation of IWeakEventListener that returns false constantly to indicate it’s not handling the event. Finally, I hold onto both of these references to keep them from being garbage collected.

private class PropertyChangedEventManagerProxy {// We need to hold on to these refs to keep it from getting GC'd private readonly NotifyPropertyChangedProxy _notifyPropertyChangedProxy; private readonly IWeakEventListener _weakEventListener; private PropertyChangedEventManagerProxy() {_notifyPropertyChangedProxy = new NotifyPropertyChangedProxy(); _weakEventListener = new WeakListenerStub(); PropertyChangedEventManager.AddListener(_notifyPropertyChangedProxy, _weakEventListener, string.Empty); }public void RaisePropertyChanged(object sender, string propertyName) {_notifyPropertyChangedProxy.Raise(sender, new PropertyChangedEventArgs(propertyName)); }private static PropertyChangedEventManagerProxy _instance; public static PropertyChangedEventManagerProxy Instance { get { return _instance ?? (_instance = new PropertyChangedEventManagerProxy()); } } private class NotifyPropertyChangedProxy : INotifyPropertyChanged {public event PropertyChangedEventHandler PropertyChanged = delegate { }; public void Raise(object sender, PropertyChangedEventArgs e) {PropertyChanged(sender, e); }} private class WeakListenerStub : IWeakEventListener {public bool ReceiveWeakEvent(Type managerType, object sender, EventArgs e) { return false; }} }

The only thing left to do is add the coerce value function as an optional parameter on SetProperty and hook it up:

protected void SetProperty<T>( ref T backingStore, T value, string propertyName, Action onChanged = null, Action<T> onChanging = null, Func<T, T> coerceValue = null) {VerifyCallerIsProperty(propertyName); var effectiveValue = coerceValue != null ? coerceValue(value) : value; if (EqualityComparer<T>.Default.Equals(backingStore, effectiveValue)) {// If we coerced this value and the coerced value is not equal to the original, we need to // send a fake PropertyChanged event to notify WPF that this value isn't what it thinks it is. if (coerceValue != null && !EqualityComparer<T>.Default.Equals(value, effectiveValue)) PropertyChangedEventManagerProxy.Instance.RaisePropertyChanged(this, propertyName); return; }if (onChanging != null) onChanging(effectiveValue); OnPropertyChanging(propertyName); backingStore = effectiveValue; if (onChanged != null) onChanged(); OnPropertyChanged(propertyName); }

And there you have it. All the benefits of dependency property coercion, and the same short, sweet SetProperty syntax.

As before, everything is on GitHub so you can take a look at the whole thing and fork it if you see something to be improved.

I want to inform you all that you’re doing it wrong.

The Problem

Most projects choose to implement INotifyPropertyChanged instead of extending DependencyObject for the sole reason that dependency properties are an enormous pain to write. Having worked on a large project that used the latter approach, I’ve entirely sworn it off, and I don’t care what kind of tooling you say makes life better. It’s just too complicated. On top of that, it’s really difficult to subscribe to changes programmatically. Sure you can go through DependencyPropertyDescriptor and subscribe yourself, but that’s a hell of a lot harder than +=. So that leaves some people with properties that look like this:

private int _myValue; public int MyValue {get { return _myValue; }set { _myValue = value; OnPropertyChanged("MyValue"); }}

Not bad, perhaps. It just doesn’t do much. What about INotifyPropertyChanging? How do I do things internally when that property changes other than subscribing to my own PropertyChanged event?

These things could almost be overlooked. The real kicker is that this isn’t actually a proper implementation of INotifyPropertyChanged. According to MSDN, the PropertyChanged event is described as, “Occurs when a property value changes,” and this code will raise an event even if the value doesn’t. Their implementation looks like this:

private string customerNameValue = String.Empty; public string CustomerName {get {return this.customerNameValue; }set {if (value != this.customerNameValue) {this.customerNameValue = value; NotifyPropertyChanged("CustomerName"); }} }

Aside from their code style (or lack thereof), allow me to point out a few things. First – and foremost – they’re comparing string equality with operators, no mere venial sin in my book. Secondly, this puts the onus of figuring out how to compare two types on the property writer ad-hoc, rather than using something more intelligent. Finally, does anyone else think that 17 lines is a few too many for a dirt simple property? It’s crazy.

The Trouble with Lambda

Some very smart programmers I know like to have their NotifyPropertyChanged method take an Expression<Func<TProp>> so you can do something like:

private int _myValue; public int MyValue {get { return _myValue; }set { _myValue = value; NotifyPropertyChanged(() => MyValue); }}

They say it improves type checking and the ease of refactoring. But as it turns out, it’s god-awful for performance, which can run you into a lot of problems if you don’t keep track of even moderate update bursts.

I’d almost be willing to accept the performance trade-off if it weren’t for the fact that the only place that’s calling it is inside the property setter. When you’re refactoring, I don’t see it as too much effort to go ahead and fix the string in the underlying property setter while you’re there; you have to fix the backing member anyway.

Really what we need is lambdas for subscription. All those places out in your code that subscribe to PropertyChanged and then you check the property name with a string! That’s the real refactoring nightmare. What I want to do is myViewModel.SubscribeToPropertyChanged(vm => vm.Foo, OnFooChanged);

The implementation isn’t that complex at all:

public static IDisposable SubscribeToPropertyChanged<TSource, TProp>( this TSource source, Expression<Func<TSource, TProp>> propertySelector, Action onChanged) where TSource : INotifyPropertyChanged {if (source == null) throw new ArgumentNullException("source"); if (propertySelector == null) throw new ArgumentNullException("propertySelector"); if (onChanged == null) throw new ArgumentNullException("onChanged"); var subscribedPropertyName = GetPropertyName(propertySelector); PropertyChangedEventHandler handler = (s, e) => {if (string.Equals(e.PropertyName, subscribedPropertyName, StringComparison.InvariantCulture)) onChanged(); }; source.PropertyChanged += handler; return Disposable.Create(() => source.PropertyChanged -= handler); }public static IDisposable SubscribeToPropertyChanging<TSource, TProp>( this TSource source, Expression<Func<TSource, TProp>> propertySelector, Action onChanging) where TSource : INotifyPropertyChanging {if (source == null) throw new ArgumentNullException("source"); if (propertySelector == null) throw new ArgumentNullException("propertySelector"); if (onChanging == null) throw new ArgumentNullException("onChanged"); var subscribedPropertyName = GetPropertyName(propertySelector); PropertyChangingEventHandler handler = (s, e) => {if (string.Equals(e.PropertyName, subscribedPropertyName, StringComparison.InvariantCulture)) onChanging(); }; source.PropertyChanging += handler; return Disposable.Create(() => source.PropertyChanging -= handler); }private static string GetPropertyName<TSource, TProp>(Expression<Func<TSource, TProp>> propertySelector) {var memberExpr = propertySelector.Body as MemberExpression; if (memberExpr == null) throw new ArgumentException("must be a member accessor", "propertySelector"); var propertyInfo = memberExpr.Member as PropertyInfo; if (propertyInfo == null || propertyInfo.DeclaringType != typeof(TSource)) throw new ArgumentException("must yield a single property on the given object", "propertySelector"); return propertyInfo.Name; }

The Best Solution

This leaves me with a pretty explicit spec for implementing INotifyPropertyChanged:

1. It will take no more lines than a basic backing field setter.
2. It will raise INotifyPropertyChanged and INotifyPropertyChanging.
3. It will only raise the properties if the value has changed.
4. it will provide an easy way to have internal property changed subscriptions.
5. It will not use Lambdas to identify the property.

My solution allows you to do this:

private int _myValue; public int MyValue {get { return _myValue; }set { SetProperty(ref _myValue, value, OnMyValueChanged, OnMyValueChanging); }} private void OnMyValueChanged() { } private void OnMyValueChanging(int newValue) { }

Isn’t that nice? SetProperty looks like this:

protected void SetProperty( ref T backingStore, T value, string propertyName, Action onChanged = null, Action onChanging = null) {VerifyCallerIsProperty(propertyName); if (EqualityComparer<T>.Default.Equals(backingStore, value)) return; if (onChanging != null) onChanging(value); OnPropertyChanging(propertyName); backingStore = value; if (onChanged != null) onChanged(); OnPropertyChanged(propertyName); }[Conditional("DEBUG")] private void VerifyCallerIsProperty(string propertyName) {var stackTrace = new StackTrace(); var frame = stackTrace.GetFrames()[2]; var caller = frame.GetMethod(); if (!caller.Name.Equals("set_" + propertyName, StringComparison.InvariantCulture)) throw new InvalidOperationException( string.Format("Called SetProperty for {0} from {1}", propertyName, caller.Name)) }

The default parameters on SetProperty allow you to specify the changed callback or the changing callbacks in any permutation with the usual syntax and the stack trace analysis happens at debug time to make sure that the string you provide actually represents the property it’s being called from.

So how do we update dependent properties? The one thing you don’t want to do is ever have external property changing logic directly in your set method. It’s better to call out to a OnMyPropertyChanging event and have that method update your dependent method. The syntax is a little verbose, but it’s the most readable and extensible way to do it:

private int _foo; public int Foo {get { return _foo; }set { SetProperty(ref _foo, value, "Foo", OnFooChanged); }} private int _bar; public int Bar {get { return _bar; }set { SetProperty(ref _bar, value, "Bar", OnBarChanged); }} private int _foobar; public int Foobar {get { return _foobar; }private set { SetProperty(ref _foobar, value, "Foobar"); }} private void OnFooChanged() { UpdateFoobar(); }private void OnBarChanged() { UpdateFoobar(); }private void UpdateFoobar() { Foobar = _foo + _bar; }

In general you should never call OnProeprtyChanged; that decision is best left to the property and its underlying helper.

I hope this clears the water  for some people new to WPF and provides reasons for the veterans to change their tune. If you think I’m wrong, feel free to try and prove it to me in the comments section. After all, duty calls.

The source and XML documentation for all this and a few tests is on github.

Naturally, the first thing I did was to Google the answer. The first result looked pretty good so I implemented that:

<Application.Resources> <ResourceDictionary> <ResourceDictionary.MergedDictionaries> <ResourceDictionary Source="/PresentationFramework.Aero;V4.0.0.0;31bf3856ad364e35;component/themes/aero.normalcolor.xaml" /> </ResourceDictionary.MergedDictionaries> </ResourceDictionary> </Application.Resources>

This worked great up until I started styling things. Any time I created a style for one of the system controls, it’d change the underlying style back to the original system them, rather than the one I wanted. It turns out that naturally styles are based on the system theme, rather than implicit style (for reasons we’ll get into later). So the way to base a theme off of the implicit style is like this:

<Style x:Key="GreenButtonStyle" TargetType="Button" BasedOn="{StaticResource {x:Type Button}}"> <Setter Property="Background" Value="Green"/> <Setter Property="Foreground" Value="White"/> </Style>

Maybe a little inconvenient, but it gets the job done:

All was well up until I needed to make a new style based off of the default theme. Like many WPF applications, we had some default styles defined to give our controls a clean, consistent look. But in my particular case, I needed to make a whole new visual “branch,” using the default theme as the “trunk.” If I used my BasedOn workaround, I’d just get my implicit style; if I didn’t I’d get that nasty classic theme.

The more I thought about this, the more I realized that all of these problems were being caused by the fact that I was applying a system theme (aero.normalcolor) as a style on top of the actual system theme, rather than actually changing it. So I set off on a journey in Reflector to find out how WPF picks the current theme. This sounds hard, and it’s actually a lot harder than it sounds. After a dozen or so hours (spread out over a few weeks) and a some guidance by a blog that got really close (unfortunately, my link is now dead), I found out that however WPF calls a native method in uxtheme.dll to get the actual system theme, then stores the result in MS.Win32.UxThemeWrapper, an internal static class (of course). Furthermore, the properties on the class are read only (and also marked internal), so the best way to change it was by directly manipulating the private fields. My solution looks like this:

public partial class App : Application {public App() {SetTheme("aero", "normalcolor"); }/// <summary> /// Sets the WPF system theme. /// </summary> /// <param name="themeName">The name of the theme. (ie "aero")</param> /// <param name="themeColor">The name of the color. (ie "normalcolor")</param> public static void SetTheme(string themeName, string themeColor) {const BindingFlags staticNonPublic = BindingFlags.Static | BindingFlags.NonPublic; var presentationFrameworkAsm = Assembly.GetAssembly(typeof(Window)); var themeWrapper = presentationFrameworkAsm.GetType("MS.Win32.UxThemeWrapper"); var isActiveField = themeWrapper.GetField("_isActive", staticNonPublic); var themeColorField = themeWrapper.GetField("_themeColor", staticNonPublic); var themeNameField = themeWrapper.GetField("_themeName", staticNonPublic); // Set this to true so WPF doesn't default to classic. isActiveField.SetValue(null, true); themeColorField.SetValue(null, themeColor); themeNameField.SetValue(null, themeName); }}

I call the method in the App constructor so it sets the values after WPF does its detection, but before the system theme is loaded for rendering.

Here’s what I got back:

Success!

There’s still a couple downsides to this approach. First, I can’t change the theme whenever I want to, and secondly, if the user changes their Windows theme while the application is running, the theme I set gets wiped out.

Wait a minute. If Windows can change the application theme, I can too!

Poking around a little more, I found out that when WPF wants to find a system resource, it also calls System.Windows.SystemResources.EnsureResourceChangeListener() which makes sure there’s a listener attached to the Windows message pump, ready to interpret events like a theme change.

[SecurityCritical, SecurityTreatAsSafe] private static void EnsureResourceChangeListener() {if (_hwndNotify == null) {HwndWrapper wrapper = new HwndWrapper(0, -2013265920, 0, 0, 0, 0, 0, "SystemResourceNotifyWindow", IntPtr.Zero, null); _hwndNotify = new SecurityCriticalDataClass<HwndWrapper>(wrapper); _hwndNotify.Value.Dispatcher.ShutdownFinished += new EventHandler(SystemResources.OnShutdownFinished); _hwndNotifyHook = new HwndWrapperHook(SystemResources.SystemThemeFilterMessage); _hwndNotify.Value.AddHook(_hwndNotifyHook); }}

The handler looks like this:

[SecurityTreatAsSafe, SecurityCritical] private static IntPtr SystemThemeFilterMessage(IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled) {WindowMessage message = (WindowMessage) msg; switch (message) {// abridged case WindowMessage.WM_THEMECHANGED: SystemColors.InvalidateCache(); SystemParameters.InvalidateCache(); OnThemeChanged(); InvalidateResources(false); break; }return IntPtr.Zero; }

So the plan was to remove the hook so the message pump couldn’t change the theme out from under me and then call the handler whenever I want. As it turns out, it’s even easier than that. Because the list of hooks is implemented with a WeakReferenceList, all I need to do is null out the private _hwndNotifyHook field. There’s just one problem here: killing the handler prevents us from receiving system notifications for the other messages like device changes, power updates, etc. So we really need to inject a filter into the pipeline and pass along everything except the ThemeChanged event.

Fair warning: if you get queasy around reflection, you should probably stop reading here.

public static class ThemeHelper {private const BindingFlags InstanceNonPublic = BindingFlags.Instance | BindingFlags.NonPublic; private const BindingFlags StaticNonPublic = BindingFlags.Static | BindingFlags.NonPublic; private const int ThemeChangedMessage = 0x31a; private static readonly MethodInfo FilteredSystemThemeFilterMessageMethod =typeof(ThemeHelper).GetMethod("FilteredSystemThemeFilterMessage", StaticNonPublic); private static readonly Assembly PresentationFramework =Assembly.GetAssembly(typeof(Window)); private static readonly Type ThemeWrapper =PresentationFramework.GetType("MS.Win32.UxThemeWrapper"); private static readonly FieldInfo ThemeWrapper_isActiveField =ThemeWrapper.GetField("_isActive", StaticNonPublic); private static readonly FieldInfo ThemeWrapper_themeColorField =ThemeWrapper.GetField("_themeColor", StaticNonPublic); private static readonly FieldInfo ThemeWrapper_themeNameField =ThemeWrapper.GetField("_themeName", StaticNonPublic); private static readonly Type SystemResources =PresentationFramework.GetType("System.Windows.SystemResources"); private static readonly FieldInfo SystemResources_hwndNotifyField =SystemResources.GetField("_hwndNotify", StaticNonPublic); private static readonly FieldInfo SystemResources_hwndNotifyHookField =SystemResources.GetField("_hwndNotifyHook", StaticNonPublic); private static readonly MethodInfo SystemResources_EnsureResourceChangeListener =SystemResources.GetMethod("EnsureResourceChangeListener", StaticNonPublic); private static readonly MethodInfo SystemResources_SystemThemeFilterMessageMethod =SystemResources.GetMethod("SystemThemeFilterMessage", StaticNonPublic); private static readonly Assembly WindowsBase =Assembly.GetAssembly(typeof(DependencyObject)); private static readonly Type HwndWrapperHook =WindowsBase.GetType("MS.Win32.HwndWrapperHook"); private static readonly Type HwndWrapper =WindowsBase.GetType("MS.Win32.HwndWrapper"); private static readonly MethodInfo HwndWrapper_AddHookMethod =HwndWrapper.GetMethod("AddHook"); private static readonly Type SecurityCriticalDataClass =WindowsBase.GetType("MS.Internal.SecurityCriticalDataClass`1") .MakeGenericType(HwndWrapper); private static readonly PropertyInfo SecurityCriticalDataClass_ValueProperty =SecurityCriticalDataClass.GetProperty("Value", InstanceNonPublic); /// <summary> /// Sets the WPF system theme. /// </summary> /// <param name="themeName">The name of the theme. (ie "aero")</param> /// <param name="themeColor">The name of the color. (ie "normalcolor")</param> public static void SetTheme(string themeName, string themeColor) {SetHwndNotifyHook(FilteredSystemThemeFilterMessageMethod); // Call the system message handler with ThemeChanged so it // will clear the theme dictionary caches. InvokeSystemThemeFilterMessage(IntPtr.Zero, ThemeChangedMessage, IntPtr.Zero, IntPtr.Zero, false); // Need this to make sure WPF doesn't default to classic. ThemeWrapper_isActiveField.SetValue(null, true); ThemeWrapper_themeColorField.SetValue(null, themeColor); ThemeWrapper_themeNameField.SetValue(null, themeName); }public static void Reset() {SetHwndNotifyHook(SystemResources_SystemThemeFilterMessageMethod); InvokeSystemThemeFilterMessage(IntPtr.Zero, ThemeChangedMessage, IntPtr.Zero, IntPtr.Zero, false); }private static void SetHwndNotifyHook(MethodInfo method) {var hookDelegate = Delegate.CreateDelegate(HwndWrapperHook, FilteredSystemThemeFilterMessageMethod); // Note that because the HwndwWrapper uses a WeakReference list, we don't need // to remove the old value. Simply killing the reference is good enough. SystemResources_hwndNotifyHookField.SetValue(null, hookDelegate); // Make sure _hwndNotify is set! SystemResources_EnsureResourceChangeListener.Invoke(null, null); // this does SystemResources._hwndNotify.Value.AddHook(hookDelegate) var hwndNotify = SystemResources_hwndNotifyField.GetValue(null); var hwndNotifyValue = SecurityCriticalDataClass_ValueProperty.GetValue(hwndNotify, null); HwndWrapper_AddHookMethod.Invoke(hwndNotifyValue, new object[] { hookDelegate }); }private static IntPtr InvokeSystemThemeFilterMessage(IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, bool handled) {return (IntPtr)SystemResources_SystemThemeFilterMessageMethod.Invoke(null, new object[] { hwnd, msg, wParam, lParam, handled }); }private static IntPtr FilteredSystemThemeFilterMessage(IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled) {if (msg == ThemeChangedMessage) return IntPtr.Zero; return InvokeSystemThemeFilterMessage(hwnd, msg, wParam, lParam, handled); }}

What a mess! What’s really annoying is that this entire process could have (and, I argue, should have) been made public to user code. In the very least, I’d like to be able to decorate my assembly with an attribute declaring which theme I’d like to run under, but I think it’s completely reasonable to have an public static class provide the same API I have here.

The whole test suite is available here on GitHub.

DM F#, Clojure, and Scala are all fairly new and popular languages this decade, the former two with striking resemblance to OCaml and Lisp, respectively, and the lattermost being more original in syntax. In what way do these languages represent forward thinking in language design, or how do they fail to build upon lessons learned in more venerable languages like Haskell?

KT It’s a common story in the world of software that time brings increasingly precise and accurate programs.  Anybody who grew up playing games on an Atari 2600 has witnessed this firsthand.  The image of a character has changed from one block, to five blocks, to fifty blocks, to (eventually) thousands of polygons.  By this modern analog of the Greeks’ method of exhaustion, Mario’s facial structure has become increasingly clear.  This inevitable progression, squeezed out between the increasing sophistication of programmers and the decreasing punishment of Moore’s Law, has primarily operated at three levels: the images produced by game programs, the logic of the game programs themselves, and finally the programming languages with which programs are produced.  It’s this last group that deserves special attention today.

The lambda calculus (and its myriad derivatives) exemplifies this progression at the level of programming languages.  In the broadest terms, you have the untyped lambda calculus at the least-defined end (which closely fits languages like Lisp, Scheme, Clojure, Ruby and Python), and the calculus of constructions at the most-defined end (which closely fits languages like Cayenne and Epigram).  With the least imposed structure, you can’t solve the halting problem, and with the most imposed structure you (trivially) can solve it.  With the language of the untyped lambda calculus, you get a blocky, imprecise image of what your program does, and in the calculus of constructions you get a crisp, precise image.

Languages like F#, Scala and Haskell each fall somewhere in between these two extremes.  None of them are precise enough to accept only halting programs (although Haskell is inching more and more toward a dependently-typed language every year).  Yet all of them are more precise than (say) Scheme, where fallible human convention alone determines whether or not the “+” function returns an integer or a chicken.  But even between these languages there is a gulf of expressiveness.  Where a language like C is monomorphically-typed (and an “int_list” must remain ever distinguished from a “char_list”), F# introduces principled polymorphic types (where, pun intended, you can have “’a list” for any type ‘a).  Beyond that, Haskell (and Scala) offer bounded polymorphism so that constraints can be imposed (or inferred) on any polymorphic type – you can correctly identify the “==” function as having type “Eq a => a -> a -> Bool” so that equality can be determined only on those types which have an equivalence relation, whereas F# has to make do with the imprecise claim that its “=” function can compare any types.

No modern programming language is perfect, but the problems that we face today and the history of our industry points the way toward ever more precise languages.  Logic, unreasonably effective in the field of computer science, has already set the ideal that we’re working toward.  Although today you might hear talk that referential transparency in functional languages makes a parallel map safe, tomorrow it will be that a type-level proof of associativity makes a parallel reduce safe.  Until then, it’s worth learning Haskell (or Scala if you must), where you can count the metaphorical fingers on Mario’s hands, though not yet the hairs of his moustache.

DM One might assume that increased “precision” in a language would come at the cost of increased complexity in concepts and/or syntax. In a research scenario, the ability to solve the halting problem certainly has its merits, but is that useful for modern commercial application development, and, if so, does it justify the steeper learning curve?

KT It’s absolutely true that languages that can express concepts more precisely also impose a burden on programmers, and that a major part of work in language design goes into making that burden as light as possible (hence type-inference, auto roll/unroll for recursive types, pack/unpack for existential types, etc).

However — to your point about the value of that increased precision – I would argue that it’s even more important in commercial application development than in academia.  For example, say you’ve just rolled out a new pricing server for a major client.  Does it halt?  There’s a lot more riding on that answer than you’re likely to find in academia.  And really, whether or not it halts is just one of the simplest questions you can ask.  What are its time/space characteristics?  Can it safely be run in parallel?  Is it monotonic?  In our business, these questions translate into dollars and reputation.  Frankly I think it’s amazing that we’ve managed to go on this long without formal verification.

DM Most dynamic languages have some air of functional programming to them, while still being fundamentally imperative. Even more rigorous imperative languages like C# are picking up on a more functional style of programming. The two languages you mentioned earlier, Cayenne and Epigram, are both functional languages. Are we moving toward a pure functional paradigm, or will there continue to be a need for imperative/functional hybrids?

KT What is a “functional language”?  If it’s a language with first-class functions, then C is a functional language.  If it’s a language that disallows hidden side-effects, then Haskell isn’t a functional language.  I think that, at least as far as discussing the design and development of programming languages is concerned, it’s well worth getting past that sort of “sales pitch”.

I believe that we’re moving toward programming languages that allow programmers to be increasingly precise about what their programs are supposed to do, up to the level of detail necessary.  The great thing about referential transparency is that it makes it very easy to reason about what a function does — it’s almost as easy as high school algebra.  However, if you’re not engaged in computational geometry, but rather need to transfer some files via FTP, there’s just no way around it.  You’ve got to do this, then this, then this, and you’ll need to switch to something more complicated, like Hoare logic, to reason about what your program is doing.  Even there, you have plenty of opportunity for precision — you expect to use hidden side-effects but only of a certain type (transfer files but don’t launch missiles).

But maybe the most important tool in programming is logic itself.  It’s a fundamental fact – well known in some circles as the “Curry-Howard isomorphism” – that programs and proofs are equivalent in a very subtle and profound way.  This fact has produced great wealth for CS researchers, who can take the results painstakingly derived by logicians 100 years ago and (almost mechanically) publish an outline of their computational analog.  Yet, although this amazing synthesis has taken place rivaling the unification of electricity and magnetism, most programmers in industry are scarcely aware of it.  It’s going to take some time.

I think there’s a good answer to your question in that correspondence between logic and programming.  The function, or “implication connective” (aka “->”), is an important tool and ought to feature in any modern language.  As well, there are other logical connectives that should be examined.  For example, conjunction is common (manifested as pair, tuple, or record types in a programming language), but disjunction (corresponding to variant types) is less common though no less important.  Negation (corresponding to continuations consuming the negated type), predicated quantified types, and so on.  The tools for building better software are there, but we need to work at recognizing them and putting them to good use.

Anybody interested in understanding these logical tools better should pick up a copy of Benjamin Pierce’s book Types and Programming Languages.

DM Many frameworks have a one or more DSLs to express things as mundane as configuration to tasks as complicated as UI layout, in an effort to be more express specific kinds of ideas more concisely. Do you see this as an expanding part of the strategy for language designers to increase expressiveness in code? Is it possible that what we consider “general purpose languages” today will become more focused on marshalling data from one DSL to another, or will DSLs continue to remain a more niche tool?

KT I guess it depends on what you mean by “DSL”.  Like you say, some people just have in mind some convenient shorthand serialization for data structures (I’ve heard some people refer to a text format for orders as a DSL, for example).  I’m sure that will always be around, and there’s nothing really profound about it.

On the other hand, by “DSL” you could mean some sub-Turing language with non-trivial semantics.  For example, context-free grammars or makefiles.  Modern programming languages, like Haskell, are often used to embed these “sub-languages” as combinator libraries (the “Composing Contracts” paper by Simon Peyton-Jones et al is a good example of this).  I think it’s likely that these will continue as valuable niche tools.  If you take monadic parser combinators for example, it’s very attractive the way that they fit together within the normal semantics of Haskell, however you’ve got to go through some severe mental gymnastics to determine for certain what the space/time characteristics of a given parser will be.  Contrast that with good old LALR(1) parsers, where if the LR table can be derived you know for certain what the space/time characteristics of your parser will be.

On the third hand, if a “DSL” is a data structure with semantics of any kind, your description of a future where programs are written as transformations between DSLs could reasonably describe the way that compilers are written today.  Generally a compiler is just a sequence of semantics-preserving transformations between data structures (up to assembly statements).  I happen to think that’s a great way to write software, so I hope it’s the case that it will become ever more successful.

I’ve broken each code sample into independent blocks wherein all necessary dependencies are contained, so you can take any extension method a la carte or you can get everything from the attached zip file. My solution was built in .NET 4.0 in Visual Studio 2010, but everything should work just fine in .NET 3.5 with Visual Studio 2008.

Also included in the zip file are my unit tests, which may help you understand usage of some of the more esoteric extensions, such as ChainGet, and XML comments for your IntelliSense and XML documentation generator.

Edit: The whole solution is now available on github!

Here’s the table of contents, so you can jump around more easily:

• IEnumerable
  • ForEach
  • Append
  • Prepend
  • AsObservable
  • AsHashSet
  • ArgMax
  • ArgMin
• ICollection
  • AddAll
  • RemoveAll
• IDictionary
  • Add
  • AddAll
  • Remove
  • RemoveAll
  • RemoveAndClean
  • RemoveAllAndClean
  • Clean
• Object
  • As
  • AsValueType
  • ChainGet
• DependencyObject
  • SafeGetValue
  • SafeSetValue

IEnumerable

ForEach is pretty straightforward. It mimics List<T>.ForEach, but for all IEnumerable, both generic and weakly typed.

public static void ForEach<T>( this IEnumerable<T> collection, Action<T> action) {if (collection == null) throw new ArgumentNullException("collection"); if (action == null) throw new ArgumentNullException("action"); foreach (var item in collection) action(item); }

public static void ForEach( this IEnumerable collection, Action<object> action) {if (collection == null) throw new ArgumentNullException("collection"); if (action == null) throw new ArgumentNullException("action"); foreach (var item in collection) action(item); }

Table of Contents

Append and Prepend simply take an item and return a a new IEnumerable<T> with that item on the end or beginning, respectively. Prepend is the equivalent of the cons operation to a list.

public static IEnumerable<T> Append<T>( this IEnumerable<T> collection, T item) {if (collection == null) throw new ArgumentNullException("collection"); if (item == null) throw new ArgumentNullException("item"); foreach (var colItem in collection) yield return colItem; yield return item; }

public static IEnumerable<T> Prepend<T>( this IEnumerable<T> collection, T item) {if (collection == null) throw new ArgumentNullException("collection"); if (item == null) throw new ArgumentNullException("item"); yield return item; foreach (var colItem in collection) yield return colItem; }

Table of Contents

AsObservable and AsHashSet yield their respective data structures, but check to see if they are already what you want, saving valuable time when dealing with interfaces.

public static ObservableCollection<T> AsObservable<T>( this IEnumerable<T> collection) {if (collection == null) throw new ArgumentNullException("collection"); return collection as ObservableCollection<T> ?? new ObservableCollection<T>(collection); }

public static HashSet<T> AsHashSet<T>(this IEnumerable<T> collection) {if (collection == null) throw new ArgumentNullException("collection"); return collection as HashSet<T> ?? new HashSet<T>(collection); }

Table of Contents

ArgMax and ArgMin are corollaries to Max and Min in the System.Linq namespace, but return the item in the list that produced the highest value from Max or least value from Min, respectively.

public static T ArgMax<T, TValue>( this IEnumerable<T> collection, Func<T, TValue> function) where TValue : IComparable<TValue> {return ArgComp(collection, function, GreaterThan); }private static bool GreaterThan<T>(T first, T second) where T : IComparable<T> {return first.CompareTo(second) > 0; }public static T ArgMin<T, TValue>( this IEnumerable<T> collection, Func<T, TValue> function) where TValue : IComparable<TValue> {return ArgComp(collection, function, LessThan); }private static bool LessThan<T>(T first, T second) where T : IComparable<T> {return first.CompareTo(second) < 0; }private static T ArgComp<T, TValue>( IEnumerable<T> collection, Func<T, TValue> function, Func<TValue, TValue, bool> accept) where TValue : IComparable<TValue> {if (collection == null) throw new ArgumentNullException("collection"); if (function == null) throw new ArgumentNullException("function"); var isSet = false; var maxArg = default(T); var maxValue = default(TValue); foreach (var item in collection) {var value = function(item); if (!isSet || accept(value, maxValue)) {maxArg = item; maxValue = value; isSet = true; }} return maxArg; }

Table of Contents

ICollection

AddAll imitates List<T>.AddRange.

public static void AddAll<T>( this ICollection<T> collection, IEnumerable<T> additions) {if (collection == null) throw new ArgumentNullException("collection"); if (additions == null) throw new ArgumentNullException("additions"); if (collection.IsReadOnly) throw new InvalidOperationException("collection is read only"); foreach (var item in additions) collection.Add(item); }

Table of Contents

RemoveAll imitates List<T>.RemoveAll. A second overload allows you to specify the removals if you already have them.

public static IEnumerable<T> RemoveAll<T>( this ICollection<T> collection, Predicate<T> predicate) {if (collection == null) throw new ArgumentNullException("collection"); if (predicate == null) throw new ArgumentNullException("predicate"); if (collection.IsReadOnly) throw new InvalidOperationException("collection is read only"); // we can't possibly remove more than the entire list. var removals = new List<T>(collection.Count); // this is an O(n + m * k) operation where n is collection.Count, // m is removals.Count, and K is the removal operation time. Because // we know n >= m, this is an O(n + n * k) operation or just O(n * k). foreach (var item in collection) if (predicate(item)) removals.Add(item); foreach (var item in removals) collection.Remove(item); return removals; }

public static void RemoveAll<T>( this ICollection<T> collection, IEnumerable<T> removals) {if (collection == null) throw new ArgumentNullException("collection"); if (removals == null) throw new ArgumentNullException("removals"); if (collection.IsReadOnly) throw new InvalidOperationException("collection is read only"); foreach (var item in removals) collection.Remove(item); }

Table of Contents

IDictionary

All of these methods have to do with hash tables. I use them pretty frequently and these methods are able to make life a lot easier.

Add and AddAll insert the key and a new collection into the dictionary if the key doesn’t already exist and then adds the item or items to the collection.

public static void Add<TKey, TCol, TItem>( this IDictionary<TKey, TCol> dictionary, TKey key, TItem item) where TCol : ICollection<TItem>, new() {if (dictionary == null) throw new ArgumentNullException("dictionary"); if (key == null) throw new ArgumentNullException("key"); TCol col; if (dictionary.TryGetValue(key, out col)) {if (col.IsReadOnly) throw new InvalidOperationException("bucket is read only"); }else dictionary.Add(key, col = new TCol()); col.Add(item); }

public static void AddAll<TKey, TCol, TItem>( this IDictionary<TKey, TCol> dictionary, TKey key, IEnumerable<TItem> additions) where TCol : ICollection<TItem>, new() {if (dictionary == null) throw new ArgumentNullException("dictionary"); if (key == null) throw new ArgumentNullException("key"); if (additions == null) throw new ArgumentNullException("additions"); TCol col; if (!dictionary.TryGetValue(key, out col)) dictionary.Add(key, col = new TCol()); foreach (var item in additions) col.Add(item); }

Table of Contents

Remove and RemoveAll simply remove items from the collection associated with the specified key, if there is one. For the predicate overloads, you need to explicitly construct your Predicate<T> delegate because the C# compiler has trouble doing the type inference.

public static void Remove<TKey, TCol, TItem>( this IDictionary<TKey, TCol> dictionary, TKey key, TItem item) where TCol : ICollection<TItem> {if (dictionary == null) throw new ArgumentNullException("dictionary"); if (key == null) throw new ArgumentNullException("key"); TCol col; if (dictionary.TryGetValue(key, out col)) col.Remove(item); }

public static void RemoveAll<TKey, TCol, TItem>( this IDictionary<TKey, TCol> dictionary, TKey key, IEnumerable<TItem> removals) where TCol : ICollection<TItem> {if (dictionary == null) throw new ArgumentNullException("dictionary"); if (key == null) throw new ArgumentNullException("key"); if (removals == null) throw new ArgumentNullException("removals"); TCol col; if (dictionary.TryGetValue(key, out col)) foreach (var item in removals) col.Remove(item); }

public static IEnumerable<TItem> RemoveAll<TKey, TCol, TItem>( this IDictionary<TKey, TCol> dictionary, TKey key, Predicate<TItem> predicate) where TCol : ICollection<TItem> {if (dictionary == null) throw new ArgumentNullException("dictionary"); if (key == null) throw new ArgumentNullException("key"); if (predicate == null) throw new ArgumentNullException("predicate"); var removals = new List<TItem>(); TCol col; if (dictionary.TryGetValue(key, out col)) {foreach (var item in col) if (predicate(item)) removals.Add(item); foreach (var item in removals) col.Remove(item); }return removals; }// Usage: Dictionary<int, List<int>> myDictionary; myDictionary.RemoveAll(4, new Predicate<int>(i => i < 42));

Table of Contents

RemoveAndClean and RemoveAllAndClean both remove items from the collection associated with the specified key and if the resulting collection is empty, they remove the key from the dictionary as well. These come in both predicate and list forms.

public static void RemoveAndClean<TKey, TCol, TItem>( this IDictionary<TKey, TCol> dictionary, TKey key, TItem item) where TCol : ICollection<TItem> {if (dictionary == null) throw new ArgumentNullException("dictionary"); if (key == null) throw new ArgumentNullException("key"); TCol col; if (dictionary.TryGetValue(key, out col)) {col.Remove(item); if (col.Count == 0) dictionary.Remove(key); }}

public static void RemoveAllAndClean<TKey, TCol, TItem>( this IDictionary<TKey, TCol> dictionary, TKey key, IEnumerable<TItem> removals) where TCol : ICollection<TItem> {if (dictionary == null) throw new ArgumentNullException("dictionary"); if (key == null) throw new ArgumentNullException("key"); if (removals == null) throw new ArgumentNullException("removals"); TCol col; if (dictionary.TryGetValue(key, out col)) {foreach (var item in removals) col.Remove(item); if (col.Count == 0) dictionary.Remove(key); }}

public static IEnumerable<TItem> RemoveAllAndClean<TKey, TCol, TItem>( this IDictionary<TKey, TCol> dictionary, TKey key, Predicate<TItem> predicate) where TCol : ICollection<TItem> {if (dictionary == null) throw new ArgumentNullException("dictionary"); if (key == null) throw new ArgumentNullException("key"); if (predicate == null) throw new ArgumentNullException("predicate"); var removals = new List<TItem>(); TCol col; if (dictionary.TryGetValue(key, out col)) {foreach (var item in col) if (predicate(item)) removals.Add(item); foreach (var item in removals) col.Remove(item); if (col.Count == 0) dictionary.Remove(key); }return removals; }// Usage: Dictionary<int, List<int>> myDictionary; myDictionary.RemoveAll(4, new Predicate<int>(i => i < 42));

Table of Contents

Clean simply goes through all the keys and removes entries with empty collections.

public static void Clean<TKey, TCol>(this IDictionary<TKey, TCol> dictionary) where TCol : ICollection {if (dictionary == null) throw new ArgumentNullException("dictionary"); var keys = dictionary.Keys.ToList(); foreach (var key in keys) if (dictionary[key].Count == 0) dictionary.Remove(key); }

Table of Contents

Object

As casts an object to a specified type, executes an action with it, and returns whether or not the cast was successful.

public static bool As<T>(this object obj, Action<T> action) where T : class {if (obj == null) throw new ArgumentNullException("obj"); if (action == null) throw new ArgumentNullException("action"); var target = obj as T; if (target == null) return false; action(target); return true; }

Table of Contents

AsValueType does the same thing as As, but for value types.

public static bool AsValueType<T>(this object obj, Action<T> action) where T : struct {if (obj == null) throw new ArgumentNullException("obj"); if (action == null) throw new ArgumentNullException("action"); if (obj is T) {action((T)obj); return true; }return false; }

Table of Contents

ChainGet attempts to resolve a chain of member accesses and returns the result or default(TValue). Since TValue could be a value type, there is also an overload that has an out parameter indicating whether the value was obtained.

Be careful with this extension. Since it uses reflection, it’s a bit slow. For discrete usage, each call is under 1 ms, but if you use it in a loop with many items, the performance hit will become more tangible.

public static TValue ChainGet<TRoot, TValue>( this TRoot root, Expression<Func<TRoot, TValue>> getExpression) {bool success; return ChainGet(root, getExpression, out success); }public static TValue ChainGet<TRoot, TValue>( this TRoot root, Expression<Func<TRoot, TValue>> getExpression, out bool success) {// it's ok if root is null! if (getExpression == null) throw new ArgumentNullException("getExpression"); var members = new Stack<MemberAccessInfo>(); Expression expr = getExpression.Body; while (expr != null) {if (expr.NodeType == ExpressionType.Parameter) break; var memberExpr = expr as MemberExpression; if (memberExpr == null) throw new ArgumentException( "Given expression is not a member access chain.", "getExpression"); members.Push(new MemberAccessInfo(memberExpr.Member)); expr = memberExpr.Expression; }object node = root; foreach (var member in members) {if (node == null) {success = false; return default(TValue); }node = member.GetValue(node); }success = true; return (TValue)node; }private class MemberAccessInfo {private PropertyInfo _propertyInfo; private FieldInfo _fieldInfo; public MemberAccessInfo(MemberInfo info) {_propertyInfo = info as PropertyInfo; _fieldInfo = info as FieldInfo; }public object GetValue(object target) {if (_propertyInfo != null) return _propertyInfo.GetValue(target, null); else if (_fieldInfo != null) return _fieldInfo.GetValue(target); else throw new InvalidOperationException(); }} // Usage: var myValue = obj.ChainGet(o => o.MyProperty.MySubProperty.MySubSubProperty.MyValue);

Table of Contents

DependencyObject

These extensions work just like their non-safe counterparts on DependencyObject, but will call Dispatcher.Invoke to do operations if the current thread isn’t a UI thread.

public static object SafeGetValue( this DependencyObject obj, DependencyProperty dp) {if (obj == null) throw new ArgumentNullException("obj"); if (dp == null) throw new ArgumentNullException("dp"); if (obj.CheckAccess()) return obj.GetValue(dp); var self = new Func <DependencyObject, DependencyProperty, object> (SafeGetValue); return Dispatcher.Invoke(self, obj, dp); }

public static void SafeSetValue( this DependencyObject obj, DependencyProperty dp, object value) {if (obj == null) throw new ArgumentNullException("obj"); if (dp == null) throw new ArgumentNullException("dp"); if (obj.CheckAccess()) obj.SetValue(dp, value); else {var self = new Action <DependencyObject, DependencyProperty, object> (SafeSetValue); Dispatcher.Invoke(self, obj, dp, value); }}

public static void SafeSetValue( this DependencyObject obj, DependencyPropertyKey key, object value) {if (obj == null) throw new ArgumentNullException("obj"); if (key == null) throw new ArgumentNullException("key"); if (obj.CheckAccess()) obj.SetValue(key, value); else {var self = new Action <DependencyObject, DependencyPropertyKey, object> (SafeSetValue); Dispatcher.Invoke(self, obj, key, value); }}

Table of Contents

Part II. On the Phone

Sunday evening, January 18, I decided it might be a good idea to brush up on my .NET framework knowledge to prepare for my interview the next morning. Judging by the latter questions of Lab49’s “preliminary screening test,” these guys really didn’t mess around. I pulled off my bookshelf my trusty copy of CLR via C#, which is, in my opinion, the best book you can read if you really want to take your understanding of C# and .NET from “intermediate” to “expert”. C#  Developers: no excuses, read this book cover to cover. As it turns out, my interviewer, Nick, must be a fan of the same book. When he called me that Monday morning, after introducing himself, Nick threw me a couple softballs before turning up the heat. I was queried at length about generics, delegates, anonymous methods, and the garbage collector (among other things), all of which I was more than happy to explicate in the greatest of detail, having refreshed myself on their inner workings the night before. Nick’s attention then turned to the newer .NET 3.5 features, which I had been using for almost two years, and I was more than happy to talk about those, too. I must admit, he stumped me on a concept called “attached behaviors”. I was familiar with attached properties, but it wasn’t until recently that I’ve become fully aware of attached behaviors. I’ll have another article discussing what I learned in the future.

After Nick finished grilling me for information, I had my turn to ask him questions. I seem to remember having a list of things to talk about, but I was suffering from some strange variant of vertigo, so I went with my usual developer talking points. For the record, Nick is one of the nicest guys ever. As I would find out later, Lab49 is composed solely of superb people. You may be thinking that I’m generalizing or hyperbolizing, but in all seriousness, I have yet to find a single bad apple or even mildly distasteful person at Lab49. Every time I think I’ve found one, they prove me wrong. Even the Java guys are top notch, and that’s saying something. In any case, I finished the interview enjoying a discussion of the usual programmer minutiae, talking about podcasts and developer philosophy. I’m not sure if it’s normal for one to feel a sense of camaraderie with his interviewer, but I know I sure did.

Later that day, I received an email from James, a Recruitment Coordinator at Lab49, asking for a “telephone conversation” with Nemo, the Director of Recruitment. I figured it was one of those psychological profiles one of my friends had been subjected to in a recent interview. I don’t think I could have been more wrong. The next morning, Nemo called, introduced himself, asked for clarification on a few points of my résumé, and opened himself up for questions. I asked him the usual questions on how Lab49 was structured, the promotion strategy, and what Lab49 does in general.

Nemo explained to me that Lab49 is somewhat loosely structured, with no real middle management. As projects start and finish, you report to the project manager and engagement manager, but every project is custom-tailored to the clients’ needs. I pressed Nemo to explain how a successful company works without the infrastructure almost every company of equal size has. Nemo couldn’t really explain how it worked, but only that it did, and quite well. Some may say, “when the cat’s away,” but I might interject that maybe without the threat of imminent death, a mouse might be more free to do something more constructive with its time than cower for its life.

Coming from a consulting background, I could really tell Lab49 really is a consulting firm in the greatest sense of the term. The answer to everything is, “it depends,” and, “what the client needs,” which is clearly working out well for them, but can be frustrating trying to pin down something concrete on which to make a decision. In general, Labbers work on-site with the clients to best make use of human and information resources. In my opinion, there’s definitely a beneficial side effect to it: when Lab49 is working side-by-side with the client, the work has a face, rather than some unknown bunch of people dropping code in a folder every week or two.

Nemo continued by telling me that Lab49 has titles out of necessity, but they don’t play as much of a role as in other companies. I really appreciated that. Being a young guy, it’s typically difficult for me to get my ideas out in a space where I’ve got a title that’s easily negligible. I can say with authority that’s not been the case at Lab49. Every day I work with guys who have decades of experience on me, but are interested to hear what I have to say. It’s not about seniority; it’s about being the best at what you do and bringing new and innovative ideas to the table. For that very reason, I prefer to keep my title to myself. There’s no real company policy on the publicity of your title, but it’s not on my business card and I certainly wouldn’t wear it on my sleeve.

Nemo concluded the interview by saying that Daniel Chait, one of the founders of Lab49 would want to talk to me over Skype before an in-person interview. That was the first time anyone had said anything about an in-person interview, and while I had expected it at some point, I couldn’t help but be excited to see some light at the end of the tunnel. I confidently told Nemo that I was available for the remainder of the day, which might have caught him a bit off-guard. He said Daniel was a busy guy and he’d see when he was available. I was confirmed for later that afternoon by email not too long after hanging up.

I’m not sure if it was then or later that I started feeling extremely skeptical of this Lab49 place. I remember telling my friend, Chris, that, “I know Nirvana Corp burned down, but Lab49 makes me wonder,” quoting from an episode of Dilbert, The Animated Series.  I found it difficult to believe that such a place could exist, so little was my faith in humanity, let alone developers.

I then realized I was about to have my fourth communiqué with a company in three business days, whereas almost no Dallas-based company had insomuch as replied in a week.  These guys really don’t mess around, I remember thinking.

At 3:00 PM and after a few quirky Skype issues, I was on webcam with Daniel Chait, now fully appreciating the extra money I’d spent on a nice webcam. Chait went over my experience thoroughly, fully examining the roles I had played on each team. I’m not sure why, but at times I felt very intimidated. It’s perplexing because going over the conversation in my head Daniel wasn’t in the least bit condescending nor indirect in his questions. We discussed my work in WPF and LINQ and lambda expressions. Having taken Advanced Programming Languages and feeling like (but by no means) an expert in functional languages, I was glad to be on more solid ground. I had a compulsion to share with Daniel a couple extension methods I had written to succinctly state an equation in lambda. Unfortunately, since it was a compulsion I wasn’t prepared at all with the code snippet. I brought up my Machine Learning projects folder, which I knew it was in but couldn’t remember exactly which project I had written the extension for. Fumbling around, I must have spent a full five minutes of awkward silence finding the thing, which seemed anti-climactic for such a long wait, and even more so for me, since it felt like an eternity.

Afterward, I told my roommate that I wasn’t sure how that interview had gone. It was without a doubt by my estimation my weakest interview, and likewise the most important. I spent a little while going over the events, trying to find out why I felt I had done so poorly. All I can say is that I feel very confident of my technical skills and far less strongly about my experience, which was a major focus of Chait’s interview. That and that awkward silence.  I told myself that if they wanted a guy with more experience, that was a perfectly legitimate reason not to go forward, and I shouldn’t be worried about it. And if they rejected me outright for not having that bit of code on hand, they could go to Hell.

I was glad some of my friends wanted to go out that night so I could get my mind off the whole thing. When I got back, there was an email from James waiting for me, inviting me to New York for an in-person interview. I slept well that night.

The next day I made reservations airfare and lodging and ran both by James, as Lab49 kindly picked up the tab. It might be standard operating procedure for companies, but the fact is that they don’t have to do it, and I would have paid for the ticket myself if they hadn’t. Now all that was left to do was wait out the week and familiarize myself as best I could with my travel plans.

Clearly this company had their act together and was vigorously pursuing this discovery period. I was also reacquainting myself with the ever-greater possibility of leaving everything for New York. My father was supportive, and because he’s not usually too keen on my adventures, I took it for a good sign. That being the case, I started warming my close family and friends up to the idea that I might end up in New York. Most of them were surprised, marveling at the apparent spontaneity and haste in the whole process. Only months before had I told them I’d be staying in Dallas for the “foreseeable future” as a consultant writing software for my own company.

Who knew the foreseeable future was so short?

This semester I’ve finally qualified to take the CS Senior Design Project, which, this semester, is to design a Chord client. At the heart of the system is the identifier, which is essentially a glorified hash. We decided to go with SHA-256, partially because the instructor mentioned it was in the class of his favorite hashes. It didn’t really matter to me, since C# supports it just as easily as anything else.

Identifiers have two essential purposes, comparison and addition, from which every method can be derived. I wasn’t particularly intrigued by either of these topics, since almost without exception, you can rely on existing structures in the FCL to do whatever you need to do, and it’s always better to do so. Now, I said “almost without exception,” namely because I stumbled upon the fact that these identifiers unequivocally are exceptions. The thing about using a SHA-256 hash – or any hash for that matter – is that it’s an enormous value – 256 bits to be precise. There aren’t any real 32 or even 64 bit hashes available (I doubt they’d be useful), so relying on the good ol’ FCL goes out the window.

Other students, stuck in the abyss of Java, were having difficulty getting their socket implementations going, so mucking around in massive integer arithmetic was right out. I believe the professor advised them to take the 32 MSB and just rely on an unsigned integer. (Side note: does Java even have unsigned integers?) I considered a few similar options: taking the 64 LSB or separating the hash into several equal groups of 64-bits each and getting the xor result of the segments. Arrogantly I proceeded on, demanding that whatever I wrote work not only for SHA-256 but also SHA-512 or anything else using the full bit-length.

Comparison wasn’t difficult. Essentially, you start at the MSB in your array (which happens to be at Length - 1) and compare until you find a comparison != 0:

public int CompareTo(Identifier other) {for (int i = _hash.Length - 1; i >= 0; i--) {int compare = _hash[i].CompareTo(other._hash[i]); if (compare != 0) return compare; }return 0; }

Now comes the fun part. First, I wrote a little method called Pad(byte[] list, int size) which pads the MSB of an array with 0′s until it’s the proper length. After several iterations, I finally ended up with this:

public static byte[] Add(byte[] a, byte[] b) {if (a.Length > b.Length) b = Pad(b, a.Length); else if (a.Length < b.Length) a = Pad(a, b.Length); byte[] result = new byte[a.Length]; int carry = 0; for (int i = 0; i < a.Length; i++) {int value = carry + a[i] + b[i]; // get the bottom part of the bits result[i] = (byte)value; // knock off the bottom bits that we already got and // put the rest into carry. carry = (value - result[i]) / byte.MaxValue; }return result; }

The data holder for carry has to be large enough for the worst case scenario 0xFF + 0xFF + carry 1, which is why I chose an int rather than a byte (although a short could have worked just as well, I suppose).

Subtraction was more difficult. Consider the way you subtract, with carrying, etc. Carrying on its own is a recursive process, and I can’t imagine writing it out. It was at that time that I remembered how the ALU does subtraction- by taking the two’s compliment of the second operand and doing addition. So I followed suit:

public static byte[] Subtract(byte[] a, byte[] b) {if (a.Length > b.Length) b = Pad(b, a.Length); else if (a.Length < b.Length) a = Pad(a, b.Length); byte[] twoscomp = new byte[b.Length]; // get the 1's compliment for (int i = 0; i < b.Length; i++) twoscomp[i] = (byte)(byte.MaxValue - b[i]); // add 1twoscomp = Add(twoscomp, new byte[] { 1 }); // NOTE: twoscomp is now the two's compliment of b. return Add(a, twoscomp); }

I also needed multiplication, but I’m still working on the solution. I’ve got a working draft, but I’m not entirely sure I’m happy with it just yet. I’ll keep you notified.