Pages

October 26, 2010

Side Effect Of Type Constraints in .NET

Experimenting With Generic Constraints


As I was doing some experiments as exercise in reminding myself the difference between virtual and non-virtual method overloading behavior in C#, and stumbled on an unexpected side-effect of the constraint mechanism in Generics.

I have class B inherit from A, and wanted to write a generic test harness that would invoke methods f1 and f2 on an instance passed in, but with a generic capability to have the runtime cast the instance to the most specific type. So if I passed in B, the variable would be manipulated as B, etc. So I thought a generic method would deal with this quite nicely: void Do<T>(T arg) { arg.f1(); arg.f2; }. Now, the declaration will not compile as is, the compiler insists to know the type of T so that it can figure out whether the class supports f1(), f2() functions, fine, i go ahead and add where T : A to the signature, which turns out to introduce the side effect which is the subject of this blog entry.

When I implemented this scheme (see code below) I was surprised that even when invoking Do<B>(bInstance); , bInstance was treating as if it was A.

So the generic behavior is very different from what one would thing to be logical, and in fact C++ template mechanism does do what one expects if you were to implement a similar method, bInstance will be treated as a B.

using System;
using System.Text;

namespace SimpleMethodInheritanceTest
{
    class A
    {
        internal virtual void f1() { System.Console.WriteLine("f1 in A"); }
        internal void f2() { System.Console.WriteLine("f2 in A"); }
    }

    class B : A
    {
        internal override void f1() { System.Console.WriteLine("f1 in B"); }
        internal void f2() { System.Console.WriteLine("f2 in B"); }
    }

    class Program
    {
        static void DoBase(A a)
        {
            System.Console.WriteLine("======================== DoBase ========================");
            a.f1();
            a.f2();
        }

        static void DoBBB(B a)
        {
            System.Console.WriteLine("======================== DoBBB ========================");
            a.f1();
            a.f2();
        }


        static void Do<T>(T a) where T : A
        {
            System.Console.WriteLine("======================== Do<T> ========================");
            var specificClass = a.GetType().Name;
            System.Console.WriteLine("======================== Do<T>: T = {0}", specificClass);
            a.f1();
            a.f2();
        }
        static void Main(string[] args)
        {
            var a = new A();
            DoBase(a);
            Do<A>(a);
            var b = new B();
            DoBase(b);
            Do<B>(b);
            DoBBB(b);
            System.Console.ReadKey();
        }
    }
}

Output


I expected line 14 to say f2 in B, since T = B, so I was expecting the argument to Do<T>() to be treated as a B. But instead I get f2 in A!!!
======================== DoBase ========================
f1 in A
f2 in A
======================== Do<T> ========================
======================== Do<T>: T = A
f1 in A
f2 in A
======================== DoBase ========================
f1 in B
f2 in A
======================== Do<T> ========================
======================== Do<T>: T = B
f1 in B
f2 in A
======================== DoBBB ========================
f1 in B
f2 in B

Discussion



So we would expect D: T = B to have f2 in B, but we got f2 in A instead. So to investigate as to what might be going on under the covers, I decompiled the executable into IL using redgate's reflector.

Let's take a look at the results below.

Do<T> Dissassembled



If we dissasemble D<T> we see that on Line 25 f2 is explicitly invoked on class A. This is causes the unexpected results of f2 in A.

.method private hidebysig static void Do<(SimpleMethodInheritanceTest.A) T>(!!T a) cil managed
{
    .maxstack 2
    .locals init (
        [0] string specificClass)
    L_0000: nop 
    L_0001: ldstr "======================== Do<T> ========================"
    L_0006: call void [mscorlib]System.Console::WriteLine(string)
    L_000b: nop 
    L_000c: ldarga.s a
    L_000e: constrained !!T
    L_0014: callvirt instance class [mscorlib]System.Type [mscorlib]System.Object::GetType()
    L_0019: callvirt instance string [mscorlib]System.Reflection.MemberInfo::get_Name()
    L_001e: stloc.0 
    L_001f: ldstr "======================== Do<T>: T = {0}"
    L_0024: ldloc.0 
    L_0025: call void [mscorlib]System.Console::WriteLine(string, object)
    L_002a: nop 
    L_002b: ldarga.s a
    L_002d: constrained !!T
    L_0033: callvirt instance void SimpleMethodInheritanceTest.A::f1()
    L_0038: nop 
    L_0039: ldarga.s a
    L_003b: constrained !!T
    L_0041: callvirt instance void SimpleMethodInheritanceTest.A::f2()
    L_0046: nop 
    L_0047: ret 
}

DoBBB Disassembled



The helper method that specifies class B as the explicit type of argument, works as expected (calling B::f2()). As we can see below in line 12, compiler emits IL that specifies B::f2 explicitly.
.method private hidebysig static void DoBBB(class SimpleMethodInheritanceTest.B a) cil managed
{
    .maxstack 8
    L_0000: nop 
    L_0001: ldstr "======================== DoBBB ========================"
    L_0006: call void [mscorlib]System.Console::WriteLine(string)
    L_000b: nop 
    L_000c: ldarg.0 
    L_000d: callvirt instance void SimpleMethodInheritanceTest.A::f1()
    L_0012: nop 
    L_0013: ldarg.0 
    L_0014: callvirt instance void SimpleMethodInheritanceTest.B::f2()
    L_0019: nop 
    L_001a: ret 
}

October 15, 2010

CloudCamp NYC 2010



Cloud computing, and more specifically effecient virtualization is a very interesting topic.

Here is an opportunity to mingle and learn more if you are in NYC, on October 21st, 2010.

It's downtown, near Wall St.







App Engine For The Enterprise?

I like writing software, not the process of deploying it. The idea of abstracting away configuration, hosting and complex network setup has always been very appealing to me. This is one of the big promises of cloud computing; so I have been looking at and playing various cloud technologies as they have been popping up over the last 3-4 years.

Recently I have started using Google App Engine for non-trivial web apps. My experience so far has been a pleasure! You may say: What!? that's not a very precise description, true, but this is not meant to be a thorough analysis or comparison of every feature that one can think of or use in a real-world applications. What I can say, is that it's been painless to do all the regular things that you have to during a standard application development and deployment process. The API's are elegantly simple to learn and to use, and make it easy for you to start handling requests, persisting data, doing background processing, all on a scalable platform that can handle from 1 to thousands concurrent requests (at least in theory).

At the same time, having worked at a number of large corporations I have seen a number of good and bad attempts at streamlining the deployment processes and abstracting away architecture complexities. Again, I have mostly been in the end user role, not the implementor of the abstraction/automation frameworks and systems, so my view may be biased/limited/etc, but at least it represents an experience of a typical developer at a large organization. Typically it's been a rather arduous experience. Some firms are better than others, but in almost all cases you have to pay lots of attention to fairly low level architecture of your application. You can't just write a simple script and deploy it into the cloud.
Given my recent experiments with the App Engine, and extensive experience with large enterprise IT environments, I have starting thinking about how cloud computing can apply to the enterprise.

Low Level Infrastructure ------> High Level API


At the low level, you have to worry about how you are going to scale your application and the exact technology stack needed for your particular application. This is essentially what is being offered by various virtual machine cloud providers: Amazon EC2, Rackspace Cloud, and others. You have to figure out what VM you want to run (ie pick OS: linux flavor, windows), and what goes inside the VM (mysql/postgres/couchdb/other db, apache, lighthttpd, j2ee, IIS), how many nodes you want to have for failover and scalability, how many tiers you need to scale, etc. This gives you ultimate flexibility, but requires a LOT of extra architecture work. This comes at the expense of agility.

At the mid level of the spectrum, you have something like Windows Azure. You get to use familiar development tools and components (SQL, asp.net). But you still have to think about the architecture: how many worker/web/db nodes you want to have, some design of the schema.

At the highest level of architecture abstraction you will find Google App Engine. As a developer on this platform you don't have to worry about configuring VMs, figuring out the number and size of logical tiers. You are able to focus directly on solving your business problem. To Be Continued...

October 14, 2010

MSBuild Environment Sandboxing

Sandboxing 101

In any non trivial, team oriented project you have multiple configurations:
  • team-member-one-sandbox, team-member-two-sandbox
  • QA
  • UAT
  • PROD-Americas, PROD-Europe, PROD-Asia
  • Etc
For each environment you want to configure database connection strings, web services, various limits and runtime settings; all possibly located across multiple configuration files. It is also convinient to reuse some sane defaults for a lot of the settings, and have the option to override some settings for a specific environment. Conceptually you want to have the following hierarchy of settings, where the most nested overrides or adds to the properties defined at the previous level:
  • Defaults
    • PROD-Defaults
      • Americas
      • Europe
      • Asia

.NET/MSBuild Solution

There are a couple of choices available:
  • T4
  • MSBuild
  • NAnt
  • Custom PostBuild event scripts in Visual Studio
  • An AdHoc combination of the above
When we were designing the solution described in this post, we wanted to keep the process as simple and as integrated with Visual Studio as possible, while at the same time making it simple to do automated builds from the command line. In the end, we were able to combine the Configuration functionality within with Visual Studio, with targeting of specific environments in a fairly natural way; if you follow this approach you will be able to select a target environment within VS via the configuration drop-down. At a very high level you will have the following: 1. One or more templated configuration files with ${setting1} for various configurable settings. For example you will have a bunch of settings in your app.config. 2. A set of configuration files containing the settings that need to be replaced in your templated files: The proposed solution will allow you to define your settings in the following configuration files:
  • defaults.cfg
  • prod-americas.cfg
  • prod-europe.cfg
  • prod-asia.cfg
3. A couple of custom MSBuild tasks in your CoolApp.csproj file. We will utilize open source build task library: http://msbuildtasks.tigris.org/ 4. A couple of custom Configurations set up within Visual Studion (these go into CoolApp.sln file)

App.config

<configuration>
  <appSettings>
    <add key="setting_one" value="${setting_one}"/>
    ${custom_settings}
  </appSettings>
</configuration>

defaults.cfg

<settings>
  <x key="setting_one">Hello World From DEFAULTS!</x>
</settings>

debug.cfg (ie dev/qa/uat/etc)

<settings>
  <x key="custom_settings">
    <add key="custom_setting_1" value="This is DEBUG custom setting ONE."/>
    <add key="custom_setting_2" value="This is DEBUG custom setting TWO."/>
  </x>
</settings>

release.cfg

<tokens>
<settings>
  <x key="setting_one">Hello World From RELEASE!</x>
  <x key="custom_settings">
    <add key="custom_setting_1" value="This is RELEASE custom setting ONE."/>
    <add key="custom_setting_2" value="This is RELEASE custom setting TWO."/>
  </x>
</settings>

MSBuild Task Detail

<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
  <PropertyGroup>
    <ConfigOutDir>$(OutputPath)\cfg</ConfigOutDir>
  </PropertyGroup>
  <Target Name="ProcessConfig">
    <Message Text="==&gt;&gt;&gt;&gt; CurrentEnvironment = $(EnvId)" />
    <!-- BEGIN default (ie common) settings -->
    <XmlQuery XmlFileName="$(MSBuildProjectDirectory)\cfg\defaults.cfg" XPath="//settings/*">
      <Output TaskParameter="Values" ItemName="DefaultSettingValues" />
    </XmlQuery>
    <ItemGroup>
      <Tokens Include="%(DefaultSettingValues.key)">
        <ReplacementValue>%(DefaultSettingValues._innerxml)</ReplacementValue>
      </Tokens>
    </ItemGroup>
    <!-- END defaults.config -->
    <!-- specific environment -->
    <XmlQuery XmlFileName="$(MSBuildProjectDirectory)\cfg\$(EnvId).cfg" XPath="//settings/*">
      <Output TaskParameter="Values" ItemName="SettingValues" />
    </XmlQuery>
    <ItemGroup>
      <Tokens Include="EnvId">
        <ReplacementValue>$(EnvId)</ReplacementValue>
      </Tokens>
    </ItemGroup>
    <!-- Remove duplicates - this is important for "overrides" -->
    <ItemGroup>
      <Tokens Remove="%(SettingValues.key)" />
    </ItemGroup>
    <ItemGroup>
      <Tokens Include="%(SettingValues.key)">
        <ReplacementValue>%(SettingValues._innerxml)</ReplacementValue>
      </Tokens>
    </ItemGroup>
    <Message Text="Parsed Tokens =&gt; %(Tokens.Identity): %(Tokens.ReplacementValue)" />
    <MakeDir Directories="$(ConfigOutDir)" />
    <TemplateFile Template="$(MSBuildProjectDirectory)\app.config" OutputFilename="$(ConfigOutDir)\$(EnvId).app.config" Tokens="@(Tokens)" />
  </Target>
  <Target Name="BeginProcessConfig">
    <ItemGroup>
      <EnvConfigFiles Include="$(MSBuildProjectDirectory)\cfg\*.cfg" Exclude="$(MSBuildProjectDirectory)\cfg\defaults.cfg" />
    </ItemGroup>
    <MSBuild Targets="ProcessConfig" Properties="EnvId=%(EnvConfigFiles.Filename)" Projects="$(MsBuildProjectFile)" />
    <!-- Copy over the correct file based on configuration in visual studio or command line -->
    <PropertyGroup>
      <EnvConfigIn>$(ConfigOutDir)\$(Configuration).app.config</EnvConfigIn>
    </PropertyGroup>
    <Message Text="EnvConfigIn = $(EnvConfigIn)" />
    <Copy SourceFiles="$(EnvConfigIn)" DestinationFiles="$(OutputPath)$(AssemblyName).exe.config" />
  </Target>
  <Target Name="AfterBuild" DependsOnTargets="BeginProcessConfig">
    <Message Text="This is the AfterBuild target: $(MSBuildToolsPath)" />
  </Target>

Demo Project


1. Install MSbuild Common Tasks



http://msbuildtasks.tigris.org/

Note: the easy way to experiment is to just install via .msi installer, alternatively you can download the .zip file with all the binaries, but then you will have to configure the Import of msbuild tasks in the .csproj file - not difficult, and definitely something you want to do for self contain projects (typically you would include the binaries in some libs sub-folder of your project and reference it via relative paths from msbuild).

2. Download Source



View envcfgdemo-zip or if you behind an evil firewall: Download Demo Visual Studio 2008 Project

When opening the project you will be prompted with a dialog telling you that the project file is using some external extensions and whether you want to trust it. This is normal and you have to do it just once.

Try selecting different Configurations in Visual Studio, or run from command line with

msbuild /p:Configuration=CustomEnv

or
msbuild /p:Configuration=Debug

Note: all of the configurations are built and placed into bin\$(Configuration)\cfg folder, you can examine the files and sometimes it's useful for further automatic deployments (ie you can copy over the right config file via a script at the time you deploy vs build time).