Where did the time go?

Windows 8 Workshops at Mall of American

Jeff Klawiter — Fri, 02 Nov 2012 15:35:23 GMT

Starting on November 5th, through November 26th I will be presenting on Windows 8 Development. Each Monday I will cover a topic about different aspects of Windows 8. On November 5th it will be Windows 8 UX Design. The following week Windows 8 Game Development with GameSalad.

To sign up, please go here http://www.msdnevents.com/workshops/

Motion Detection With Silverlight and Native Extensions for Silverlight

Jeff Klawiter — Wed, 27 Jul 2011 19:17:59 GMT

While a little late, here are is the source code and the slide deck for my recent talk at the Twin Cities Silverlight User Group.

SLMotionDetector.zip

This includes a solution with the source code for the common MotionDetection library, a silverlight application with OOB capabilities, a Windows Phone 7 app and a failed WPF attempt.

You will need the Windows Phone 7 Mango Beta 2 SDK to run the WP7 project. This does include compiled versions of the NESL libraries. The source can be found at http://archive.msdn.microsoft.com/nesl/Release/ProjectReleases.aspx?ReleaseId=5519

This code is provided as-is. It comes with no support and complete with "Works on My Machine" credentials. I have removed the certificates used for signing the libraries, you may need to create your own to run the app out of browser. I have not taken the time to clean up the code or remove commented out old code..

TCSLUG – Presentation Materials

Jeff Klawiter — Tue, 20 Jul 2010 23:04:59 GMT

As promised I’m posting the finished projects from my Silverlight Templating presentation at the Twin Cities Silverlight Users Group today.

This includes the slide deck, the Watermark Text Box and Dynamic Themes project.

SilverlightTemplates.zip

New Html Agility Pack Versions and Features

Jeff Klawiter — Sun, 06 Jun 2010 00:17:49 GMT

Recently I have added 4 new projects to SVN for Html Agility Pack.

HAPLight: a Silverlight implementation
HAPCompact: a .NET CF 3.5 version
HAP for .NET 4.0: taking advantage of DynamicObject.
Unit Tests

All of these are works in progress and should be considered in alpha stages thus no binary releases for them yet. To use them you’ll need to download them from SVN. http://htmlagilitypack.codeplex.com/SourceControl/list/changesets

HAPLight

Bringing Html Agility Pack to Silverlight was relatively simple, thanks to Silverlight supporting XPATH and XpathNavigator. There have been two losses so far, HtmlCmdLine and HtmlWeb. HtmlWeb is a big loss and I don't plan on leaving it that way. Silverlight requires all web requests to be Asyncronous which HtmlWeb surely is not. So at some point I will be making a version of HtmlWeb that exposes Asynchronous methods for downloading pages and returning them as HtmlDocuments. For now you can do this yourself without much code using WebClient.DownloadstringAsync()

HAPCompact

Again making a port of Html Agility Pack to .NET CF wasn't too difficult. One of the biggest issues is .NET CF has no XPathNavigator support. There are no good free implementations and I don't expect there ever will be. So HAPCompact will need to rely on using LINQ to Objects. This project needs to be built with Visual Studio 2008. Unfortunately VS2010 did not include any .NET compact framework support. I've been trying to look into a way of taking advantage of VS2010's multi-targeting to add back in compilation support. I have many projects at work that are in .NET CF 2.0 and 3.5.

Html Agility Pack for .NET 4.0

.NET 4.0 shipped with the Dynamic Language Runtime included. C# was updated in turn to include a dynamic typing system. I thought it would be interesting to see if HAP could take advantage of these features to dynamically access HtmlNodes and HtmlAttributes. This project so far is a partial class that makes HtmlNode inherit from DynamicObject. This may change later to have it just implement an interface instead. The advantage of this is you can access first level child nodes and attributes without . Something like documentElement.Html.Body.Div to get the first <div> on the page.

In C# to use these features you need to indicate the object is dynamic. Simply assigning the node to a variable typed as dynamic will suffice. I had hoped to use @ for getting attributes but found that it is completely lost so to access attributes a prefix of _ is needed. Here are some examples taken from the unit tests:

[Test]
public void TestGetAttribute()
{
    var doc = new HtmlDocument();
    doc.LoadHtml("<html><body class=\"asdfasd\"><p>asdf asdf sdf</p></body></html>");
    dynamic docElement = doc.DocumentNode;
    var item = docElement.Html.Body._Class;
    Assert.IsNotNull(item);
    Assert.IsInstanceOf<HtmlAttribute>(item);
}

[Test]
public void TestGetMember()
{
    var doc = new HtmlDocument();
    doc.LoadHtml("<html><body><p>asdf asdf sdf</p></body></html>");
    dynamic docElement = doc.DocumentNode;
    var item = docElement.Html.Body;
    Assert.IsNotNull(item);
    Assert.IsInstanceOf<HtmlNode>(item);
}

Other ideas I’m having with this is to introduce some kind of domain specific language for doing more specific accessing like documentElement.Html.Body.First_Div or documentElement.Html.Body.ById_Header . This will be limited of course due to lack of symbols that could be used.

Unit Tests

I’ve begun adding Unit Tests to Html Agility Pack. This will be a long process to even approach a good code coverage percentage. There is quite a bit of code in the library and some of it could use a good refactoring. So as I’m writing unit tests I may be doing some refactoring as well. Along with this may come some introductions of breaking changes with some of the methods or properties within the API. Thus this next version may be 2.0.

Html Agility Pack 1.4.0 Released

Jeff Klawiter — Sat, 08 May 2010 00:01:55 GMT

Today I finally was able to get some time to get Html Agility Pack 1.4.0 released.

This latest release brings many subtle new features and many bug fixes. While it doesn’t attack some of the major pain points (as when joining the project I was not aware of them) it does bring HAP into the modern age for .NET. Gone are the .NET 1.0 ArrayLists and most of the HashTables. In are Generic lists and functions that return IEnumerable that mimic LINQ to XML. Things like Descendants() and Ancestors(). These new functions serve as an alternative to using XPATH.

Among the new LINQ compatible features 1.4.0 brings with it

Support for Medium Trust environments
Updates to Charset detection and the ability to override it
Many bug fixes
Ability to preserve tags original case when writing out the HtmlDocument
A new sister program HAPExplorer for browsing the HtmlDocument tree and searching said tree
Large cleanups and optimizations of the underlying code. Utilizing Resharper and Code Metrics the underlying code is in better shape than it was before. There is still a long way to go but it is a start.
Added a new Xpath property to HtmlNode that will get the direct path to that particular node. Easy to find via HAPExplorer
MSDN like CHM documentation

Download Html Agility Pack 1.4.0 Now!

I had hoped to release this a long time ago but I got piled up with one rush project after the next. When I wasn’t at work working I was at home working or delving deep into the guts of Silverlight.

HAP’s Past

Html Agility Pack was originally created by Simon Mourrier while he was at Microsoft as a System.Xml look-alike for parsing HTML documents. At that time extremely malformed HTML was the standard across the web. HTML 3.01 still had a good share of web pages out there. While we had tools like Dreamweaver popping up it was still very common to see unclosed <li> and <option> tags. HAP was a godsend. Converting HTML to XML was (and still can be) a pain. As such in those times HAP was set to by default handle the non-standard HTML browsers let slide by in those days.

HAP’s Present

These days the web has come quite a long way with many people pushing for standards and people actually taking them seriously. XHTML came to pass as a default for many HTML editors. Finding sites with horrendous non-conforming HTML is not as likely. Yes there’s still very ugly HTML (loads of tables) but it is still closer to standard than it was then. HAP in recent years has been showing some weaknesses. While I wouldn’t really call them weaknesses others might. It is more in the perception of what people expect HAP to do and what it really does.

HAP’s parsing engine is extremely efficient and can be very flexible when it comes to when tags can be closed. The problem is no one realizes this due to lack of documentation and examples. Also the location of the list of tags and their defaults is not in an easily discoverable position. This has lead to many discussion posts all ending with basically the same outcome, remove this tag or that tag from the list. This list, fyi, is HtmlNode.ElementsFlags. From a parser perspective this list is very handy and efficient, from an end users perspective it can be a bit of a nightmare to work with.

HAP’s Future

I originally joined the project to update it for my own purposes. I hate Xpath (personal reasons) and love LINQ. I had done so much work to update HAP to support LINQ I wanted to share it. I have since abandoned the VS Extension I was working on that I was going to use HAP for. I do not intend to abandon HAP. That being said it might be a bit until a new update is out. I do have a bunch of ideas on how to address the ElementFlags situation. Among these are building in some defaults for different (X)HTML specs, creating a fluent interface for adding them, making them able to be loaded from a config and things like that. Along with that there are many parts of HAP that are now implementing features that have been added to .NET in the last 7 years. The HtmlWeb class does quite a bit that WebClient handles now.

Another space I want HAP to tackle is Silverlight. I know it is being used in the Facebook Silverlight Client (just check it’s end user license). With Silverlight 4 they have added Xpath support. HAP does need to slim down if it were to be ported to SL. 130KB is quite heavy for a Silverlight dll. Plus HAP isn’t compatible with async web calls.

Another thing HAP needs desperately is Unit Testing. Coming up with a testing suite is quite a challenge, there will need to be quite a bit of refactoring to really do it properly.

Another interesting idea I had with HAP is now with .NET 4.0 and the Dynamic keyword and Expando objects we could have quite a bit of fun. Access attributes and child tags as if they were properties on the node? It could be done and would be an interesting endeavor

I welcome anyone that has input on where they want to see HAP go or want to join the project. Right now I don’t have the power to add any new developers but we can make our case to Simon to add new people on.

Silverlight Window Navigation in Safari

Jeff Klawiter — Sat, 24 Apr 2010 21:50:26 GMT

There is a “bug” with Silverlight in Safari when trying to do window navigation. Both HtmlPage.PopupWindow and HtmlPage.Window.Navigate will not work on Safari due to plugin limitations. This makes doing something like using AddThis’ API rather hard to do cross browser. Chris Idzerda over at Vertigo had posted a partial solution a while back but it had the drawback of needing to always have more html along with your silverlight app. I decided to take his solution and make it a bit more automatic.

Chris’ solution involved having certain HTML exist on the page already that Silverlight can call. While this works, I hate having to dictate more and more things that need to be on the page. Below is his solution

public static void  OpenBrowser(string url)
{
  if(HtmlPage.BrowserInformation.UserAgent.Contains("Safari"))
  {
    HtmlElement anchor= HtmlPage.Document.GetElementById("externalAnchor");
    anchor.SetProperty("href", url);
    HtmlElement button= HtmlPage.Document.GetElementById("externalButton");
    button.Invoke("click", null);
  }
 else
    HtmlPage.Window.Navigate(new Uri(url, UriKind.RelativeOrAbsolute), "_blank");
}

<a  id="externalAnchor" style="display:none;"></a>
<input id="externalButton" type="button" onclick="window.open(document.getElementById('externalAnchor').href)" style="display:none;" />

My idea was to take his html and dynamically create it via Silverlight’s Html bridge. (Note this will only work if the xap is on the same domain or “enablehtmlaccess” is set to true for the plugin). To do this is fairly straight forward. Below is my code (with the exception handling simplified)

if (HtmlPage.BrowserInformation.UserAgent.ToLower().Contains("safari"))
{
    try
    {
        if (HtmlPage.IsEnabled)
        {
            var anchor = HtmlPage.Document.CreateElement("a");
            anchor.Id = "externalAnchor";
            anchor.SetStyleAttribute("display", "none");
            HtmlPage.Document.Body.AppendChild(anchor);

            var button = HtmlPage.Document.CreateElement("input");
            button.Id = "externalButton";
            button.SetAttribute("type", "button");
            button.SetAttribute("onclick", "window.open(document.getElementById('externalAnchor').href)");
            button.SetStyleAttribute("display", "none");
            HtmlPage.Document.Body.AppendChild(button);
        }
    }
    catch (Exception e)
    {
        OnError(e.Message);
    }
}

So starting off only if we are in Safari do we even need to run the code. I opted to run this code when my AddThis component is first loaded. If we have access to the HtmlPage we new up anchor and button elements. This generates the exact same HTML that Chris’ did in his example. After this his function from above just works :). It’s a fairly straightforward fix. I hope that soon MS will be able to get SL to support popup/navigation in Safari. I’m sure it has something to do with their plugin api.

VS2010 New Website Template Updates

Jeff Klawiter — Fri, 06 Nov 2009 03:05:51 GMT

I went to create my first new website with Visual Studio 2010 the other day and was quite surprised with what I found.

In all versions prior to 2010 a new website would just get you a Default.aspx and the corresponding language codebehind file. This changes quite a bit with 2010. You actually will now be presented with a full website ready to go. Complete with a Master page, account login and more.

Digging in Further yields some nice little nuggets. First off the Site.Master isn’t just a blank master page with a ContentPlaceHolder. It actually contains a relatively decent CSS layout with LoginView and Menu controls.

Along with the Site.Master the site comes with Default.aspx, About.aspx and an entire directory dedicated to authentication. The Account directory is set up to handle all the most common authentication scenarios: Login, Register and Password Recovery. Furthermore these pages include actual code in the codebehind.

One thing however that it is missing is the database to authenticate with. If you are planning to use your own membership provider or an external database you just need to set it up in the web.config. If you’d like to just use an sqlexpress database you can use the ASP.NET Website Administration Tool to do this for you. This tool was added in .NET 2.0 and the current version doesn’t seem to have changed much. To generate the ASPNETDB.MDF SQLExpress database. Just click on the Tool and World icon in the Solution Explorer.

Another way to access this tool is via the ASP.NET Configuration command under the Website menu.

The best way to have it set up the database is to click on the “Use the security Setup Wizard..” link. This will take you through setting up the database, roles, users and locked down directories.

Beyond the account code the site comes with jQuery 1.3.2, it’s minified version and the Visual Studio Intellisense file. Unfortunately the jQuery files are not referenced by default in the master page or any of the other pages. I would have loved to have either an example usage or at least a reference to the file. To add jQuery it’s just a simple addition to the master file. After that just force VS to update its javascript intellisense using Ctrl+Shift+j

Rarely do I start out a website this way these days, most my sites I start are powered by one CMS or another. Besides those times I’ve been trying to use ASP.NET MVC more these days. I still use the default webforms site when I need to get something up and running quick with some basic data controls. I recently did a website for planning a family reunion. I could have saved myself a half an hour or so getting just the base of the site set up.

C# 4.0 Named Parameter Dangers

Jeff Klawiter — Tue, 27 Oct 2009 16:04:59 GMT

C# 4.0 introduces many new languages features, one of them being named parameters. Instead of passing parameters to a method or constructor in the sequence they are declared you can pass them in as a name/value pair.

//method signature
public SearchData(string searchText, SearchOptions SearchOption, int limit, int skip)

//Calling it the normal way
obj.SearchData("foo",mySearchOptions,10,20);

//Calling it with Named Parameters
obj.SearchData(limit: 10, skip: 20, searchText: "foo", SearchOption: mySearchOptions);

As you can see it is nice that now when passing parameters your code becomes more self documenting. You know what parameter is being assigned to what. Combine this with default/optional parameters it can make your code more clear and concise.

The problem comes when refactoring code. I hope you noticed something wrong about the second parameter in the method signature. While technically sound it fails naming guidelines. Say this was a third party library you were using and they discovered this and fixed it in the next revision. When you update to the new library you will get a compiler error. This is because the named parameters are Case Sensitive. Below is an example of just this

//New method signature
public SearchData(string searchText, SearchOptions searchOption, int limit, int skip)

//This will remain working
obj.SearchData("foo",mySearchOptions,10,20);

//This will now result in a compiler error
obj.SearchData(limit: 10, skip: 20, searchText: "foo", SearchOption: mySearchOptions);

Now I’m not saying you should never use Named Parameters. You should keep in mind if you are using code that is not your own you may be increasing your development debt. Also take into account where the code comes from. If you are doing Office development, I say go for it. Microsoft is very aware of implications of this and will no doubt do their due diligence when updating the Primary Interop Assemblies.

I also am hoping that Visual Studio 2010 will have refactoring support when renaming a parameter like that. I just tried in Beta 2 and did not get any such support. Maybe Resharper will catch it.

VS2010 Extensions Followup

Jeff Klawiter — Mon, 26 Oct 2009 18:15:10 GMT

Thanks to all those who came to my talk. When I first planned to give the talk Beta 1 had been out for a little while and I was working on my own extension. There was news that Beta 2 was no longer accepting bugs (beginning of august) and it was probably due that month. I thought that’d give me a couple months time to give the talk on Beta 2. But as we all know now, it took much longer to get Beta 2 out the door and it landed at the worst time.

When Beta 2 came out, 3 days before my talk, I set up a Windows 7 Bootable VHD up to install it and test my code, just in case I wasn’t able to get anything working and had to use Beta 1 for the talk instead. As I started trying out all the extensions I had I was almost ready to just use Beta 1 since none of them worked. As I got further into trying to use Beta 2 I realized some large fundamentals of all the extensions I was going to show had changed. Classes were removed, new classes introduced, easier ways of declaring your extension. The changes were big enough that I felt I would be doing a disservice to people by showing them the Beta 1 examples. I would rather show 1 working extension in Beta 2 than cause confusion by showing 5 in Beta 1.

Aside from the SDK changes in Beta 2 there were interface, feature and performance changes that I felt important to show. Beta 1 was much slower all around. There were new features in the Extension VSIX Manifest Editor and changes to the VSIX manifest itself. Also there were new templates included.

In the end I’m glad I used Beta 2 instead of Beta 1. While I was missing my examples of adding menu items, commands, doing search and replace in the editor, I was able to show real world working code that anyone going out and downloading Beta 2 after my talk would be able to implement. If I had another week or two between Beta 2 and the talk I would have had all of them.

I’d like to know your thoughts, I’m not above being criticized as long as the dialog is civil. I want to be a better speaker a better person. I know I’m not the most witty or animated, but I love to code and I love to share my knowledge.

Here’s links to some of the resources. The first sample code you can find below, The second two samples shown were from the Codeplex editor examples. I had more hand rolled examples but they didn’t work in Beta 2 but I ran out of time in getting them working.

Where Button Marker
WhereMarker.zip

Editor Examples on Codeplex
http://editorsamples.codeplex.com/

Visual Studio Extensibility on MSDN
http://msdn.microsoft.com/en-us/vsx/default.aspx

Black Editor Colors
JeffKlawiterBlackColoring.zip

PDC ‘09 Predictions

Jeff Klawiter — Fri, 23 Oct 2009 18:01:27 GMT

Disclaimer: These are solely my personal opinion and thoughts of what will be unveiled during PDC this year.

Windows Mobile 7

With the current rumors that WM7 will beta in Nov it’s a no brainer that it will be unveiled at PDC. I’m expecting a major UI update, rewritten graphics system and more integration with .NET CF. The .NET CF team has been rather silent on their blog for a while now and that’s normally a sign of big things to come. I have a inkling that the new UI will be WPF based. Since Silverlight Mobile is WPF, this would give a great chance to make WPF a full UI framework for windows mobile

Silverlight 4

Becoming a staple of PDC and Mix, Silverlight has been on some extremely fast paced development. I’m expecting SL 4 to have webcam and microphone support. Along with more .NET 4.0 features like dynamic types and the DLR.

Silverlight Mobile

It has been talked about more than once, last PDC had quite a large demo for it. I expect it to be announced probably for Windows Mobile 6.5 and higher.

Windows 8

With Windows 7 out of the way, I think we may be teased with some of the ideas they have for Win8. There has been a lot of activity on linkedin related to it. Like 128bit support, which I’m expecting is probably Intel’s Itanium successor and not meant for desktop chips.

Blend 3 SP1

MS has already said that Blend 3 will get .NET 4.0 support in SP1 and it was expected to ship Q3. I think most of this update will only revolve around .NET 4 and nothing in the way of new killer features.

The overloaded .NET developer

Jeff Klawiter — Wed, 21 Oct 2009 22:51:23 GMT

I write this as I’m sitting here working on a .NET CF 2.0 project in Visual Studio 2005 while installing Visual Studio 2010 Beta 2 onto my laptop which is running windows 7 installed on a vhd file booted natively. I look at the list of components being installed with 2010 Beta 1 and I’m just left feeling inadequate. The one that really tipped it was Visual F# 1.0. F# has been out in for a while now and I’ve only seen talks on it. I haven’t written a line of code in it. I can also say the same for Work Flow and many other MS/.NET technologies.

I sat in on an interview today with a rock star programmer. His code sample he sent in to us was the best we’ve ever seen. He’s got strong ASP.NET (and MVC) skill along with HTML/CSS-fu. Then I ask the questions about what else he has done in .NET.. barely any WinForms, no compact framework, little WPF. I do not hold any of this against him, I know full well what it is like to try and keep up with things when your current job doesn’t entail them. Here at Sierra Bravo my job does entail them, sometimes. We do such a large variety of projects I’ve had to learn almost everything under the .NET sun and I still feel like I’m behind.

I’ve spent the last few months trying to write a talk on Visual Studio 2010 Extensions and I sure hope my talk goes well but I’m not optimistic. After getting into doing the extensions, hoping for a nice easy to use SDK as was promised, I found still a complex system of interfaces, attributes and unclear APIs. Mind you, it is still beyond what was available in 2008. I also gave a talk on SQL CLR programming and have been working on learning ASP.NET MVC. Beyond that, reading a WPF certification book and hoping to one day expand my certifications passed the VS 2005 realm. I spend nearly every minute of my spare time reading, coding and learning.

Things I have done

.NET CF 1-3.5 (about 10 applications)
ASP.NET 1.1 – 3.5 (lost count of how many sites)
.NET WinForms 1.1 – 3.5 (about 20 applications)
Windows Services (around 10)
ASMX (lost count)
WCF ( a few)
SQL Server 2000-2008 (full and express)
SQL Reporting Services
SQL CLR
SharePoint integration (2)
Visual Studio Extensions (a few)
Command line Programs (20+)
Library development (HtmlAgilityPack, Sierra Bravo Connector aka PickDB)
TCP Client/Server protocols
Serial Port controlled vending machines
RFID integration
Surface
Silverlight 1.1-3.0
C# and VB.NET
Dabbled in XNA
Dabbled in MVC
Office Integration
LINQ to SQL, LINQ to Entities
T4 programming
Got MCTS WinForms/ASP.NET 2.0 and MCPD ASP.NET 2.0 certified

Things I still need more experience in, have yet to do, work with or even look at

TDD/DI/IoC – I’ve dabbled a bit but feel like I’m really falling behind with these dev patterns
Micro Framework
F#
Work Flow
TFS
Full WPF Application
Full Mono application
M/Oslo
Parallel Extensions
Tons of the new stuff in VS2010/.NET 4.0 (like the asp.net 4 features)
Write a LINQ provider

While the latter list seems small, I’m sure it will grow again. The first list covers 4 1/2 years of work. To me it is mind boggling that I’ve done so much and also a testament to how powerful the .NET framework is. In the PHP world there are a couple great frameworks these days but it has taken many many years to get to that point. Also PHP 6 is turning into PERL 6 with it being on the horizon for many years now. It also has the issue of only being a web language.

I really hope after VS2010 Microsoft slows down for a bit and waits until VS2015 or something. I’m not sure if I can keep up on this pace much longer. Some days I wish for a job where I’m working on a product, or in a slower paced project. Other days I think I’d go crazy if I was working on the same thing all the time. One thing is for certain, I need to start doing things outside of work again.

Uninstalling VS2010 beta 1 – Work Around

Jeff Klawiter — Sun, 27 Sep 2009 00:05:35 GMT

Today I was cleaning off my old laptop for my girlfriend to use. While trying to Uninstall Visual Studio 2010 Beta 1 I ran across an error. It was on the step trying to uninstall the “TFS Object Model”. The error said it was looking for “TFSObjectModel-x86_ENU.exe”, it asks for the install DVD but the exe is not on the disc at all. After some digging I found a way to get the uninstall to work.

The trick is to uninstall the component on it’s own. It’s called “Microsoft Team Foundation Server 2010 Beta 1 Object Model – ENU”

After uninstalling that I was able to run the VS2010 uninstall without a hitch.

HTML Agility Pack - Contributor

Jeff Klawiter — Tue, 15 Sep 2009 16:02:42 GMT

For a few months now I’ve been working on a VS2010 extension I’m calling Funky Search. It’s basic intent is to bring tag based search and replace functionality to Visual Studio. My first order of business when creating this extension was the need for an HTML Parsing Engine. I had used HTML Agility Pack (HAP from now on) in the past. One downside of it is that it uses XPATH for querying the HTML. While in it’s day XPATH was a decent solution for searching XML structures, there are better searching solutions available today namely LINQ.

I set out and updated HAP to have all of it’s Node and Attribute collections to inherit from IList<T> instead of implementing their own Enumerators. I then added many helper methods to mimic LINQ to XML. With this I could now work on creating dynamic LINQ statements to power my extension.

While working on this I got into the community of people using HAP and I came across a larger issue, it had not been updated in years and the creator and other developer on the project had seemed to abandon it. I sent many emails to the creator Simon Mourier (former MS employee, and current CTO of SoftFluent) over the summer with no reply. I finally found his work email and discovered he was on vacation until early September. I was finally able to get in contact with him today and he added me as a developer on the project.

This will mark the first time in about 5 years I’m a developer on an open source project. Before coming to Sierra Bravo I was huge into open source, also at that time MS had no free versions of Visual Studio. I was working as a PHP developer and had contributed to some small projects and even worked on part of the Mozilla project adding in an easier way to code-sign your Mozilla/Firefox extensions.

I’m looking forward to advancing HAP, fixing bugs and making it easier to use. It sits in a unique position as being the only freely available HTML parser that works. While it can be used for dubious purposes as a page scraper it can also be used for good. I’ve used it in the past where we had a client that had their hosting provider go out of business, their site was going to only be up for another day and we had no direct access to their database server. We had FTP access to get the code of the site and access to a readonly front end that displayed the contents of the tables in html with no export functionality. I wrote a scraper with HAP to get those tables and put them into an importable format. With it I was able to download and import their database and save their site.

The Case for Partial Properties

Jeff Klawiter — Mon, 14 Sep 2009 22:18:17 GMT

With the advent of ASP.NET MVC 2.0 and the new templated helpers we’re once again having to write out our entire data objects. This is something that ORM tools like LINQ to SQL and LINQ to Entities were supposed to alleviate. Even better is we must now implement a class in 3 files.

Case in point: http://blog.pagedesigners.co.nz/archive/2009/08/06/asp.net-mvc-2-ndash-buddy-classes-for-your-models.aspx

The idea of “Partial Properties” follows the convention for Partial Classes and not Partial Methods. With Partial classes you can have the same class defined in two files. This is extremely helpful with extending generated code. With Partial Methods, one partial class file defines a method and another partial class file may if it choose implement that method. If the method is never implemented then the compiler completely throws away any code that may have referenced that method.

During a PDC talk Anders said that he couldn’t see a reason why we should implement Partial Properties but it’s becoming clear to me that we are starting to have the need.

Here’s the current situation:

You create your models with LINQ to SQL or LINQ to Entities.
You have no control over the auto-generated code so you implement partial classes for each class made in a separate file
You need to annotate the properties in the generated classes so you then have to create a buddy class
You take all the properties of the generated class and type them in and add attributes to them and thus mostly defeating the purpose of #1

//GeneratedClass.Designer.cs (the generated file)
public partial class GeneratedClass
{
	private string _name;
	public string Name
	{
		get{return _name;}
		set{_name = value;}
	}
}

//GeneratedClass.cs (your file)
[MetadataType(typeof(GeneratedClass_Metadata))]
public partial class GeneratedClass
{
	
}

//GeneratedClass.cs or GeneratedClass.Meta.cs (your other file)
public partial class GeneratedClass_Metadata
{
	[Required(ErrorMessage="Name Required")]
	public string Name{get;set;}
}

In the end you end up with 2-3 files and 3 implementations of your class. You also end up with the compiler generating more classes that their only use are for attributes.

With a “Partial Property” generated code could expose all of the public properties like so

//GeneratedClass.Designer.cs (the generated file)
public partial class GeneratedClass
{
	private string _name;
	public partial string Name
	{
		get{return _name;}
		set{_name = value;}
	}
}
//GeneratedClass.cs (your file)
public partial class GeneratedClass
{
	[Required(ErrorMessage="Name Required")]
	public partial string Name;
}

This may all be moot if we can at least get some tooling support for MetaDataType. We could use a Refactor –> Generate Buddy Class. This could auto-gen the metadata class for you so all you have to do is write the attributes in. Maybe even Class Diagram support?

Recovering Mozilla Weave Passphrase

Jeff Klawiter — Mon, 14 Sep 2009 19:16:03 GMT

I recently got a new laptop and was trying to get Mozilla Weave set up on it and I could not remember my passphrase. So I went to the site to see if I could recover it and could not. They will reset it and delete all of your data when you do. After that you have to resync from one of your working versions. My problem was that my only current working version was my desktop at work which was shut down due to me being on vacation.

Now today I’m back from vacation and I was wondering if there was some way to retrieve my passhprase. After searching my profile folder I realized, maybe they just use the password system built into FireFox. And sure enough there were two entries for weave.

After finding it in your list it just takes hitting “Show Passwords” to retrieve it. If you’re like me and have a Master Password set you will have to enter it again at this point.

Speeding up the ASP Development Webserver in Firefox

Jeff Klawiter — Tue, 21 Jul 2009 19:33:17 GMT

I recently was directed to a nice trick on how to make the Visual Studio Web Development (WebDev.WebServer.exe) server run faster when accessing it in FireFox. I’ve long noticed it seem to lag quite a bit compared to running in IE.

Basically there is something that goes wrong when FireFox tries to connected the server via IPv6. The trick is to just disable it. Unfortunately this disables it for all sites and may cause problems in the future as more places switch over to IPv6. But for now it should be just fine. Changing the setting is quite easy

Type in “about:config” in your address bar (get passed the “are you sure you want to do this” nag screen) and search for ipv6. You should find a settings like below

The setting network.dns.disableIPv6 should be set to True. All it takes is double-clicking on the setting to change it. The effects should be immediate.

Twin Cities User Groups and Events

Jeff Klawiter — Thu, 09 Jul 2009 17:18:02 GMT

I recently sent the list below to a colleague. I figured I should share it further.

This is a list of user groups I attend fairly regularly.

.NET User Group

First Thursday of the month.
General .NET talks covering all areas of .NET
Held at Microsoft offices in Bloomington.
http://www.ilmservice.com/twincitiesnet/Membership.aspx
(the site sucks, just sign up for the newsletter to find out about upcoming events.

Developers Guild

Second Tuesday of the month
In depth talks on .NET, programming patterns, project patterns. Talks have been in depth azure, dependency injection/inversion of control
http://www.twincitiesdevelopersguild.com/
Held at New Horizons Mn in Edina
I may be giving a talk here this fall

Languages User Group

Second Thursday of the month
Held at Magenic in Golden Valley
A new language or part of a language every month. Talks on java, lisp, creating your own language. The next one is going to be great, esoteric languages.. the languages that are made for fun
http://www.twincitieslanguagesusergroup.com/TCLUG/Default.aspx

Silverlight/WPF Users Group

Third Tuesday of the month, held during the day noon to 12
Covers Silverlight mostly, some WPF
Held at Microsoft in Bloomington
(site also sucks) http://www.ilmservice.com/silverlight/Membership.aspx

XNA User Group

Third Thursday of the month
Covers the XNA framework and general game development ideas
Held at Magenic in golden valley
http://www.twincitiesxnausergroup.com/

Twin Cities Code Camp

Twice a year, sometime in spring and fall. Around March/October
Large free event that's all about code and is always full of great talks and cutting edge topics. It takes place on a Saturday and ends with tons of prizes, normally an Xbox 360 or two in the top prizes. (done by random name selection of attendees). I've been to everyone so far and I always come away feeling like it was worth it, something I have not been feeling after going to conferences that I paid for.
http://www.twincitiescodecamp.com/TCCC/Default.aspx
I'm giving a talk this fall on Visual Studio 2010 extensions. I gave a talk in the spring about Visual Studio Tips and Tricks

Twin Cities Web Design:

Meetings don’t seem to be on a schedule,
Covers general web design things like JavaScript frameworks, CSS tools
Meets at Sierra Bravo
http://tcwebdesign.org/

MySQL/Drizzle user group

No set schedule
Covers MySQL topics and what not
Sometimes meets at Sierra Bravo
http://www.meetup.com/mn-mysql/

Sorry for the Silence

Jeff Klawiter — Wed, 08 Jul 2009 19:26:33 GMT

I’ve been so busy working on things at and out of work I’ve fallen behind with updating my blog. I’ve got many things I want to blog about but finding time for putting one together is becoming difficult. Things I’ve been working on lately

Visual Studio Extension: I’m working on a VS 2010 extension that is going to be whole worlds of awesome for ASP.NET developers. Most of the work so far has been in getting the backend pieces in place for doing the actual “work” and figuring out where to get the things I need from Visual Studio.
ASP.NET MVC: I’ve been working on my first ASP.NET MVC site so I can start using it at work. I was reading the ASP.NET MVC book put out by the scotts and others. The first Chapter of the book covers creating a site from start to finish
Preparing for the WPF MCTS exam
Wrote a bunch of SQL CLR code
Testing Blend 3 and Sketchflow
Preparing a talk on VS2010 Extensions for Twin Cities Code Camp 7

Upcoming Blog Articles

VS2010 Extensions Take 2
SQL CLR

Getting Started with Visual Studio 2010 Extensions

Jeff Klawiter — Sat, 23 May 2009 19:43:05 GMT

I searched far and wide and was unable to find a good quick tutorial on getting a Visual Studio 2010 extension up and running. There are a few things wrong in the verbiage used that can be quite confusing.

Obtaining the VS 2010 Beta 1 SDK

The first order of business is to get the Visual Studio 2010 Beta 1 SDK, which will add the new templates for creating an extension.

http://www.microsoft.com/downloads/details.aspx?FamilyID=d197feb6-ced5-40d4-949d-a51f02309ee8&displaylang=en

After downloading it and trying to install it you may run across an uncaught exception (like I did)

The error lies in the Bootstrapper (setup.exe). Since the VsSDK_sfx.exe is a self-extracting zip archive you can use your favorite unzipping utility to get the contents. (or you can take the hard way and get the files in your temp folder after it’s been unzipped). I prefer to use WinRar, it makes it extremely easy

After this there are only 5 files extracting.

We only care about 2 of them. vssdk.cab and vssdk.msi.
Launch the vssdk.msi . This is the main installer for the SDK. It gives little feedback and will auto close when it’s done installing.

Creating Your First VS2010 Extension Project

Launch Visual Studio 2010. In your New Projects dialog under <Your Language>/Extensibility you should now have “VSIX Project”

This project defines the basic extension for visual studio. Out of this you will be able to build your VSIX file for installation into Visual Studio.

After giving your new project a name you are given a barebones extension.

The Visual Studio Extension Manifest

First we’ll start off with an unfamiliar file, the extension.vsixmanifest file. This defines your extension, from title, to license agreement to pictures. While this file is a fairly simple XML file. The VS Team provided a nice interface for editing it.

While most of the form is pretty self explanatory, there are some specific points to make

ID: This is your global ID for your extension. After you first publish your extension, it is probably a good idea to not change this.
Version: this can be viewed as your installer version. It may not necessarily mirror your dll versions.
Supported VS Editions: This is a big one and is also forward thinking. Here you can select from all the different versions of VS2010, including Express, Integrated and Isolated Shell. It can also be expanded later to include the next version of VS.

For more information on the vsixmanifest file, see the documentation http://msdn.microsoft.com/en-us/library/dd393700(VS.100).aspx

The last two pieces are where things get interesting. There hasn’t been much said about how extensible Extensions are in VS2010.

Under References you can create references to other VS2010 extensions that your extension may depend on. Clicking on the Add Reference button gives you this dialog.

You’re given the options to select an extension you already have installed, add an external VSIX package or manually define one and a URL to download it from. The URL part is the real beauty. When installing your Extension it has the ability to get the latest and greatest of an extension. You can of course limit it as well to certain version numbers to avoid breaking changes. This creates a lean, mean on demand Extension.

The Content editor doesn’t seem to be fully baked. Here you can add extra content into your vsix package and have it registered upon install. One example is a registering a Project Template. I’ve borrowed an example from the Card Game Starter Kit .

The Code

You will find one lone code file in your new project. Here is the CS version

using System;
using System.Collections.Generic;
using System.Threading;
using Microsoft.VisualStudio.ExtensibilityHosting;

/// <summary>
/// Empty VSIX Project.
/// </summary>
namespace MyFirstVs2010Extension
{

}

Not much in there. This is where your imagination comes in

Here is where I’ll leave you to fend on your own for now. Navigating the Visual Studio SDK assemblies is another post all in itself. You’ll find all assemblies you need under the Microsoft.VisualStudio namespace in your add references dialog. It is also possible to tie into Team System via the Microsoft.TeamSystem .

For some full source examples of VS 2010 Extensions, check out the Editor Samples on codeplex
http://editorsamples.codeplex.com/

Small Basic Materials

Jeff Klawiter — Fri, 15 May 2009 22:31:39 GMT

I gave a talk on Microsoft Small Basic last night at the Twin Cities Languages User Group. I promised to post my materials and here they are.

smallbasic.zip

The zip contains the powerpoint presentation file, the example programs and the example extension.

For the guy that asked about mp3s. It looks like they will work as long as the computer it’s running on has an mp3 codec installed (which is very unlikely).

Twin Cities Code Camp Presentation

Jeff Klawiter — Tue, 07 Apr 2009 17:12:30 GMT

Just a quick post here for the people that attended my presentation. Thank you for coming to listen to me blabber about snippets and little tips and tricks. I hope you found it of some use.

Visual Studio Productivity.pptx

Silverlight 3 Shader Effects

Jeff Klawiter — Wed, 18 Mar 2009 23:41:54 GMT

With the release of Silverlight 3 Beta we can now run PixelShader effects on any control. This can produce some powerful stuff as shown during the Mix 09 Keynote this morning. I delved in as fast as I could to find out how to make your own and work with them in VS.

I downloaded the WPF shader library from http://wpffx.codeplex.com/ . I only imported the ShaderEffectLibrary so far, I have not tried the transition one.

I simply copied the EffectFiles, ShaderSource and EffectLibrary.cs files into my project and tried to compile. Issues I ran into

UIPropertyMetadata needs to be replaced with PropertyMetadata
Color.FromScRgb needs to be replaced with Color.FromArgb and it's inputs run through Convert.ToByte
MagnifyEffect.cs and SwirlEffect.cs reference classes that do not exist in Silverlight like the Vector class. I could have got them from reflector but in sake of time I didn't (that and being able to release this code)
Getting the ps resources required a minor change to the Global static class. Basically changing the path to do a Relative URI lookup.

The basics here for doing your own shader effects is included save 1 thing. You need the DirectX SDK to compile your HLSL code. You can add a new build event or do it from the commandline. This is a good tutorial on how to do it http://windowsclient.net/wpf/wpf35/wpf-35sp1-hlsl-primer.aspx.

So the basics of doing your own Pixel shader in Silverlight is damn near identical as WPF. You inherit from the ShaderEffect class, load your compiled HLSL bytecode resource and expose any properties you need to modify as dependency properties.

Attached is my ugly but working example. I couldn't find anyway to bind to properties on an effect so I had to just do it by event.

SlShaderEffectDemo.zip

VS2010 CTP Expiring Jan 1st

Jeff Klawiter — Wed, 17 Dec 2008 17:05:58 GMT

For those of you still playing with the VS2010 and .NET 4.0 CTP bits, be aware that it will stop working on Jan 1st. MS has not released any word if we will be getting an updated CTP or a beta anytime soon. This is a bit unlike them where they had a bit of overlap between a VS CTP and the beta.

There are some ways around this. I found a nice post that covers some of the options and links to the information
http://blogs.msdn.com/granth/archive/2008/12/10/visual-studio-2010-ctp-vpc-will-expire-jan-1-2009.aspx

Basically you need to disable time syncronization for the VPC image.

I was worried about this since the get go. I wanted to do some talks coming up on .NET 4.0 and the new parallelism features but it's very hard to do that when you're not sure if you'll even have a working copy to demonstrate on.

MCPD: Web

Jeff Klawiter — Tue, 09 Dec 2008 15:25:58 GMT

Well I took my MCPD: ASP.NET 2.0 test yesterday and passed. It has been just over a year since I took my .NET 2.0 Foundation test. In that time I've taken MCTS for windows and web and now MCPD. There are many more certs to go. Next stop I think are the new .NET 3.5 TS certs.

PDC Showoff: DevXpress VS2008 Clipboard History

Jeff Klawiter — Tue, 28 Oct 2008 17:29:58 GMT

I'm definitely installing this one when I get a chance. http://channel9.msdn.com/posts/briankel/PDC2008-ShowOff-Entry-Clipboard-History-for-Visual-Studio/

Basically it keeps a history of your clipboard items like Office but with one killer twist. "Paste as {Language}". It will take the C# you copied and Paste it as VB. I'm sure it's not 100% but still helps quite a bit.

CLR 4.0 to include the DLR - With Limitations

Jeff Klawiter — Tue, 28 Oct 2008 16:23:43 GMT

All of these new C# 4.0 dynamic features require parts of the DLR. Thus it looks like MS is taking the DLR and making it a first class citizen in the CLR. This also I'm guessing will make IronPython and IronRuby first class citizens as well. A huge win for the dynamic languages community. For C# 4.0 it is bittersweet. It means better interoperability when calling things created in IronRuby or IronPython but there are limitations. Below is an excerpt from the C# 4.0 WhitePaper (available here http://code.msdn.microsoft.com/csharpfuture)

Open issues

There are a few limitations and things that might work differently than you would expect.

· The DLR allows objects to be created from objects that represent classes. However, the current implementation of C# doesn’t have syntax to support this.

· Dynamic lookup will not be able to find extension methods. Whether extension methods apply or not depends on the static context of the call (i.e. which using clauses occur), and this context information is not currently kept as part of the payload.

· Anonymous functions (i.e. lambda expressions) cannot appear as arguments to a dynamic method call. The compiler cannot bind (i.e. “understand”) an anonymous function without knowing what type it is converted to.

One consequence of these limitations is that you cannot easily use LINQ queries over dynamic objects:

dynamic collection = …;

var result = collection.Select(e => e + 5);

If the Select method is an extension method, dynamic lookup will not find it. Even if it is an instance method, the above does not compile, because a lambda expression cannot be passed as an argument to a dynamic operation.

There are no plans to address these limitations in C# 4.0.

To me this is a very huge limitation. I can already see that most of my interop with dynamic languages will probably involve collections of some sort. Also this would come into play with collections from Dynamic COM objects. LINQ is so powerful and easy to use, it may end up being a major annoyance to have to move away from it for dynamic typing. I hope they work on this for C# 4.5

Update on new C# 4 features

Jeff Klawiter — Tue, 28 Oct 2008 14:13:13 GMT

While browsing through MSDN blogs I came across this nice little post. http://blogs.msdn.com/dparys/archive/2008/10/28/neue-m-glichkeiten-in-c-4-0.aspx . After translating the page I found that he linked to the new C# 40 page http://code.msdn.microsoft.com/csharpfuture

I played around with VS 2010 last night. I was able to test the dynamic keyword. It works as advertised but the biggest thing one has to realize is using it removes intellisense for that variable. Compiling type safety as well. I hope they'll be able to add some sort of limited intellisense by looking at the last assigned type.

Also on the Dynamic front is DynamicObject. A new base object type that allows for on the fly Property declaration. The DynamicObject uses a PropertyBag (looks like a Dictionary<string,object>). You can declare properties on the fly. Like

public class MyBag : DynamicObject
{
// überschreibt Getter / Setter
} 
dynamic b = new MyBag();
b.Id = 124;
b.Name = "Windows 7"
b.Price = 499.99m;
b.IsAvailable = false;

One thing I was unable to figure out was the optional, default and named parameters. Again the blog provided some answers.

public void InsertCustomer( int customerId,
                          string companyName = "Neue Firma",
                          decimal creditLimit = 2000m )
{
}

InsertCustomer( 1, creditLimit: 2000m );  

InsertCustomer( creditLimit: 2000m, customerId: 1 );

PDC Announcements Day 1

Jeff Klawiter — Tue, 28 Oct 2008 01:03:34 GMT

Well I spent most of the day working on some peculiar problems with PICK. Then left to take my girlfriend out for her birthday. Just now catching up on whats gone on today at PDC. Man it seems like it was a huge day.

First off Windows Azure and the new .NET Services. From what I've read so far I can see some compelling uses for the .NET services and sharing content, id's and roles around the web.

The big one that I haven't seen much press on yet is C# 4.0. Looks like we are getting dynamic binding in the language. While this loses compile type safety it gives C# good ground against things like Ruby and PHP.

Named, Optional and Default parameters. Oh how I've been waiting for these since switching to .NET from PHP. I used to take advantage of these features all the time. It annoys me when I can use them in Attributes in C# but not on actual methods. I'm going to love this.

VS2010 and C# will CTP are available in a virtual PC here

Here's a better overview stolen from http://blogs.msdn.com/samng/archive/2008/10/28/microsoft-visual-studio-2010.aspx

Dynamic binding. We've introduced a new type, dynamic, which behaves much like object, but allows the operations performed on your object to be bound at runtime instead of compile time.

Named and Optional parameters. You can now specify default values for your parameters, allowing them to be optionally specified at the call site. We've also added the ability for your arguments to be passed by name, so that you can specify exactly which arguments you want to give, and refrain from specifying the rest (assuming they're optional).

Com interop features. We've done quite a bit of work to improve COM interop. These include:

No ref for COM calls. For all COM calls that take ref arguments, you can specify an argument without a ref, and the compiler will generate a local for you and generate a ref to that local as the argument.

No PIA. We have introduced the ability to deploy your applications which use Primary Interop Assemblies (PIAs) without referencing the actual PIA at runtime. This allows compiling against them, but not needing to ship them with your application.

Implicit dynamic for COM types. We now give you the option of turning all objects returned from COM into dynamics so that you can perform late bound calls off of them instead of having to cast the result in order to make it useful.

>

Windows Troubleshooting Tip of the Day: Failed ActiveX object creation

Jeff Klawiter — Fri, 24 Oct 2008 15:25:44 GMT

So today I was tasked with helping a client move an old custom VB6 app to a new Windows 2003 server. The company that created it was asking for an arm and a leg for a simple transfer. After we moved the entire program directory over everything seemed to be fine until we encountered an "ActiveX component can't create object" error. I was perplexed on how to track down the issue. There were not accompanying DLLs or OCX files. I tried Process Explorer to see if I could find any failed openings and the strings contained in the exe.

After some searching on the net I found this lovely page http://www.cryer.co.uk/brian/windows/trbl_nt_axccco.htm . It details on how to use Sysinternals Regmon to track down issues with failed ActiveX creation. Basicall you watch your program for registery key openings and look for any failed opens in the HKCR/Classes path. After running the program and encountering the error I found it.
HCKR\Classes\cdonts

When I saw that come up I immediately knew what to do. cdonts is no longer included with windows server. It is an emailing library using cdo. I've had to install it on new Windows 2003 servers more than once so old ASP classic sites could run. All I needed to do after figuring it out was copy the cdonts.dll over to the C:\windows\system32\ directory and run regsvr32 on it. Everything was golden.

I've been using Sysinternals for years. I'm always amazed at what new ways I can use their utilities to track down issues. Process Explorer has been a godsend for me over the years. After discovering it, removing malware from computers took me less than half the time it used to. It gives me so much information I'm not sure how I got by without it.

The Sysinternals Suite is a must have for any serious windows programmer or administrator. It ranges from programs to monitor network connections to being able to see and suspend individual threads in a program.

Deprecation of API's and Native Calls

Jeff Klawiter — Fri, 24 Oct 2008 02:15:23 GMT

The other day I was posed with a problem to implement a "Send as Attachment" feature to an existing application. I thought, well this should be easy I've seen many other do it before. I quoted the work at a few hours, figuring some quick searching would give my my answers. I was wrong.

Sending an email using the default email client is very easy with the mailto: protocol. The problem lies in adding an attachment. Common mail programs like Outlook and Thunderbird do not support the attachment option. So, I started looking at MAPI the native API for doing this. I could not find any .NET wrappers. I did find that there were controls created by MS for VB6. Most of the .NET examples I found used COM interop with the ActiveX controls. But after trying these controls (which look like they were last updated in 98) I found them to not work in XP SP3. I'd get random exceptions and was never able to actually get an email dialog to pop up.

After failing with the ActiveX controls I was out of time. The MAPI native commands may have worked but it would take too long to set up the native calls and test it. After discussing with the client we ended up coming up with a bit hackish but workable solution. I would use the mailto: protocol and then launch explorer with the attachment selected. To them this is a minor annoyance that is acceptable. To me this an unfortunate sign of the times.

Things that were once "easy" for native applications become complicated and buggy for managed. It seems too often that I must make use native calls or old unsuported COM objects to get the job done. While MS has done a great job building up the .NET Framework. It seems that many native API's still remain unwrapped. Furthermore some places I read that the MAPI api is unstable on Vista. It seems that some API's are slipping through the cracks.

SilverLight 2 Released

Jeff Klawiter — Mon, 13 Oct 2008 18:22:40 GMT

There are blogs on MSDN popping up that SilverLight 2 is indeed gold and will be public tomorrow Oct 14th. They also announced Silverlight Tools for Eclipse with the promise of cross platform development for Silverlight. I think this is a huge step in the right direction for MS. By giving developers on OS's other than windows to do Silverlight Development in a fairly competent IDE it will give them a better position against Flex.

I have done some playing around in Silverlight 2 and really loved it. I'm just hoping for a Silverlight 2 project to come our way at Sierra Bravo.

I look forward to downloading the public bits and seeing what I can do with the full version.

Twin Cities Code Camp 5: Over and Out

Jeff Klawiter — Sun, 12 Oct 2008 17:59:53 GMT

Well Code Camp 5 is in the bag. It was a large turn out and many great talks. My talk however drew no interest. Oh well, I tried and will try again. I'm thinking of making my post about LINQ and Refactoring into a talk.

The talks I went to:
The Intersection of F# and LINQ
This was a great talk about how F# takes LINQ and adds much more value to it with it's functional nature. The demonstration was a ray tracer that used LINQ to build up all of the reflected pixels. It was quite impressive. Combine it with the Arc talk and I am now very interested in Functional programming.

The Arc Programming Language
I had seen this talk a few days earlier. The speaker was hilarious and did a great job of bringing excitement back to lisp. Arc is a new dialect of Lisp that aims to cut down on parenthesis and add shorthand for common operations. He added objects to Arc with six lines of code, quite impressive.

BOO! A Wrist-Friendly Language for the CLI
BOO! is a another language along the lines of Lisp where it can redefine parts of itself through macros. The difference is it is statically typed like C#. It does offer quite a bit of reduction in code written. The presenter Justin Chase did a good job showing many aspects of the language. I also won a shirt for figuring out that the mystery function was calculating a fibonocci sequence. The code for it was quite cool, a,b = b + a or something like that.

Building MyTube with Microsoft Silverlight 2
Jeff Brand from Microsoft was giving this talk. It was a basic overview on SilverLight, the purpose for it and how to implement a simple youtube interface. I didn't stay for the entire thing, I had to go get ready for my talk.

My talk "Pick/Multivalue 101"
No one that was interested showed. Not that I was surprised but still a bit disappointed. While giving the talk at my work we had over 15 people show. One of my co-workers showed and the other guy that showed was just looking for a place to sit. So instead we talked about LINQ, customized programming in Linux and a bit about Multivalue

I went to the speakers after party, got to gossip with some people from Magenic. Learned that Justin Chase left Magenic a few weeks ago to work on the Blend team at MS. He's a lucky guy and seemed rather excited for the features they are working on that he can't talk about. I'm sure some of them may come up soon at PDC. I did hear some gossip earlier in the day that SilverLight 2 RTM is going to be released as early as tomorrow (Oct 13th).

Advice: How to be a successful developer

Jeff Klawiter — Mon, 29 Sep 2008 21:28:37 GMT

Being a developer is one of the luckiest professions in the world. Just take a minute and think about this: How many professions out there are you paid to learn? Seriously, if we wish to succeed we must learn. Many times we need to learn during the project. We are given the opportunity and often encouraged to learn new ways of doing something, come up with innovative ideas, implement new technologies. Many companies offer compensation for developers to get certified, keep up, to learn. I've also found that many of the people I know that went to college for CS came out ill prepared for the real development world.

I was lucky in some ways. I got a job running websites during high school, I was paid to learn and mature as a programming. I wasn't lucky that pay sucked and I had to get food from charity at times. But there was always being able to learn on the job and the outlook that real developers jobs paid very well.

Over the years as I've matured as a developer I've learned quite a lot. I strive to keep on learning, branching out into new areas. I've found that my inner urge to learn is also what has allowed me to succeed. Below are some of my tips to being a successful programmer

Be passionate

The one thing I've seen in every great developer I've ever met is passion. Passion for the job, passion for learning, passion for being challenged. There is no way to improve without having a need for it. If being a developer is just a job, just a way to make money, I'm sorry but you should find something else to do. I've seen many developers that have that attitude and they just don't make it. They make mistakes and never learn from them, they have difficulties completing anything on time, and wind up getting fired quite a bit.

Never be afraid to try something new

While sometimes it's hard to break out of things you know that work, learning that things can/cannot be done another way expands your experience and keeps your mind trying to work outside the box.

Keep up with the Joneses

This one can be very hard if you have a full schedule. But trying to keep up with the new technologies will expose you to new ideas, new ways of tackling problems. One cannot grow if one does not see another way.

Pursue certifications

Some people see certifications as a croc. They see that lots of tech schools teach for the test and thats it. I can understand that. I have persued certifications as a professional goal. Going through the MCTS certifications for ASP.NET and Winforms has exposed me to many things I probably wouldn't have known about in .NET. I try to read through the entire training kit books MS puts out for the certifications.

Gaining the certifications are also good for ones ego. Having the goal to set out and earn the certification can keep you discleplined and give you a sense of accomplishment. It's something you can show your bosses and peers that you do indeed know what you're talking about. It gives you more bargaining power when negotiating a raise or looking for a new job. While some may not help you as much as you'd hope, having both real world experience and certifications can be a 1-2 combo.

Microsoft's certification track is the best I've seen. You can start out by getting Technological specific certifications with the MCTS series. You can get just certified in ASP.NET, Windows Applications, Windows Mobile.. and on. Then you can expand on those with Professional certifications with the MCPD's. These ones include application lifecycle components. Professional ways of breaking down an application into logical, component and class diagrams to implementation of architectures and finishing with deployment and maintenance.

Learn new languages

This one can be hard for people that only use one language at work. Learning a new language again helps you get exposed to new ideas and ways of tackling problems.

The hardest thing is actually finding something to do in that language to learn it. I've found that the best way is to have something you implement in every language. Whether it's some tool you've developed or a little game. I had the Al Bhed translator. I originally developed it in PHP to help me out with the game Final Fantasy 10. I then tried doing it in JavaScript, then Java, then C++, XUL (firefox extension) and then in C#.NET Winform. It gave me a common goal in each of the languages and forced me to find out how to do it in each one. They all had their own strengths and weaknesses. PHP's Array functions made some of it very easy to implement, where javascript was a bit clunky. C# wasn't as elegant as the PHP but it was much faster in execution.

Try to find languages that have a different goal than just another application. Languages like F# for example. While it's possible to write every day applications with it, it's real power lies in analyzing and transforming data. Learning it opens up a world of mathmatics in new ways.

Learn new platforms

Simply this means do not just be a web developer or an application developer. Continuously expanding the platforms you know can increase your understanding of how other people develop. I started out as a web programmer and over the years I've written programs for the commandline, windows services, web services, mobile phones, automated scripts, desktop applications, rich internet apps (flash, silverlight), dynamic web sites and more. I'm currently picking up XNA and trying to learn what programming a game is like. I'm learning a newfound respect for game programmers.

Seek out challenges

If your current job is not challenging you find a way to be challenged. Sometimes even playing around things outside of work is enough. Go to your boss and see if theres ways you can increase your workload, or get into more complex projects. Express an interest in new things. If you are stuck at a job where that is impossible, I recommend it may be time to look elsewhere. Get some certifications and new skills under your belt and start looking. There are some great tech jobs out there, good places to find a place to be challenged are in start ups or smaller custom software companies. Places like where I work at, Sierra Bravo. When I joined I had 6 years experience but still would consider myself a novice programmer. Within the next year I worked on more than 20 projects, on many platforms and felt like I had finally become a professional developer.

Now being on the verge of 4 years later I feel like I'm an accomplished developer, I'm able to start incorporating more advanced concepts in to my projects. I can see a much larger picture. As the years have gone on the scope of the picture I can see has expanded. I once could only see the single web page I was working on. Then I was able to start understanding the system of web pages I would design. I then started graduating onto being able to write a complete web site. That is when I decided that desktop applications were next. The scope expanded to understanding state, threads, and desktop experience. My first applications were cluttered, did not have much for layers. I started to learn more about the concept of breaking out applications into libraries and layers. These days I've graduated to an architect that can take the project proposal. Break it down to sections, logical components and build the applications architecture. Define the database, the data layers, the data objects and the front end. I can incorporate n-tier designs, abstract out business logic to a web service. Write API's that work for websites as well as windows mobile.

Really it all boils down to is: Have passion for your job, passion to learn new things and keep yourself challenged. Never be afraid to learn new skills. In this day an age a developer needs to know more than just one platform. They need to understand a larger scope to make it.

Rant: Some VB.NET developers...

Jeff Klawiter — Mon, 29 Sep 2008 19:14:14 GMT

Ok in the last few years I've had to take over a number of websites written in VB.NET, or I should say written in VB but somehow using .NET. My issue really isn't with VB.NET itself. It's obvious the developers that did these sites (none of them affiliated in any way) came from doing classic ASP and moved onto ASP.NET. The problem is they continued with their classic ASP practices and did not seem to come close to grasping what ASP.NET really was.

Issues I've had:

None of the sites will compile in Visual Studio
All of the sites were running in ASP.NET 2.0 but none used master pages. They all used Server-Side Includes (which should be banned) to include common blocks.
No use of controls or classes.
No use of code-behind
All SQL is dynamic and full of SQL Injection vulnerabilities
All used include files which set up global variables and functions to be used in the page lifecycle (one of the main reasons the sites wont compile)
One of the sites had every folder set up as it's own application in IIS. Their hosting provider has no interface for even seeing this, the developer used to call in to have this set up. More on this site later (refered to as BS1)
Most of the sites only make use of the most basic controls
One site (BS1) tried to use more advanced controls like Formview but failed miserably. In this instance, used all over the site, the developer made a formview, used an EditTemplate and bound a dataset to the formview. Sounds decent so far right? The issue came in after I made some changes that my data was not being updated in the dataset. Tracking down the issue was very hard to do when you have no "Go to Definition" and there are tons of "Include Files" that are copied and pasted throughout the site. I found that there was a "Magic" function that did what databinding is supposed to do. It looked for any controls in the formview that matched the exact name of a column and bound the data.
BS1 includes so many functions that recreate .NET functionality that his developer seemed to give up on getting to work within an hour or trying. I want to go home sick everytime a change comes up on this site.

Basically on every one of these sites we have pitched rewrites. Most of them on grounds of security, which is completely true. One of the sites we've had to take down due to rampant SQL Injections. It wont be up until the client decides on which overhaul to rewrite proposal they want to to sign off on.

If the BS1 rewrite ever happens I hope I am not the one to do it. While the site needs it, the business rules and database behind it are majorly complex and in some points completely bonkers. Redesigning the database will be completely out of the question due to the amount of time to migrate that and other business apps the client uses to work with it.

After all of the things I've seen I'm scared of how many classic ASP developers have moved to ASP.NET without taking the time to understand what ASP.NET (or even .NET) is all about. The thing that scares me the most is that it has been 6 years since .NET was released. Some of the sites were even written starting with ASP.NET 2.0. Yet the developers probably used Visual Studio as nothing more than a bloated notepad.

I came from doing site development in PHP 3/4. Classic ASP and PHP 3/4 development were alike in many ways. One would use server side includes to include headers/footers for the page. There would more than likely be a main include script that set up all the functions/classes and global variables to be used on the page. All the HTML was generated with inline code. I can understand moving from an environtment like that to ASP.NET you may start out recreating what you know.

What I don't understand why these developers never felt a need to move beyond that when there is overwhelming evidence that ASP.NET can do so much more. The old way is horrible to maintain, can take a long time to develop and there are much better practices out there. While I was a PHP developer I started using Smarty templates. I loved having templates that abstracted the presentation from the real code. Smarty has become one of the defacto standards in PHP development over the years. It gave me the introduction to using "controls" and "master pages". It made reuse easier and presentation cleaner. To go so long continuing the same bad practices is unfathomable to me.

Another thing that amazes me is ASP.NET actually works with server side includes. The dynamic compiler for the site will included the code and build a DLL for the page. I'm a bit pissy at MS for even allowing this. The included code is compiled into every DLL and whenever a change is made every one of those pages must be recompiled.

Join the Nerdery - Get $50 for just interviewing

Jeff Klawiter — Wed, 24 Sep 2008 14:49:02 GMT

My company just launched a new Recruitment site http://www.nerdery.com.

We are willing to pay you $50 for sending in a decent code sample and making it to the interview. I personally love working here, I get paid to learn. As I've said before I felt like I learned more in the first year working here than in the 5 years I did prior. The availability of so many cutting edge projects keeps us always on our toes. When I joined we had 11 employees, now we are up to 75 and are still going strong. The demand for us keeps going up and we are adding new partners in other parts of the US to keep it that way.

The site has some testimonials and videos . Alas I was on vacation when they did the videos so I didn't get my butt on there.

If you're looking for a fast paced job in the programming industry and live in the midwest, come check us out.

DeflateStream and zlib

Jeff Klawiter — Fri, 19 Sep 2008 14:42:40 GMT

The company I work for, Sierra Bravo Corp, got its start connecting legacy PICK systems to the modern world. They accomplished this via a proprietary client-server protocol we internally call db_server (Official name is SierraDBC or BravoConnecter). Sure there are other PICK connection technologies out there none of them support multiple PICK systems and OS Platforms. Currently we support : D3, Universe, UniData, JBase, MvBase and others I don't remember right now.

I maintain the .NET client library. It was originally a port of the Java library which was a port of the COM library. It's come quite a long way since then. Recently we've found the need for compression to be added for some of our client. One in particular has many employees out in the field connecting via AirCards where bandwidth availability can be a problem. On top of that the application needs to retrieve quite a bit of real time data from the home office. Retrieving 2mb of client history is not unheard of.

After the server developer added compression to the result stream I started working on the client end. I figured it should be a snap. If we have compression turned on, just pass in a DeflateStream (fed by the NetworkStream of the TcpClient connection) to the StreamReader we already use to retrieve the results.

StreamReader ResultReader;
if (IsConnectionTypeSet(ConnectionOptionTypes.EnableCompression))
{
    DeflateStream dfs = new DeflateStream(ns,CompressionMode.Decompress);
    ResultReader = new StreamReader(dfs, Encoding.ASCII, false);
}
else
{
    ResultReader = new StreamReader(ns, Encoding.ASCII, false);
}

One would think that's all that it would take, right?

I was sadly mistaken. I would receive an exception (System.IO.InvalidDataException:"Block length does not match with its complement.") every time I would try to read from the stream.

The issue lies in the use of zlib for the compressed stream. zlib and DEFLATE use the same algorithm for compression. The difference is zlib sends two bytes of header data. So all the answers I found were to pop off the first two bytes of the stream.

StreamReader ResultReader;
if (IsConnectionTypeSet(ConnectionOptionTypes.EnableCompression))
{
    DeflateStream dfs = new DeflateStream(ns,CompressionMode.Decompress);
    ResultReader = new StreamReader(dfs, Encoding.ASCII, false);
    ns.ReadByte();
    ns.ReadByte();
}
else
{
    ResultReader = new StreamReader(ns, Encoding.ASCII, false);
}

This worked just fine but I was annoyed about how ugly it looked. I needed to move that ugliness out of there. So I wrote a ZlibStream class to do this for me. I made it for only Decompressing streams since I didn't have the budget to go and actually implement the zlib headers.

[ZlibStream.cs]

    /// <summary>
    /// Supports decompressing a DeflateStream created by the zlib library
    /// </summary>
    class ZlibStream : DeflateStream
    {
        #region Fields

        private bool HasRead = false;

        #endregion

        #region Constructors
        /// <summary>
        /// Initiates ZlibStream in Decompress mode
        /// </summary>
        /// <param name="stream">One of the System.IO.Compression.CompressionMode values that indicates the action to take</param>
        public ZlibStream(Stream stream)
            : base(stream, CompressionMode.Decompress)
        {

        }
        /// <summary>
        /// Initiates ZlibStream in Decompress mode
        /// </summary>
        /// <param name="stream">One of the System.IO.Compression.CompressionMode values that indicates the action to take</param>
        /// <param name="leaveOpen">true to leave the stream open; otherwise, false.</param>
        public ZlibStream(Stream stream, bool leaveOpen)
            : base(stream, CompressionMode.Decompress, leaveOpen)
        {

        }
        #endregion

        #region Public Methods

        public override int Read(byte[] array, int offset, int count)
        {
            if (HasRead == false)
            {
                this.BaseStream.ReadByte();
                this.BaseStream.ReadByte();
                this.HasRead = true;
            }
            return base.Read(array, offset, count);
        }

        #endregion

    }

As you can see it is very simple. Since all the StreamReader does is call the Read method I just needed to remove those bytes during the first pass. This class is extremely simple and limited in its functionality. Doing Asynchronous reads with BeginRead will not work. I checked via reflector and it does not use the DeflateStream.Read method. It handles the BaseStream's Read methods on its own.

The one thing I may need to expand is doing the popping of the two bytes. Currently it's not checking to see if the stream has any bytes to read. So far in testing this hasn't been a problem. I'm going to try and see if I can create a situation where the bytes are not available initially. I have a suspicion that the ReadByte() waits for a Byte to be available and errors out on the Timeout value

So my result is as elegant as can be

StreamReader ResultReader;
if (IsConnectionTypeSet(ConnectionOptionTypes.EnableCompression))
{
    ZlibStream dfs = new ZlibStream(ns);
    ResultReader = new StreamReader(dfs, Encoding.ASCII, false);
}
else
{
    ResultReader = new StreamReader(ns, Encoding.ASCII, false);
}

LINQ: Refactoring Inline Instantiation

Jeff Klawiter — Sat, 23 Aug 2008 20:39:02 GMT

Over the summer I was able to run my first large project in .NET 3.5. I had a chance to put to use all the new features and learned quite a bit on the way. I've blogged a bit about this project before. The project contained 2 data backends a local SQL database and a 3rd-party ASMX service. I took the approach of having a business object library that contain only class definitions that I had full control over. I split out the backends to their own libraries with a main Datalayer library that handled the communication with the two underneath it.

Initially I started coding the two bottom layers to do the object instantiation inline in the LINQ queries. As the layers grew I began to refactor much of the instantiation to methods that mapped the layer objects to the business objects.

Converting an inline expression like this

var result = from id in dc.OrderDetails
             where id.OrderID == OrderId
             select new DataObjects.OrderItem()
             {
                 PartID = id.PartID,
                 Price = id.Price,
                 Quantity = id.Quantity
             };

To this

var result = from id in dc.OrderDetails
             where id.OrderID == OrderId
             select MapOrderDetailToOrderItem(id);

Using the LINQ to SQL Classes was a perfect fit. They are by far much easier to use as an ORM than SQL Datasets and DataAdapters are. When retrieving lists such as items on an order the relationship propertis on the SQL objects made it extremely easy to have clean data access.

After completing the project I started thinking about what the performance impact of the refactoring had on the data access. So I decided to run some tests. Initially I thought that the inline instantiation would probably be faster since it was constructing the object in the expression instead of getting the SQL objects and then passing it to a function.

I wrote a quick program to do some performance testing on both implementations. I set up examples using a common real world call. Retrieving an order from a database with the line items on the order Below you can see the inline call and the refactored call.

[Tests.cs]

    static class Tests
    {
        public static void RunInlineInitialization()
        {
            using (SqlOrdersDataContext dc = new SqlOrdersDataContext())
            {
                var result = from o in dc.OrderHeaders
                             select new DataObjects.Order()
                             {
                                 CustomerID = o.CustomerID,
                                 OrderDate = o.OrderDate,
                                 OrderID = o.OrderID,
                                 OrderTotal = o.Total,
                                 ShippingAddress1 = o.ShippingAddress1,
                                 ShippingAddress2 = o.ShippingAddress2,
                                 ShippingCity = o.ShippingCity,
                                 ShippingDate = o.ShippingDate,
                                 ShippingMethod = o.ShippingMethod,
                                 ShippingState = o.ShippingState,
                                 ShippingTotal = o.ShippingTotal,
                                 ShippingZip = o.ShippingZip,
                                 SubTotal = o.SubTotal,
                                 TaxTotal = o.TaxTotal,
                                 TrackingNumber = o.TrackingNumber,
                                 Details = o.OrderDetails.Select(od => new DataObjects.OrderItem()
                                 {
                                     PartID = od.PartID,
                                     Price = od.Price,
                                     Quantity = od.Quantity
                                 }).ToList()
                             };
                DataObjects.Order order = result.FirstOrDefault();
            }
        }
        public static void RunRefactoredInitialization()
        {
            using (SqlOrdersDataContext dc = new SqlOrdersDataContext())
            {
                var result = from o in dc.OrderHeaders
                             select MapOrderHeaderToDataObjectOrder(o);
                DataObjects.Order order = result.FirstOrDefault();
            }
        }

        private static LinqTest.DataObjects.Order MapOrderHeaderToDataObjectOrder(OrderHeader o)
        {
            return new DataObjects.Order()
            {
                CustomerID = o.CustomerID,
                OrderDate = o.OrderDate,
                OrderID = o.OrderID,
                OrderTotal = o.Total,
                ShippingAddress1 = o.ShippingAddress1,
                ShippingAddress2 = o.ShippingAddress2,
                ShippingCity = o.ShippingCity,
                ShippingDate = o.ShippingDate,
                ShippingMethod = o.ShippingMethod,
                ShippingState = o.ShippingState,
                ShippingTotal = o.ShippingTotal,
                ShippingZip = o.ShippingZip,
                SubTotal = o.SubTotal,
                TaxTotal = o.TaxTotal,
                TrackingNumber = o.TrackingNumber,
                Details = MapOrderDetailToDataObjectOrderItem(o)
            };
        }

        private static List<LinqTest.DataObjects.OrderItem> MapOrderDetailToDataObjectOrderItem(OrderHeader o)
        {
            return o.OrderDetails.Select(od => new DataObjects.OrderItem()
            {
                PartID = od.PartID,
                Price = od.Price,
                Quantity = od.Quantity
            }).ToList();
        }
    }

As you can see both public methods do the same thing. The second test was refactored easily using the Refactor-Extract Method menu item in Visual Studio. I load the orders from the database, take the SQL OrderHeader object and map it to the business object. The Details property on the Order object is simply a generic list of OrderItems. To retrieve them I do a quick lambda expression to query the OrderDetails relationship property.

For some the refactoring goes without saying. Modularizing code like this makes it more maintainable and reusable. This concept can be foreign to some procedural programmers. With Visual Studio and addins like Resharper refactoring becomes so easy it's almost an afterthought to do it. For anyone that still doesn't see the benefit with refactoring, I hope this article will help you.

The testing program is pretty simple, pass in the amount of iterations and a boolean to turn pre-JITing on or off.

[Program.cs - (some code removed for brevity)]

        static System.Diagnostics.Stopwatch stp = new System.Diagnostics.Stopwatch();
        static int Runs = 10;
        static bool PreJitRoutines = false;
        
        static void Main(string[] args)
        {
            ProcessCommandLineArguments(args);

            //Lets get JIT over all the methods in question
            if (PreJitRoutines)
            {
                PreJitTestRoutines();
            }
            //Display Current Selected Options
            Console.WriteLine("Number of Runs: {0}", Runs);
            Console.WriteLine("Pre JIT Enabled: {0}", PreJitRoutines);
            
            //Run and Measure Inline Test
            stp.Start();
            for (int i = 0; i <= Runs; i++)
            {
                Tests.RunInlineInitialization();
            }
            stp.Stop();
            //Display Test Results
            Console.WriteLine("Inline Initialization Test: {0} , Average: {1}", stp.Elapsed, new TimeSpan(stp.ElapsedTicks/Runs));
           
            //Save Result for later calculations
            TimeSpan FirstRun = stp.Elapsed;
            
            //Reset StopWatch
            stp.Reset();

            //Run and Measure Refactored Test
            stp.Start();
            for (int i = 0; i <= Runs; i++)
            {
                Tests.RunRefactoredInitialization();
            }
            stp.Stop();
            //Display Refactored test results
            Console.WriteLine("Refactored Initialization Test: {0} , Average: {1}", stp.Elapsed, new TimeSpan(stp.ElapsedTicks / Runs));
            
            //Perform and report comparisons between tests
            if (FirstRun.CompareTo(stp.Elapsed)<0)
                Console.WriteLine("Inline Construction Faster: {0:f}", stp.Elapsed.TotalMilliseconds / FirstRun.TotalMilliseconds);
            else
                Console.WriteLine("Refactored Construction Faster: {0:f}", FirstRun.TotalMilliseconds / stp.Elapsed.TotalMilliseconds);

        }

I ran the tests in release mode with iterations of 1, 10, 100 and 1000.

>LinqTest.exe 1  true
Number of Runs: 1
Pre JIT Enabled:  True
Inline  Initialization Test: 00:00:00.0124051 , Average: 00:00:00.0177619
Refactored  Initialization Test: 00:00:00.0121951 , Average: 00:00:00.0174613
Refactored  Construction Faster: 1.02


>LinqTest.exe  10 true
Number of Runs: 10
Pre JIT Enabled:  True
Inline  Initialization Test: 00:00:00.0701888 , Average: 00:00:00.0100497
Refactored  Initialization Test: 00:00:00.0650985 , Average: 00:00:00.0093209
Refactored  Construction Faster: 1.08


>LinqTest.exe  100 true
Number of Runs:  100
Pre JIT Enabled:  True
Inline  Initialization Test: 00:00:00.6291376 , Average: 00:00:00.0090081
Refactored  Initialization Test: 00:00:00.5354964 , Average: 00:00:00.0076673
Refactored  Construction Faster: 1.17


>LinqTest.exe  1000 true
Number of Runs:  1000
Pre JIT Enabled:  True
Inline  Initialization Test: 00:00:06.2699034 , Average: 00:00:00.0089773
Refactored  Initialization Test: 00:00:05.3725538 , Average: 00:00:00.0076925
Refactored  Construction Faster: 1.17

As you can see at 1 iteration there's barely a difference. As we move up the scale the refactored code does consistently outperform the inline expression. This outcome was different from my initial hypothesis. I decided to dig a bit deeper and find out why. So I pulled out ILDasm to see what was going on. I was surprised to see that the IL generated for inline test was twice as long as the refactored test. Looking at the code it became clear what was going on.

Inline IL

 IL_0042:   stloc.3
 IL_0043:   ldloc.3
 IL_0044:   ldc.i4.0
 IL_0045:   ldtoken    method instance void  LinqTest.DataObjects.Order::set_CustomerID(int32)
 IL_004a:   call       class [mscorlib]System.Reflection.MethodBase  [mscorlib]System.Reflection.MethodBase::GetMethodFromHandle(valuetype  [mscorlib]System.RuntimeMethodHandle)
 IL_004f:   castclass  [mscorlib]System.Reflection.MethodInfo
 IL_0054:   ldloc.2
 IL_0055:   ldtoken    method instance int32  LinqTest.OrderHeader::get_CustomerID()
 IL_005a:   call       class [mscorlib]System.Reflection.MethodBase  [mscorlib]System.Reflection.MethodBase::GetMethodFromHandle(valuetype  [mscorlib]System.RuntimeMethodHandle)
 IL_005f:   castclass  [mscorlib]System.Reflection.MethodInfo
 IL_0064:   call       class  [System.Core]System.Linq.Expressions.MemberExpression  [System.Core]System.Linq.Expressions.Expression::Property(class  [System.Core]System.Linq.Expressions.Expression,
 class  [mscorlib]System.Reflection.MethodInfo)
 IL_0069:   call       class  [System.Core]System.Linq.Expressions.MemberAssignment  [System.Core]System.Linq.Expressions.Expression::Bind(class  [mscorlib]System.Reflection.MethodInfo,
 class  [System.Core]System.Linq.Expressions.Expression)

Refactored IL

 IL_0005:   stloc.0
 IL_0006:   ldloc.0
 IL_0007:   ldarg.0
 IL_0008:   callvirt   instance int32 LinqTest.OrderHeader::get_CustomerID()
 IL_000d:   callvirt   instance void  LinqTest.DataObjects.Order::set_CustomerID(int32)

The inline call uses reflection on the objects to build the instantiation into the expression tree. It has to load the information about the SQL OrderHeader.CustomerID property via reflection. It then does the same thing for Order.CustomerID on the business object. After that it takes the value loaded from the sql object and binds it to the business object.

The refactored code skips the reflection entirely. Since the method is expecting an order object LINQ to SQL just needs to do what it does best, load data from the database and map it to the ORM object. The refactored methods just need to do straight property assignment

Now most of this testing was doing with the C# LINQ syntax and not the chained function calls. I'm going to dig a bit deeper and recreate this with pure lambda expressions and see how that stacks up. I have a feeling they will probably perform close to the refactored examples

Another thing that makes me curious is the plateau reached and the differences between 1 to 100 iterations. I have a sneaking suspiscion that some of this maybe related to the JIT compiler and the garbage collector optimizing for the pattern of execution. I'll probably throw in some garbage collection counters to see how different they are.

So for now the moral of the story is refactoring is not only good for code reuse, simplicity it can also help increase performance. Without moving the mapping into another method other calls would have increased JIT time due to having more code than needed. The mapping call only needs to be JITed once.

Twin Cities Code Camp

Jeff Klawiter — Fri, 22 Aug 2008 18:58:20 GMT

I figured I should mention this. I will be speaking at the Twin Cities Code Camp this fall. My talk is going to be on the PICK/Multivalue systems that are still in use today. It is one of those legacy systems like COBOL that isn't going to fade away anytime soon. IBM actually has taken the lead with it in recent years but modernizing the U2 products (Universe and Unidata).

For the people that don't know Code Camp is a day of talks on whatever speakers want to talk about. It's free and a rather fun time. In the past there have been talks ranging from starting out in ruby, writing custom modules for IIS 7 to hacking a hardware platform to tell you the whether. It is run by Magenic and hosted at New Horizons in Edina (I heard that the next spring one may be moving elsewhere).

InteliShade.Net: HLSL Syntax Highlighting and Intellisense

Jeff Klawiter — Fri, 22 Aug 2008 18:29:45 GMT

I was playing around with shaders the other day and found the lack of syntax highlighting in Visual Studio to be a bit annoying. I had downloaded the Nvidia FX composer but it's a bit more complex than I really need. After a bit of searching google I came across InteliShade.NET.

InteliShade seems to have full syntax highlighting and InteliSense for base types. So far the InteliSense support is fairly simple. It will autocomplete object types and function names. It does not detect any variables or structures you've created nor does it give parameter completion on functions. I'm sure this will change as time goes on.

It looks like there's just one guy writing it and also doesn't seem to be open source. I'm sure if there's enough interest he may open things up. I've already begun talking this thing up. It wasn't too long ago Shawn Hargreaves gave a talk at our local XNA user group and he mentioned he didn't know of any VS HLSL sytnax highlighting and urged us to bug the VS team to add it. After finding InteliShade I did send him a quick email through his blog to let him know there is now an option.

C# Event Declarations

Jeff Klawiter — Mon, 11 Aug 2008 19:20:39 GMT

Over the weekend I started Reading "CLR Via C#". I picked it up at Borders earlier this year with a certification book. I was trying to get more MCTS certs at the time and never ended up reading it. Also every time I look at it I feel like it's that black box I'll never be able to understand. I realized this is a bad attitude to have, so I opened it up and began reading. Instead of becoming lost, I feel like some things are becoming found.

I realize I should have read this book right away. So far nothing has been entirely over my head. I have had to reread some paragraphs due to sentences being heavy on the the terms. Also the author "Jeff Richter" tries to use the correct CLR terms and they sometimes clash with C# and general OO terms. An example is he uses "type" as the term for a static class.

I've made it 11 chapters in and so far it has been full of those "so thats how that works" and "thats good to know" moments. The biggest one so far has been the Events chapter. He covers a C# feature that has not been covered in any of the C# books I have read.

In C# you can declare Events like Properties. That is instead of using "get" and "set" blocks, one can use "add" and "remove" blocks. He covers this because the default implementation of events in the CLR is a bit flawed when it comes to thread safety. The CLR will lock the full object when adding and removing event delegates. He gives an example of overcoming these flaws by implementing manual locking in the "add" and "remove" blocks.

private event EventHandler<SomeEventArgs> somethingHappend;
private Object mLock = new Object();
public event EventHandler<SomeEventArgs> SomethingHappend
{
    add
    {
        lock (mLock)
        {
            this.somethingHappend += value;
        }
    }
    remove
    {
        lock (mLock)
        {
            this.somethingHappend -= value;
        }
    }
}

As you can see it looks much like a C# property. It uses a private event that is exposed publically through two blocks. Other than the keywords the other major difference is you cannot set access modifers on the "add" and "remove" blocks. It would be very bad programming to let someone add an event delegate but not let them remove it.

I have one application that I maintain where this will come in very handy. Planning on testing out the difference very soon.

I highly recommend this book for any serious .NET developer. I kick myself for not learning more about the CLR in the first place. Visual Studio sometimes babies the programmer too much I think. That and many programmers are lazy and love to just know enough to get by. I find these kinds of programmers have a very hard time working at Sierra Bravo. We have such a large variety of projects that one must learn quickly and know enough to jump the entire gammut of .NET applications. As a .NET developer here it's impossible to just be an ASP.NET developer or WinForms developer. I like it that way, it ensures that I'm never board and am learning all the time.

In my tenure so far I have written, Winforms Applications, entire .NET CF warehouse management applications, fully globalized ASP.NET web sites, various web services, various windows services, System Tray utilities, a Vending Machine controller program (implementing various serial protocols), written TCP clients for propietary protocols and much more. I have also helped companies migrate to/from MSSQL databases with various front ends. Heck, I'm even working on a C++ Win32 mobile application this week. I have done more in 3 1/2 years than a typical programmer does in 10 years.

Understanding how something works makes it much easier to implement a project with it. I found when I was a PHP programmer that I really went from amatuer to professional when I studied how PHP parses the script, allocates memory for variables. This is not an easy thing to do without looking at the source code. There are no good books out there about the "PHP Runtime". Knowing the rammifications of taking advantage of variable variables and not unsetting after use helped me take one of my more complext php programs and reduce it's memory usage 3 fold.

Got a new PC

Jeff Klawiter — Fri, 04 Jul 2008 13:43:41 GMT

I did a bit of impulse shopping yesterday. I had $15 in reward zone coupons. So I went to Best Buy after work to see if I could find a new mouse to replace my broken MX Revolution. To my dismay they had it for $20 more than what I paid for it at the same store a year ago. I started looking at wireless keyboards as well since my current set up involves a ps2 extender cable and a ps2 to usb converter.

After finding the Logitech LX 710 keyboard/mouse combo was pretty nifty I went browsing the PC section. Low and behold I found in the back of the section that have one aisle end with clearance PC's. I have no idea why I haven't noticed this before but I haven't. Most of the PC's there were unboxed due to them being display models.

I've been wanting a Home Theatre PC for quite some time now. I bought a 360 hoping that it could work as one but it refusing to play most of the videos I throw at it. I have a bit of an Anime habit and gotta watch my Bleach, Naruto and Code Geass R2 every week.

After a bit of comparison shopping on what they had their I decided to pick up a Pentium Dual Core gateway for $434.
Vista Home Premium
Pentium Dual Core E2180 (upgradable to Core 2 Quad)
2GB DDR2 PC-6400
500GB 7200 HD
Geforce 7100 Integrated Graphics (with VGA and DVI output)
Integrated 5.1 Surround Sound (with Digital IN/OUT)
12-in-1 card reader media bay
DVD Writer with Lightscribe

There was a similar one for $120 less but it used Intel graphics, had half the RAM, no media bay, no built in surround sound and no DVD burner. It's easy to do the math it would cost at least $200 to upgrade the cheaper one.

It's nice that it came with Home Premium. Saves me from using my Vista Ultimate I got from Heroes Happen {Here}. I still may install it since it'd be nice to do development on it. I have been mulling over a flash card program for the little one. I don't have the time I wish I did to help her learn her alphabet and things like that.

When I went to ring it up I found out that on top of the $15 in coupons I had it was also 5% off for it being unboxed. Then they drop the bombshell that they need to do a restore on it to remove their display software. It'd take a half an hour and the store closed before then. I was able to convince them that I could do it myself. (after all it comes with a restore disc right?).

I get home, connect it to my Sharp Aquos 42D64U, connected my new wireless mouse and keyboard and booted it up. The first thing I get is command prompt full of diagnostic utilities. Geek Squad forgot their disc in the PC. I'll probably bring it back later today. It's got all these nice warnings about property of best buy. I find this funny because at my work we've done a bunch of software for Geek Squad. The founder has been to our place many times. We just got done doing a whole new scheduling system for them. I'm not sure but it's possible we may have even worked on one of these utilties.

After taking the disk out it boots to windows and auto logs in. This is where things get fun. Their display software is called A.R.C.H.I.E (didn't write down what it meant). I thought I'd be able to just stop it from booting up and all would be good. Boy was I wrong. This software rootkits itself to the system. Everytime windows boots it gets wiped clean. All new accounts, settings, programs are wiped and everything is put back to display mode. It launches tons of little programs at the start. Most I found where actually .NET programs (easy to tell cause they used the default .NET Winforms App icon).

I wish I had monkeyed around with this A.R.C.H.I.E thing longer to find out how they rootkited it so well but I was anxious to get a fully working media center that handle any format I throw at it. I go to open the big envelop they gave me with all the PC material and there's no disk. I'm a bit dissappointed but figure, I got that Vista Ultimate I can use that. After that I realize I have my Anytime Upgrade disk for my laptop and the gateway has the Home Premium CD key on the side so I can just do a clean copy of that. I get all booted into the installer, put in the CD Key and pick the clean install option. It then gives me an option of what drive to install it on. And here I see there's another partition, SYSTEM_RESTORE. The ARCHIE stuff hid it from Vista somehow (something I'd like to know how to do.. hate having drives show up in My Computer that don't need to).

I canceled out of the setup and rebooted. I did a better look over of the boot options. There are 4: Boot Options, Setup, System Recovery and Diagnostics. I'm rather impressed by this. Normally on a Dell you get 2. So I selected System Recovery and it brings me into a nice HP/Compaq branded system restore. It lets me revert back to factory settings. Exactly what I wanted to do.

The whole recovery process took about 15 minutes. Much faster than an install. It's pretty easy when all you're doing is a quick format and then copying files on the HD. While this was installing I realized I hadn't eaten yet today, the girl and the little one left that morning to head out to see family for the weekend and we hadn't done our grocery shopping this week. So I got on my laptop and ordered a Pizza (pizza hut Grilled Combo with stuffed crust, so decadent).

I got booted in and started my process of installing FireFox first, then removing Norton and all the HP crap. After I had it clean I installed my favorite codecs, mapped a drive to my network storage and tested out the PC. I was pleased to see it handles 720p content well and almost handles 1080p. I have a sneaking suspicion that the issues I'm seeing with the 1080p is the limitations of my 100Mb network. The video and sound get out of sync after a bit. I'm going to try moving one of them locally later to check this out.

My Pizza arrived during this time and it was tempting me o so much. I sat down had a couple slices, browsed the web and installed a few more little things here and there like AVG, Flash Player and Acrobat Reader. I started up Media Center and started watching The Hogfather. It was around 1am at this point and I fell asleep quickly.

After waking up around 4 I started playing again. I got profiles set up for Media Center (set up MC to run at start up), Lacey as an administrator since she'll probably be using this when I'm not here and Avalon as a standard user. I set up Parental Controls for Avalon. I must say Vista's Parental Controls are rather nice. Don't think many people even notice they exist. Does Mac even have this??

I went and customized their profiles to add bit of personal flair. I think Avalon will like the wierd pink animal clock sidebar gadget that ships with Vista. I also went into the Games folder for Avalon's and set it to hide the blocked programs. I am also impressed with this folder. No one ever talks much about the new Windows Explorer options in vista. I find being able to combine sorts and groupings very handy.

All in all I'm pretty pleased with the set up. While the case is a bit too big to be hid away nicely I can deal with it. The PC is not loud at all. While I was running The Matrix 1080p and had the sound off I could just barely hear the fan going. It is definitely quieter than the 360. I'm probably going to put some tape over the Power and HD LEDs. Too many of those things and my living room looks like a scene out of a sci-fi movie at night.

My next plans are to pick up a digital coax cable and hook it up to my surround sound receiver and get a DVI-to-HDMI adapter. VGA cannot handle doing 1080p output on the TV. The TV doesn't support it through VGA and when I do force support it works but I only get about 5fps. I wish my TV had come with both DVI and VGA connectors.. the one thing that it's missing in my eyes.

I also may go pick up a new video card. I was disappointed when I saw that Best Buy was out of their All in Wonder Radeons. I had just read they were coming back but didn't know that they already hit the store shelves. From what I could gather from the name it was 4830 with the 650 tuner chip and HD support. AMD has much better HD decoding support and having an HD tuner on top of it would be awesome.

Well I think I've rambled on long enough. I'll try to post some pics later

Unit Testing for the first time

Jeff Klawiter — Mon, 30 Jun 2008 02:14:50 GMT

While I've heard lots about Unit Testing for years now I've never been able to force myself to implement them. I've always thought it'd take too much time. Time I could have spent writing code, getting work done. Then I go the project I am on now. It is a large project with a very tight deadline. A project that needs to marry different back end sources.

I've been put in charge of architecting the SQL Server back end and the entire middle layer for data access and business objects. I'm using many things I've not had a chance to use before at this scale (LINQ being the largest) in this project so there were many unknowns. The front end may not been done for a bit and I needed to test my data access functions. I decided to bite the bullet and write unit tests.

Normally I would write a small WinForms app to test the most important pieces. This is a dirty approach but it's fast. The biggest hurdle in my mind for Unit Testing was there was no quick way for me to implement and run the tests. Nunit is out there and has some VS integration but whenever I tried it I got confused of where I needed to go to get things done.

Now we have VS 2008 Pro with built in unit testing. While the testing framework included I'm told is decent it's not as good as Nunit. I personally don't care about that. The ability to go to the menu and create unit tests for methods or entire projects on the fly is a godsend for me. VS will create a project for you, include tests for any classes/methods you've selected out of all the projects in your solutions and then do it's best to create close to complete tests.

You can easily step into debug any test and run one or many tests. All the results are saved so they can be examined and compared. While the auto created tests need to be completed I am starting to find that writing to test my code interface is getting cleaner and more too the point. It's much more annoying to pass an object into a method that may only need one or two properties on the object. It's better to make the parameters basic types instead of complex.

The tests have to run against 2 different backends. While one of them will probably not change (the SQL Server) the webservice may end up being replaced. These tests will be invaluable for when that happens. We can then check to see if the new service is acting correctly and if not pinpoint the issues.

Project Documenting under a deadline

Jeff Klawiter — Mon, 30 Jun 2008 01:57:50 GMT

Documenting code is one of those things that programmers tend to leave out while under a tight dealine or just building a new project. It's one of those things we tend to say, if we have time when we're done we can do it. It can be time consuming and sometimes hard to see the benefit up front. I'm very guilty of poor documentation and commenting. I rarely get to work in a team so I always end up relying on my own memory for what does what and where. Then two years go by and I forget it all.

I have been trying to get better about adding at least XML comments to my classes, methods and properties. The project I am on right now is a rather large project due in less than a month and so far we have a semi-concrete middle layer, a half mocked up front end and a half done back end. We need more than just one or two people on the project and I've been in charge of part of the back end and all of the middle layer. Even though it is taking a bit more time I've been trying to add documentation and actually use Unit Tests for my first time. (that's a whole other entry).

While the built in XML Commenting system C# has is great and can really help a programmer when coding there is little to no good way of taking that and making separate documentation out of it. That and editing it can be a real pain. After pouring over the web for some way of exporting the XML as a CHM or MSDN type library I came across DocProject . This wonderfully free and open source VS 2005/2008 Addin is a godsend on this front. It can create a standalone CHM or give you an MSDN library to integrate with your VS or create a web project that has a decent Ajax interface for browsing the documentation online. All export options can be done at once as well.

DocProject lets you edit every part of the XML Documenation in a rich text editor. While I think the editor could use a bit of work (make it easier for putting code in the Example and other regions code can exist) it does a great job of making it much easier to write the documentation instead of just in code.

DocProject does have a bit of a draw back in it's requirements. To use it you must install Sandcastle, which isn't to large. If you want to output in the new Compiled HTML 2.x format you need to install the VS 2008 SDK. The SDK however is nearly 100MB. Though I'd recommend having it anyway, with it you get the ability to write VS addins and other fun stuff.

I just started using DocProject today, so far I'm impressed and I hope it turns out to be a good addition to the development process of this project. I'm hoping to use DocProject's publish features to put the documentation on a site where other developers for the project will be able to reference. I've found over the years that the projects with the most documentation end up being the easiest to work on.

Updates

Jeff Klawiter — Fri, 27 Jun 2008 12:58:48 GMT

Well I've been working on the new theme for the blog. I have been co-opting the theme from j-maxx.net since it's about the only decent theme I've ever done before. It looks bad in IE due to dashed borders having white space inbetween the dashes instead of the background color.

For some background on the blog. I chose dasBlog as my software due to it having web services built in and has a user base of programmers I look up to. That being said it was originally .NET 1.1 software and the design is getting a bit annoying in some aspects. I'm looking at adding a better syntax highlighter, but adding it so it shows up in all the themes is not straightforward. I may just remove the other themes and only allow this one to simplify things.

I've also noticed that blogs edited after the date they were posted do not show up when you click on the date their were posted in the calendar. I'll have to track that bug down.

All in all it was pretty quick and easy to get the blog up and running. Designing the theme was very easy as well. They did a great job at allowing users to separate design from logic. I'm going to delve more into the templating system to figure out how they did it.

The blog hath arrived

Jeff Klawiter — Wed, 25 Jun 2008 00:25:49 GMT

Well I'm finally opening a real blog. Not sure how much I will be updating this but hey I have a real online presence again. The old J-Maxx Net will stay as it is. Still get a decent amount of traffic on there (kicking myself for not installing adsense years ago).

To start off with.. I love LINQ. I've been looking into it for over a year now but finally have a project where I get to use it fully. C# 3.0 has added so many features I no longer pine for PHP as I once did. Here's a sample below of something I wrote the other day.

            var result = from c in CurrentDataContext.Categories
                         join localinfo in CurrentDataContext.CategoryLocalizationInfos
                            on c.CategoryID equals localinfo.CategoryID
                         where c.CategoryID == CategoryID
                         && localinfo.Language == Language
                         select new Business.Data.Category()
                         {
                             ID = c.CategoryID,
                             Description = localinfo.Description,
                             Languages = GetLanguagesForCategory(c.CategoryID),
                             Name = c.Name,
                             SortOrder = c.SortOrder,
                             Title = localinfo.Title,
                             NavImage = c.NavImage,
                             Language = localinfo.Language,
                             Links = GetLinksForCategory(c.CategoryID, Language),
                             IsDataLoadedFromSql = true
                         };