Mind over Microsoft: curing DataSet autism

The deeper I dig into Visual Studio 2005 the more I get the impression that it contains more obstacles than features. It isn't enough to just think something over and figure out how to do it: each step of the implementation may hide a showstopper - either a bug or a crippled feature. You have to check very carefuly if all you thought of can really be implemented.

Here's a good example on how you can get sucked into "Visual Studio does everything by itself" Microsoft agitprop. I wanted to work out a simple solution for a non-creative task which is making search forms for my windows application. You know, search: enter what you want to search for and you get a list of records from the database. A form with a data grid in the center, some text and combo boxes above it for the user to enter search criteria into, and a SEARCH button.

What would be the best way to implement this kind of component, to make mass production as fast as possible? Let's see... I'd have to utilise the design-time functionality to the maximum. There could be a base class containing most of the logic and all common components: say, an empty DataGridView which the user could configure from inside a derived component to add columns etc. The dataGrid would be bound to a BindingSource which is in the base class, but its DataSource type (the DataSet used for search operation) should be set from the derived component so the designer could know what properties exist in it.

Then, in the derived component, a developer would add a new data source for the BindingSource - i.e. create a DataSet, modify its sql query and voila, the grid's columns are automatically initialized and prepared for customization. The base class would pick up the DataSet type being used and wire up the logic (i.e. SearchButton_Click handler) for filling the DataSet - after all, the DataSet has a generated TableAdapter configured and ready to go.

Now, how to enter search criteria? This one could be interesting: write the DataSet SQL with prepared parameters in the WHERE part, like "SELECT something FROM something WHERE (column1 = @param1 OR @param1 IS NULL) AND (column2 etc.)". Next, create a separate BindingSource in the base class for search criteria. In the derived class, bind it to the table adapter's parameters, then add text and combo boxes to your liking and bind them to individual parameters. Ok, this could be problematic, but an ICustomTypeDescriptor wrapper class or something could help turn the adapter parameters into something bindable. So, using high-tech Visual Studio designer components, the user gets to make a complete search form in half an hour. Wow!

Um, I hope you didn't read this post from the middle? Because you could have gotten the wrong impression that this could be done. The cruel reality is that most of the abovementioned doesn't work. Let's start from the beginning.

Have you ever looked at the generated DataSet source in VS 2005? I don't know if it's the same as in VS 2003, but it's not just ugly, it's absolutely hideous. I thought that if I notify my component of the type of DataSet being used, it could somehow detect the proper DataAdapter from it. Not likely: not only there is no property on the DataSet that could point to the DataAdapter type (let's say this is ok, there could be more than one DataAdapter for a given DataSet), but the DataAdapter is not even in the same namespace! Why isn't it a class embedded in the DataSet? Nobody thought of that. It doesn't seem there was a lot of thinking invested here anyway.

But this is not the end of it: the generated TableAdapter is not a table adapter at all, it's a class derived directly from Component - in other words, a nobody! The real DataAdapter is in its private property. (Aarrggh, ok, we'll use reflection to access it then). Yeah, but the real adapter is not initialized when the quasi-TableAdapter is instantiated. It's initialized only when you call the Fill method... And since there is no base Fill method (remember, the base class is a Component), the only option is to call the generated Fill method and feed it parameter values. So, if I'm going to use reflection anyway to get the private property, why not call the Fill method directly.

Ok, never mind: use reflection to get the adapter, call all the proper initialization methods using reflection and hope their names don't change in future versions. (Fat chance, I'd bet this code survived from the VB days. I tried finding the DataSet designers using Reflector and stopped when at some point the .Net code jumped through the interop mirror over into the COM world... Well, that's true at least for the sql designer - which is the most useful part of it anyway).

Now, to make an ICustomTypeDescriptor to bind to the sql query parameters. Well, it can't be done: BindingSource doesn't support ICustomTypeDescriptor. Just think of it: what do you get when you add two cool technologies together? Nothing, because they've been made by Microsoft. Do the guys over there talk to each other at all?

So, I resorted to a brute-force solution: created a new wrapper class complete with a VS.Net designer that detects parameters from a given TableAdapter and creates an accessor property for each of them. So I can bind to the wrapper and the wrapper will then call the TableAdapter.

Have you started wondering why I haven't chosen to abandon DataSets and create my independent components? I did think about it - but at this point it is pure bloody-mindedness that drives a man to beat the damn technology. It's a challenge.

Yeah, right. Next step: the DataGrid. In the derived class, we configure the BindingSource to use the generated DataSet and then get the DataGrid to automatically configure its columns from it. Then we just reposition and format the columns and -

Well, surprise surprise: visual inheritance is turned off in VS 2005. (I know it's old news, this is a story that was a long time in the making: just look at the sheer volume of text above). So - man, do you still want to beat the technology?

I had to make my own templating mechanism (an extender provider, actually: these are very useful) so one can put the controls into the derived class and then set their "extender-ed" property that says into which area of the template it should be placed. So the base class picks up thusly marked controls and knows which is which.

Next? Binding issues: the pre-SP1 data binding designers in VS 2005 are extremely fiddly (the post-SP1 ones are just fiddly). You can bind controls to datasources but you aren't sure you'll be able to open the designer again in the future. The class properties don't always appear in the data sources toolbox. And the ones that do, don't appear when you try to bind from the properties window. If you want to bind to a property's property, you'll have to write the path manually. Jesus Christ! Today they'd give you Visual Studio .Net instead of a cross.

Wait, there's more: the DataSet designer resets most of the manually set properties (like parameter or column types or their AllowDbNull property) every time you reconfigure it. Before you start modifying a DataSet, you have to memorize its properties' values first. If you forget a single detail, you either won't be able to compile or you'll introduce a bug you'll waste some more time fixing (possibly at a later time when you forget what you did).

And more: I'm not quite sure how connection strings are supposed to be handled in VS 2005 SP2. If I try adding a new connection string to the project, it doesn't show up in the dataset designer. I suppose the designer caches existing connection strings in the XSD files but never seems to refresh them. If I try to rename one of the connection strings, I either get a refactoring error (the refactoring engine somehow destroys a property in the dataset's Designer.cs file) or I end up with crippled XSD files (half of its XML simply goes missing). Sometimes, for reasons unknown to me, a one of connection string names gets a full path with a namespace, and XSD starts reporting errors until you fix it (turns out it cannot stand dots in connection string names).

I'd already lost enough time battling with Visual Studio 2005 that any productivity improvements from using it - and after all this I seriously doubt there will be any surprises any time in the future - cannot compensate for it. It did sound like a good solution before I started implementing it, but that is because it was based on wishful thinking fueled by Microsoft marketing. One has to be very careful about these things: for any such adventure, multiply your time estimate by three.

How to get the containing folder for a ProjectItem in a Visual Studio 2005 designer

It was a long search, and I must say that the item in question was very well hidden (congratulations to whoever put it there :)). The problem was this: how do you, from inside a Visual Studio designer or add-in, get a reference to the current ProjectItem's containing folder? And I don't mean the folder on the disk, but the Visual Studio project folder that may not be the same.

You'd never find it... It's in ProjectItem.Collection.Parent! Here's some sample code (the idea is to put the code into a component designer: I haven't tried this actual code, but the basic idea is the same as what worked for me):

public ProjectItem FindActiveDocumentsSibling(string fileName)
{
// we need the DTE to get the active document
DTE dte = (DTE)Component.Site.GetService(typeof(DTE));

Project ret = null;

// try-catch is necessary because Item() throws an exception if no item is found
try
{
   ((ProjectItem)dte.ActiveDocument.ProjectItem
.Collection.Parent).ProjectItems.Item(fileName);
}
catch { }

return ret;
}

So, what's this for? Well, I'm trying to put an NHibernate *.hbm.xml file next to a code class. (Look at a previous post as to why it isn't a child item but a sibling). Now if I could only figure out how to (or whether I should) open it using Visual Studio mechanisms - but without showing it to the user...

How to restore/reset lost Visual Studio 2005 menu items

For some strange reason, installation of various add-ins and add-ons for Visual Studio 2005 causes it to reset its menus - and possibly other settings. But they don't get reset to installation defaults: rather, Visual Studio goes into some kind of minimalistic state where you don't get a "Macros" item in the "Tools" menu anymore, the "Attach To Process" and "Exceptions" commands get lost etc. At first I resorted to going into the Customize dialog and bringing back the items each time (although I never was quite able to figure out how to correctly get back the "Build [project name]" item: occasionally it either appears twice or disappears). I did so much installing/reinstalling of various CTP's that eventually I gave up on customization and started using keyboard shortcuts instead of menus. Now, finally, I found out that there actually is a simple way to reset settings to "normal" instead of "obscure" (seems I hit the right magic word with Google: my searches for "reset Visual Studio 2005 menu" and such led nowhere, but once I asked it how to restore the Macros menu, I struck gold :)). The solution is as follows: go to Tools -> Import and Export Settings and choose Reset all settings.

The original link:

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1114019&SiteID=1

Fix for the CAB sample visualiser bug

The Composite UI Application Block has a very useful feature called "visualisers". It's purpose is to give the developer an overview of what is going on inside a CAB application, which, due to a large amount of various mappings and automated dependencies, could othewise become an overwhelming task.

Great idea - but how about an implementation? Well, there is some documentation (very obscure, just like the rest of the CAB docs) and a single sample visualisation. The sample could be useful were it not buggy: it shows you a treeview with WorkItems and SmartParts in your application, but holds a strong reference to them (or something like that) which prevents them from being disposed. So, in effect you don't get a real overview of your application's state. I didn't know about this bug and wouldn't have noticed it were it not kindly pointed out by the CAB people themselves. But it seems it was too much for them to fix it (strange because the sample visualisation consists of a single windows control).

Or they waited for the community to do it - which at the end someone did. This seems to be the philosophy Microsoft is toying with these days. They seem to want to copy IBM's success with supporting open source projects like Eclipse, but are loath to give anything to open source. Microsoft, like always, wants it all - commercial software supported by a free community. Is the lacking documentation for CAB also supposed to be supplemented by community posts on CodePlex? Come on, people, it's been a year since CAB was released and all we have is a package of hands-on labs and a MSHelp file compiled from XML documentation in the source code.

Visual Studio 2005 Service Pack 1, part four: recovery

How to recover from a failed Visual Studio 2005 service pack 1 installation (or how I did it, in any case).

It seems that with a failed setup there is a hung instance of msiexec left running (it can be seen in the task manager) and some files are kept locked in the c:\Config.msi folder and cannot be deleted even when this msiexec is killed. I didn't have the time to fool around restarting the installer service but restarted the whole windows instead. I wouldn't be too surprised if the installation error was actually caused by the installer encountering a locked file.

So, these are the steps I made to recovery:

• Made some more space on the C: partition. In the end it totalled 2.7 GB, but it could be possible to succeed with less. This is not counting the TEMP folder which I moved to another partition (you do this by setting the environment variable in control panel -> system). Note that you can free up a lot of space by going to c:\windows\installer and deleting the previously copied SP1 packages - look for files of 466 MB with recent time of creation.
  
• Went to the c:\WINDOWS\WinSxS folder and deleted files with "8.0.50727.762" in their name, like it was mentioned in this post (scroll way down). Note that the post says I should delete files named like "8.0.50727.163" but I was unable to find any.
• Restarted windows. I believe this is important - at least find and kill the hung msiexec process and restart the windows installer service.

Visual Studio 2005 Service Pack 1, part three: check that!

Waitwaitwaitwait...

BEFORE you start installing the VS 2005 SP1, make absolutely sure you have enough disk space... Otherwise, you will surely regret it: if the installer runs out of space, it will mess up your .Net installation and not only Visual Studio will stop working but an assortment of other applications. I had the same problem on two installations, after the two mentioned successful ones.

The installer dumps a 460+ MB installer package into your c:\windows\installer folder and names it differently every time. So if you cancel an installation (e.g. to uninstall the damn web applications add-in that I always forget to remove) you could end up with multiple copies of the same half-gigabyte package and thusly short of disk space. The installer then reports that insufficient space is available and even if you clean up your disk and click retry it will eventually die with error 2908 and leave your system in a demented state.

So what do you do? Some say you need 4 gigs on your hard disk to make it work. I personally redirected the TEMP environment variable to another disk with lots of space, but that is only part of the solution because only part of the files go into the temporary folder. There is a batch file (see links further down) that turns off some kind of caching (haven't had time to go into details) that could save space and make things somewhat faster.

I'm trying as we speak (well, sort of) to repair a destroyed VS installation so stay tuned for results. In the meantime, some useful links:

How to disable the patch cache and make the installation somewhat faster (so you can retry more times in 24hrs :)):
http://weblogs.asp.net/jgalloway/archive/2006/12/19/things-i-wish-i-d-known-before-i-installed-vs-2005-service-pack-1.aspx

A sum-up on what goes on and how to get yourself out of the mess if you got into it:
http://asztal.net/2006/12/and-the-prize-for-worst-installer-of-2006-goes-to/

How to get a debug log from the installer (hopefully you won't need it):
http://blogs.msdn.com/astebner/archive/2005/03/29/403575.aspx

Visual Studio 2005 Service Pack 1, part two: installation

Ok, I've installed the bugger... (Umm, bugger? This is actually an interesting attribute for anything related to software development :))

First try: installation on a "clean" machine (no SP1 beta, no CTPs and the like). Forced me to uninstall Web Application Projects add-in: the installation package claims the add-in is included in the SP (although there have been controversies about this on the net). The installation takes a bit more than an hour. I don't remember how much it took to install Visual Studio, but compared to SP1 beta's 3-4 hours, this is nothing.

Upon first installation, I tried randomly opening several forms, controls and our components with our own custom-made designers. Either I was (un)lucky or Visual Studio really started behaving much better, but I couldn't get any of the errors I kept getting earlier (like "path is not a legal form"). I did find one bug from earlier though, and it was the first one I tried: if you try to "Open containing folder"while you're in a exception-breakpoint, Visual Studio will freeze until you force-close your application.

Second try, over a SP1 beta installation. Well, it's also not true that you have to uninstall SP beta, at least on Windows XP. At least in my case... Once again, no problems.

As for the quality of the service pack, it seems satisfactory. Visual Studio does seem to run more smoothly. I haven't experienced any slowdowns, component loading errors don't pop up as often etc. I even noticed that marking of container foreach statements when you type "break;" into the C# editor once again works.

So, the conclusion? Visual Studio works better with SP1 than without. It almost seems to behave nicely, but it's early to say. In any case - recommended.

Visual Studio 2005 Service Pack 1

The Visual Studio Service Pack 1 is here, hooray... We'll see what it brings, but I won't be surprised if its installation disclaimer contains a phrase like "don't expect it to solve any of your problems". According to some bloggers, you should uninstall May LINQ CTP before you install this one... Now, I'm sure I do have some of LINQ CTP's installed, but I haven't the faintest which one it is since LINQ is in its own brand of chaos with various plugins, addins and designers all being of different versions (so, is the september designer CTP the one that works with may LINQ CTP? I think I have a september CTP of something...)

But that's OK, LINQ is a pre-release software and you're expected to have some trouble with it. On the other hand, Service Pack 1 is kind of a post-release, so... We'll see. For the time being, I heard that its installation takes longer than the initial Visual Studio installation. This is because it -- according to Microsoft -- does more tasks than the original installation which just copies files. Makes sense when you say it that way. The Beta 1 had a different explanation, they said it was slow because it's beta. Some guys say you have to uninstall SP1 beta before you install this one, this could mean even more wasted time. Not cool if it's true.

And, last but not least, I'm almost looking forward to having this installation reset my Visual Studio UI again. I've lost my Debug/Exceptions, Attach To Process and various Build menu commands so many times just thinking about it happening again makes me laugh (I haven't restored them this time, so go hide them if you can!) I don't think I care anymore.

Under the hood of Microsoft applications

Several blog posts ("Ankh and AccessViolationExceptions", "Ankh and Exclude From Project", "Losing faith", and others) from Arild Fines, one of AnkhSVN authors, reminded me how peeking under the hood of Microsoft applications can be a horrific experience. Anyone who has ever tried to create a non-trivial plug-in for Office or Visual Studio can confirm this. You work with events that don't get fired or get fired at the wrong moment (like Document.BeforeSave, among many), undocumented features and bugs that will slow your brain to a crawl, let alone your development, and force you to use workarounds so ugly you'll end up hating yourself :). Of course, doing it in C# brings additional interop-related problems and you have to resort to reflection for many operations that would otherwise be straightforward. That is why I'm always suspicious of Visual Studio plug-ins, especially the ones that replace Visual Studio features that didn't work well in the first place, like source control. AnkhSVN is probably a case in point: these guys are struggling to get this thing working, but there's always a danger of triggering a ridiculous bug in the environment that will make a feature unusable. Of course, if AnkhSVN blows, people will blame AnkhSVN, not Visual Studio. It's a bit like the endless argument between Microsoft and device driver makers about whether unstable drivers caused Windows to fail or bad quality of Windows caused instability in drivers. It is sobering, though, once you see how much trash is hidden under the surface of Microsoft applications and how much effort was invested into hiding bugs rather than eliminating them.

Codename whirligig

People inventing codenames at Microsoft had finally gone mad. Microsoft's implementation of AJAX, formerly codenamed ASP.Net Atlas has now been renamed to ASP.Net AJAX. What an original idea! I have a hunch that the same guys named Microsoft Expression, formerly codename Acrylic, formerly Fractal Expression. It's a pity that court order prevents them from renaming Microsoft's implementation of Java to "Microsoft Java (formerly codenamed C#)" :).