Random thoughts about AnkhSVN, .NET, Powershell and other stuff

1.0.3 prerelease available

Mon, 18 Feb 2008 20:58:05 GMT

I invite everyone to try out the pre-release 1.0.3 version(almost done, just need to work out a kink with the installer). It goes a long way towards fixing the annoying problem where statuses aren't populated on projects that are in solution folders. The solution isn't perfect, and we have a long-term plan for a better way of doing it, but it's a huge improvement over the current (1.0.2) state of things.

Find it at our new host: http://ankhsvn.open.collab.net/servlets/ProjectDocumentList?folderID=199&expandFolder=199&folderID=0

AnkhSVN now on openCollabNet

Mon, 18 Feb 2008 20:52:57 GMT

Hosting for the AnkhSVN project has now moved to http://ankhsvn.open.collab.net/. openCollabNet is "the online community extension to CollabNet products and Subversion" and the move represents an official commitment by CollabNet (the original sponsor of Subversion) to the AnkhSVN project in the form of developer time and other resources.

The Subversion repository has also been moved, and the new repository URL is http://ankhsvn.open.collab.net/svn/ankhsvn/trunk. Unlike before, anonymous access is no longer available, but you can checkout the repository using the username "guest" and a blank password. You can also browse the repository using the web interface at http://ankhsvn.open.collab.net/source/browse/ankhsvn/

AnkhSVN 1.0.2 released (security fix)

Tue, 28 Aug 2007 06:55:33 GMT

Following the announcement of the security vulnerability in pre-1.4.5 Subversion, I am pleased to announce the release of AnkhSVN 1.0.2. This release can be downloaded from http://ankhsvn.tigris.org/servlets/ProjectDocumentList?folderID=7315

This release links to Subversion 1.4.5, which fixes the security vulnerability described in http://cve.mitre.org/cgi-bin/cvename.cgi?name=2007-3846. There are no other changes from 1.0.1.

AnkhSVN 1.0 released!

Tue, 23 Jan 2007 21:10:52 GMT

I'm happy to announce the release of AnkhSVN 1.0. The release can be downloaded from

http://ankhsvn.tigris.org/servlets/ProjectDocumentList?folderID=7315

This release was built against Subversion 1.4.2, Neon 0.25.5, openssl 0.9.8a and Berkeley DB 4.4.20

Changes from RC4(*):

Work around a couple of AccessViolations that would be thrown from the VS object model in certain cases.
Fix refreshing of projects and projectitems statuses after certain operations.
Fix some spurious expansions of the solution explorer in certain cases.
More icons from DamienG
Workaround for the error SVN throws when deleting a missing-but-added file.

* Full changelog can be viewed at http://ankhsvn.com/AnkhWiki/Changelog.ashx

Ankh and AccessViolationExceptions

Sun, 12 Nov 2006 18:24:41 GMT

Lately, I've been receiving a number of Ankh error reports that had an AccessViolationException. This was particularly weird because you cannot get that exception from safe managed code. AccessViolationException is the exception you get when you try to access memory you're not allowed to access, usually because you are dereferencing an invalid pointer. Since safe managed code does not expose raw pointers, it's not an exception you usually see in .NET development.

As it turned out, the exception came from a property accessor on an object from the Visual Studio automation model, a Project object. This is a COM object exposed to .NET and is implemented by the implementor of a specific project type, such as a C# project or a C++ project. Being a COM object, it can, of course, be implemented in a non-managed language, usually C++, which I suspect is the case here.

That answered the question of why such an exception could be caught inside Ankh, but it did not answer why we were seeing it in this particular case. It took a while for me to be able to reproduce it, but one submitter of an error report finally gave me a reproducible recipe: he had clicked on a linker error message in the Visual Studio error list. Linker errors aren't associated with a specific source file, so clicking it usually does you no good, but in this case, it was clear that Ankh was doing something in response to this clicking and in the process, triggering the exception.

All files opened in Visual Studio belong to a project. This holds true even if you open a file that's not part of your current solution, or if you don't have a solution open. In that case, Visual Studio has a special project called "Miscellaneous Items", into which all files not part of any other project are put. This project is usually not shown in the solution explorer, but you can enable it by checking the "Show Miscellaneous files in Solution Explorer" checkbox in Tools->Options->Environment->Documents.

This project is not always present; rather, VS will load this project when the first "miscellanous" file is opened and close it again when no such files remain. Ankh has code to detect when a project is loaded and unloaded, and when a project is unloaded, we hook up an event sink to handle the case where files are added to or removed from the project, so we can refresh the status icons in the solution explorer. However, when such an event is raised, we cannot perform the refresh right away, because for some weird reason, the object model for the items hasn't been updated to account for the changes yet. What we chose to do here is to schedule a delayed refresh of the item in question; ie, perform a refresh after a specific amount of time after the event.

What happened in this case was this:

User clicked on the linker error in the error list.
This, for some reason, triggered Visual Studio to load the Misc. Items project.
Ankh then dutifully hooked itself up as an event sink for events on the Misc. Items project
Visual Studio would then trigger the "Item added" event for the Misc. Items project (I'm not sure what it actually added, and don't think I'll spend much time trying to investigate it).
Ankh would then schedule a delayed refresh of the project.
Visual Studio would then, probably because it suddenly realized there was nothing to show, unload the Misc Items project again.
Ankh would unhook itself as an event sink for the Misc Items project.
However, the delayed refresh would still fire after its given interval.
The refresh would then work against a Project object representing a project that was no longer loaded. Accessing its properties at this time triggers the AccessViolationException.

The above sequence is a little simplified - Visual Studio did actually fire several Item Added and Item Removed events before unloading the Misc Items project, but it illustrates the problem.

One unanswered question here is why the COM/.NET infrastructure allowed the Project object to become invalid when there was still a reference to it in the managed heap. We've had a similar problem with ProjectItem objects, but in that case we got a more informative exception stating "ProjectItem not available".

Oh well, now that I know what the problem is, I only have to find a solution :-)

Ankh and Exclude From Project

Fri, 10 Nov 2006 21:09:59 GMT

There was a user on our users mailing list that asked if there was a way to remove a file from an Ankh-managed project without having Ankh delete it. I was about to tell him to use "Exclude from project" when I realized I should at least test it first (it's not a feature I use a lot). I was pretty shocked to see that we do in fact delete files when Exclude From Project is invoked (if they are versioned - we don't touch unversioned files).

I was even more shocked to see how VS behaves in this case: The API we work against allows handling of delete (as well as rename and add) by implementing a certain interface. This interface has two sets of methods, OnQueryRemove* and OnAfterRemove* (with * being either Files or Directories). Normally, VS will first call OnQueryRemove*, where we have the option of cancelling the delete, and if we don't, call OnAfterRemove*.

In the case of "Exclude from project", OnQueryRemove* is not called.

However, OnAfterRemove* is called, passing in a flag of VSREMOVEFILEFLAGS_IsDirectoryBased, which is documented as "Deprecated. Do not use."

Gah.

AnkhSVN 1.0 RC4 released

Sat, 28 Oct 2006 11:29:42 GMT

In the time since RC3, it became obvious from the error reports we received that renames and deletes were very buggy. Deleting a file and then committing it would often result in an error message saying "ProjectItem not available". It was obvious that we could not release it as 1.0 as it was, and another release candidate would be necessary. We also received several reports of other problems that turned out to be rather critical to some people, or so trivial to fix that we considered it worth the risk.

It took some time to rework the handling of deletes and renames to avoid this error (it's a very tricky problem, I might write about it some day), thus the long delay in producing RC4. However, it is now available at http://ankhsvn.tigris.org/servlets/ProjectDocumentList?folderID=7315.

Problems that have been fixed include:

The very common "ProjectItem not available" error.
Templates for performing merge and diff using TortoiseMerge no longer has quotes.
Reporting Services (and other Business Intelligence projects) now work.
New icons (from DamienG).
Solution load timeout in VS 2005 is increased from ten seconds to fifteen minutes.
Various embarrassing NullReferenceExceptions.
New lines entered for the log message in the configuration dialog are now treated properly.
Handle an edge-case which prevented deletes from working with F# projects.
Work around a NotImplementedException thrown in certain cases by certain third party project types.
Various bug fixes in the support for database projects.
Message box now shown if one or more items could not be Locked (previously this information was shown only in the output pane, and easily missed).
Avoid a weird race condition in the repository explorer which might throw an exception in certain edge cases.

New wiki

Mon, 23 Oct 2006 17:48:40 GMT

I have now changed the Wiki software for the AnkhSVN wiki to the wiki engine everyone is blogging about, namely ScrewTurn. I was immediately impressed by just how slick ScrewTurn was; dead easy to install and configure.

I have imported all the history from the previous Flexwiki-based wiki, however, historical versions lack proper formatting.

Unlike the previous wiki, this one requires users to be signed up in order to edit pages. This should let us avoid the wiki spam that plagued the old wiki.

Of course, it's still a bit lacking in content, but I'm sure this will pick up quickly - ScrewTurn is so much more fun to work with than FlexWiki.

Damien Guard new AnkhSVN committer

Thu, 12 Oct 2006 16:51:02 GMT

Damien Guard, more known as DamienG, is a new committer on the AnkhSVN project. He's done some very good work for us already on redesigning the command icon set, and we look forward to working with him on new features in the post-1.0 timeframe.

The new icons will be available in the upcoming RC4.

Powershell MSBuild task

Wed, 13 Sep 2006 15:35:25 GMT

I have written an MSBuild task that hosts a Powershell runspace and allows you to embed Powershell scripts in your MSBuild build files. While MSBuild ships with a large number of useful tasks (and there are also a lot of third party tasks available), there is often something you want to do which doesn't fit into the tasks provided. You can always write your own tasks to achieve what you want, but this can often become tedious to maintain since you will have to recompile and redeploy the custom task every time you need to make a little change. Having small snippets of script embedded in the MSBuild file itself can be a more easily maintainable solution, and what better language for this than Powershell?

As a custom task, the Powershell task needs to be loaded into your build file with the <UsingTask> element:

<UsingTask AssemblyFile="PowershellMSBuildTask.dll" TaskName ="Powershell"/>

Since tasks in MSBuild don't have access to the contents of their tags, the scripts themselves need to be MSBuild properties, declared in a <PropertyGroup>:

<PropertyGroup> 
	<HelloWorldScript>Write-Host "Hello world"</HelloWorldScript> 
</PropertyGroup>

Now, to invoke this script, you will use the Powershell task and invoke it from within a target. You reference the script property the same way you reference any other MSBuild script property:

<Target Name="HelloWorld">
  <Powershell Script="$(HelloWorldScript)"/>
</Target>

The full MSBuild file will now be something like this:

<?xml version="1.0" encoding="utf-8" ?>
<Project DefaultTargets="HelloWorld" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
	<UsingTask AssemblyFile="PowershellMSBuildTask.dll" TaskName ="Powershell"/>

	<PropertyGroup> 
		<HelloWorldScript>Write-Host "Hello world"</HelloWorldScript> 
	</PropertyGroup>

	<Target Name="HelloWorld">
		<Powershell Script="$(HelloWorldScript)"/>
	</Target>
</Project>

Save it as HelloWorld.proj, place PowershellMSBuildTask.dll in the same directory and run it like this:

msbuild HelloWorld.proj

The output should be as follows:

PS D:\tmp> msbuild HelloWorld.proj
Microsoft (R) Build Engine Version 2.0.50727.42
[Microsoft .NET Framework, Version 2.0.50727.42]
Copyright (C) Microsoft Corporation 2005. All rights reserved.
Build started 13.09.2006 19:02:21.
__________________________________________________
Project "D:\tmp\HelloWorld.proj" (default targets):
Target HelloWorld:
    Hello world
Build succeeded.
    0 Warning(s)
    0 Error(s)
Time Elapsed 00:00:05.09

Not horribly exciting, but it works.

Of course, such a task would have limited usefullness if there wasn't a way to pass arguments from MSBuild into the custom script. The Powershell task has two ways of doing this.

The first way is an attribute 'Arguments'. This takes in named arguments on the form 'varname1=value1;varname2=value2'. varname1 and varname2 will then be available as string variables inside the script. For example:

    <PropertyGroup>
        <HelloArgumentsScript><![CDATA[
        Write-Host ("{0}" -f (([int]$value1) + ([int]$value2)))
        ]]></HelloArgumentsScript>
    </PropertyGroup>

    <Target Name="HelloArguments">
        <Powershell Script="$(HelloArgumentsScript)" Arguments="value1=40;value2=2"/>
    </Target>

The output is as expected:

Project "D:\tmp\HelloWorld.proj" (HelloArguments target(s)):
Target HelloArguments:
    42
Build succeeded.
0 Warning(s)
0 Error(s)

Time Elapsed 00:00:02.78

The other way is through the task attribute 'TaskItems'. As the name implies, this takes in a set of taskitem objects. They are available in the Powershell script as a collection of ITaskItem objects named $TaskItems:

<PropertyGroup>
    <TaskItemsScript> <![CDATA[ Write-Host ("The answer is {0}" -f (([int]$taskitems[0].ItemSpec) + ([int]$taskitems[1].ItemSpec))) ]]> </TaskItemsScript>
</PropertyGroup>
<PropertyGroup>
    <Argument1>12</Argument1>
    <Argument2>30</Argument2>
</PropertyGroup>

<Target Name="TaskItems">
    <Powershell Script="$(TaskItemsScript)" TaskItems="$(Argument1);$(Argument2)">
</Target>

It is also possible for a script to return values to the calling task. This happens through the Output taskparameter, and works like this:

<Powershell Script="$(ValueReturningScript)">
    <Output TaskParameter="Output" PropertyName="ScriptOutput"/>
</Powershell>

<Message Lines="$(ScriptOutput)"/>

The source code can be found in a Subversion repository here, and a downloadable zip including both source and the prebuilt task can be found here.

Losing faith

Thu, 07 Sep 2006 20:56:39 GMT

(Also posted as a reply to a post on our mailing list, users@ankhsvn.hm.hm)

Gah. I'm starting to lose my faith in my fellow programmers. A little background:

Ankh uses a couple of hash tables to map EnvDTE.Project and EnvDTE.ProjectItem objects (these are from the VS automation model) representing projects and (doh) their project items to our internal objects representing the same artifacts. These hash tables are used extensively when gathering SVN resources to determine whether commands should be enabled and to provide input to the execution of said commands. In other words, when a command is invoked, the hash tables are queried for our internal objects corresponding to the currently selected ProjectItem or Project.

So, if this mechanism fails, commands won't be enabled in the first place, and if they were, they wouldn't execute correctly since no resources had been harvested.

And this mechanism will obviously fail if the hash codes retrieved from the Project and ProjectItem objects are different each time they are retrieved using GetHashCode().

I'll leave the rest of this horror story to the imagination of the reader. I will not reveal the particular type of project that is committing this atrocity; suffice it to say that the common name for it starts with an R and ends with -eport Server Project.

Debugging more events

Wed, 06 Sep 2006 21:10:23 GMT

A couple of months ago, I posted a class that used the new DynamicMethod class in .NET to dynamically hook up events to all events declared on an object and display a string in VS' output pane whenever that event is raised. While quite useful, it has some drawbacks; most importantly, if you are doing this to multiple objects of the same or similar type (think multiple controls, all of which share a large number of events), it's very hard to see which particular object the event is coming from.

In some cases, you might also want to hook up only a particular event or set of events on a class, or exclude certain events. For example, all the Mouse-related events (MouseEnter, MouseMove, MouseLeave) can quickly clog up your output and hide the information you are looking for.

With this in mind, I decided to make a better version of this method. The new version is vastly improved:

It now supports a filter string which lets you specify a regex that has to match the name of the event(s) you are after
It now supports a custom format string which lets you output a lot of relevant information whenever the event is raised

The filter string can be used like this:

DebugUtils.DebugEvents(someControl, null, ".*Changed"); // only monitor the *Changed events on the control (think databinding events)

That's pretty much all there is to the filter string, if you know your regular expressions. The format string is a little more involved, though. It supports the following special keywords (yeah, I stole some of these from the VS 2005 tracepoint syntax, so sue me...):

$EVENT - Replaced with the name of the event raised (you almost always want this one).
$TYPE - Replaced with the fully qualified type name of the object raising the event.
$TIMESTAMP - Replaced with the current time.
$CALLER - Replaced with the fully qualified method name (and possibly line number if symbols are available) of the method that raised the event.
$STACKTRACE - Replaced with a full stack trace of the code that eventually raised the event in question.

You probably don't want to use the last one unless you also have some filtering on the events (or very few events on the object to begin with); it generates a lot of output. However, it can often be very useful if you need to see why a particular event is being raised. Line numbers and file names will be included if debug symbols are available.

In addition to the special keywords, it can also evaluate simple property and field expressions on the two arguments passed into the event. These are designated as "sender" and "e", respectively (as is the custom for naming event arguments in .NET). For example:

DebugUtils.DebugEvents(this.reportsListView, "$TIMESTAMP: $EVENT raised, e.InvalidRect is {e.Invalidrect}", "Invalidated");

Which prints the following output to VS' Output Pane:

22:53:32.995125: Invalidated raised, e.InvalidRect is {X=0,Y=0,Width=627,Height=269}
22:53:33.010750: Invalidated raised, e.InvalidRect is {X=0,Y=0,Width=627,Height=269}
22:53:39.932625: Invalidated raised, e.InvalidRect is {X=0,Y=0,Width=627,Height=269}

(etc...)

The source for this file can be found in the SVN repository at http://ankhsvn.com/svn/Utils/trunk/Utils/Debugging/DebugUtils.cs, and unit tests: http://ankhsvn.com/svn/Utils/trunk/UtilsTest/Debugging/DebugUtilsTest.cs. To use it for yourself, either copy the file as is or check out the entire VS 2005 project from http://ankhsvn.com/svn/Utils/trunk/ and build it. The class is made available under a slightly modified MIT license (I removed the clause requiring derivatives to carry the notice).

UPDATE: I've removed the use of the ListUtils class so that people who want to copy the file in itself won't need to also get the ListUtils class. I've also added XML doc comments to the various overloads.

AnkhSVN 1.0 RC3 released

Sat, 26 Aug 2006 11:06:32 GMT

Following the release of Subversion 1.4 RC5, we have released a new release candidate of AnkhSVN linked against the Subversion 1.4 RC5 libraries. This Subversion release candidate fixes a bug in Subversion that was triggered by our "Add Solution To Subversion Repository" command and that caused Visual Studio to crash. Other than that, the only change to Ankh itself is support for VS 2005 web projects in the same command. Previously, they would not be detected and you would get the "The following projects are not under the solution root" message even if said web projects actually were under the solution root.

Download from http://ankhsvn.tigris.org/servlets/ProjectDocumentList?folderID=7315

Oops

Wed, 16 Aug 2006 18:12:43 GMT

/me sighs.

Ok, I screwed up. Due to a braino and a particularly brain dead "feature" of our build script, the MSI wasn't built from the release branch. It was instead built from a short-lived branch I created to integrate our build process with Subversion 1.4. The released MSI is pretty close to what was supposed to have been released (which is of course why I didn't discover this ;-), but lacks some bug fixes made in the last day.

An updated MSI can be found at the same URL as the other one.

AnkhSVN 1.0 release candidate available

Tue, 15 Aug 2006 21:25:34 GMT

An AnkhSVN 1.0 release candidate is now available at http://ankhsvn.tigris.org/servlets/ProjectDocumentList?folderID=7315. This is linked to Subversion 1.4 rc4, Berkeley DB 4.4.20, OpenSSL 0.9.8a, Neon 0.25.5 and Zlib 1.2.3.

Note that any application linked to Subversion 1.4 (including the Ankh release candidate) will silently and irrevocably upgrade your working copies to a new working copy format. Use of such an application together with a pre-1.4 client will not work. The TortoiseSVN project makes a 1.4-compatible client available at http://tortoisesvn.net/downloads. 1.4-compatible versions of the svn binaries can be found with the AnkhSVN release candidate.

Also, the release candidate is linked to Berkeley DB 4.4. If you are using a local (file://) BDB repository with Ankh, it will need to be upgraded. Our general recommendation is to avoid the use of BDB for file:// access in favour of FSFS repositories.

Changes since 0.5.5:

Internal diff no longer shows duplicates
Fix a memory leak in the NSvn.Core.Pool class
Clicking the X in the commit dialog now performs a cancel, regardless of whether the most recent action was a cancel or not
NSvn exceptions are now serializable
Commit dialog now supports Ctrl-Enter for proceeding with a commit
Commit dialog text control now supports Ctrl-A to select the whole text
NSvn.Core.Client.Delete now supports the 'force' parameter... :-/
Status icons in the solution explorer are no longer overwritten by VSS ones
Configuration dialog has templates for the most common diff and merge tools
Individual project items can now be refreshed with the Ankh->Refresh command
The '?' status no longer propagates up the solution explorer treeview
New projects are now automatically SVN-added if they are versionable
Add Solution To Subversion Repository now warns the user if some projects are not under the solution root
New configuration dialog
Config file changes are picked up without having to restart VS
Fix for a memory leak in the solution explorer
Redesigned delete support
Support for the VS refresh button
Fix for a problem in which a premature solution explorer refresh after a folder add terminates the rename of the newly added folder and forces it to be named "NewFolder1".
We now call svn_utf_initialize so that iconv SOs don't have to be loaded for every SVN error message.
Workaround for Windows Workflow Foundations beta 2 bug
GDI handle leak plugged
All dialogs now invariably support standard Enter/Esc for OK/Cancel
Blame now works correctly on compound items
Use Visual Studio SDK for better VS integration
The commit dialog can now be suppressed with the Shift key, like other dialogs
Support for alternative names for the SVN administrative directory now uses the functionality offered by SVN itself
New "Working Copy Explorer" tool window
Better support for renames and moves
Redesigned VS integration to take advantage of the Visual Studio SDK
Better unicode support for log messages
Support for VS 2005 solution folders
Dialog tab order fixes
New installer using WiX instead of the VS.NET setup projects
Fix bug in which Ankh would refuse to load if VC++ wasn't installed
Prevent .svn dir from being autoadded
Support rename on folders in projects
Prevent Ankh from autoadding ignored files
Prevent the solution explorer from repainting itself while Ankh is scanning the tree, improving solution load performance
Running of SVN commands from the VS command line is now streamy
New command that lets user send a suggestion or error report without actually experiencing an error
Diff/Patch uses path relative to the solution directory
Better support for third party project types
Support commits on working copies created from multiple repositories
Commit dialog is now "pseudomodal", allowing the user to diff or view files using VS directly
Ankh no longer loads on command line builds that use /rebuild or /clean
Support for blame
Diff/patch between arbitrary revisions
Use the correct name for the Tools menu in non-english versions of VS
Support for log
Better support for file deletions
More intelligent status cache, improving performance (especially in pathological cases)
Support for VS 2005
Error reports contains the DTE version (and thus implicitly the version of VS)
Support for lock/unlock
New design for the repository explorer
Support for Enterprise Template projects
Support for database projects
Lots of tweaks, bug fixes and performance enhancements.

Ok, maybe not this weekend, then

Sun, 13 Aug 2006 20:26:51 GMT

I promised a release candidate this weekend, but it looks like that won’t happen. Integrating our build process with Subversion 1.4 and Neon 0.25 took a lot more effort than expected. We’re now probably looking at either Monday or Tuesday.

And we still haven’t quite decided on the version number…

Ankh, how do I number thee?

Thu, 10 Aug 2006 18:22:21 GMT

We’ve decided to go ahead and make a new Ankh release. A release candidate will come this weekend, and then a proper release will follow in a week if there are no major problems found. I’ve closed the last of the issues marked 0.6 in our issue tracker, so this seems like a good time to do it

One problem remains, though: what version number to give it? The obvious choice would be 0.6, since our last “official” release was 0.5.5. However, we’re getting sort of tired of the 0.6 marker by now, and we’re considering to maybe give it a version number of 0.7 or even higher. One person I talked to on IRC yesterday suggested 0.9 or even 1.0.

There are already 35 different snapshots all marked as 0.6.0.* around, and the idea is that giving it a higher version number might give a much clearer message that this is indeed a new version.

What do you think?

Is Ankh a dead project?

Tue, 08 Aug 2006 16:25:09 GMT

To avoid any potential confusion, I’ll start out this post by answering the question above: heck no!

I’m posting this because it is starting to dawn on me that there seems to be a pretty widespread perception that we are indeed a dead project. In particular, this thread on the Subversion users mailing list made this pretty clear.

And of course, the reason for this is obvious. Our tigris.org front page has the following text: “The latest stable release of AnkhSVN was 0.5.5, released on August 17, 2004.”

Ouch.

So it’s been nearly two years since our last “stable” release. Why no releases since then? It’s not like there hasn’t been active development on it since. We’ve added tons of new features, gained VS 2005 compatibility and fixed lots and lots of bugs. “Snapshot” releases have been made at various intervals (the last one was yesterday), exposing users to the new features.

As I see it, there are primarily two reasons we never made an “official” release. First, most of the users we interact directly with are those who use the snapshots. Whenever we talk to someone still using 0.5.5, the first thing we tell them is to try the latest snapshot instead. There’s been something of a feeling that having the snapshots available is “good enough”.

The second is that work on Ankh, while it has never been dormant, has always been something of a fire-and-movement exercise. Every now and then, life and work interferes with Ankh development, and by the time I get back to it, there’s always a feeling of “well, we should at least fix this bug before a release” or “this feature needs to go in before we can call it 0.6”. Halfway into fixing those bugs and implementing those features, life and/or work interferes again, and the cycle starts from the beginning.

I’m starting to think this cycle needs to end.

There is no doubt in my mind that the current snapshots are miles ahead of 0.5.5 both in features and stability. If we took yesterday’s snapshot, renamed the MSI and released it as 0.6, it would be a huge improvement over 0.5.5. And it would definitely shatter the impression that the project is abandoned.

Maybe we should do just that. What do other people think?

Why Ankh isn’t an SCC provider

Thu, 03 Aug 2006 15:12:27 GMT

A question that comes up occasionally is why AnkhSVN wasn’t implemented using the Microsoft Source Code Control Interface (MSSCCI). This is a de-facto standard today for integrating source control into various IDEs and other application, so it would seem like a natural choice for integrating Subversion into the Visual Studio IDE. However, we chose to take another approach and implement AnkhSVN as a regular Visual Studio addin instead.

Initial planning on AnkhSVN started in December of 2002. At that time, it was a school project that was supposed to last throughout the spring semester of 2003 at Oslo University College. I had some clear ideas about what I wanted to do for my project: I wanted to make something open source, I wanted to relate it somehow to Subversion and I wanted to implement it using C#/.NET, since those were my interests at the time (and still are). Using MSSCCI conflicted with goals #1 and #3. In 2002, the MSSCCI API was under a strict Non-Disclosure Agreement, meaning we wouldn’t have been able to make something open source if we were to use it. There was also a rumor going around that getting hold of the right person inside Microsoft in order to obtain the documentation for the API and sign the NDA was something of a Kafkaesque quest. Since we didn’t attempt to go down this route, I never did find out whether this was true.

Another problem with MSSCCI is that it is essentially a specification for a number of exported C functions from a DLL. At that time I had dabbled with C#/.NET for a while and wanted to try it out for a major project. A .NET assembly cannot (easily, there are ways…) export regular functions, so implementing an MSSCCI provider using .NET would prove to be rather awkward. The technical problems here aren’t insurmountable, however, I can easily think of some ways in which we could have made Ankh using MSSCCI and still got to write most parts of it in .NET. If there hadn’t been other reasons for not using MSSCCI, this problem alone wouldn’t have caused us to make the choice we did. If MSSCCI had been the best choice, it’s also quite possible that we would have implemented the whole thing in C++ instead.

MSSCCI, unfortunately, has one more major problem: it’s not a very good API. It is more or less designed around the needs of Visual Sourcesafe, and source control systems that don’t subscribe to the same ideology as VSS plain don’t fit very well into it. Eric Sink (founder of Sourcegear, which makes the Vault source control system) , in his excellent series Source Control HOWTO, writes about MSSCCI’s deficiencies in Chapter 9: Source Control Integration with IDEs. Which is good, because then I don’t have to go on about it here.

We already knew about most of the criticisms leveraged at MSSCCI back when we were planning the project, so it proved to be the final and decisive argument against using it for Ankh.

DynamicMethod revisited: Making Powershell handle arbitrary delegates

Wed, 26 Jul 2006 17:48:44 GMT

As I posted on Monday, Powershell doesn’t let you cast a scriptblock to a delegate with a signature other than void (Object, Object). Jeffrey Snover confirmed this the same day. When I learned that this was a limitation in v1.0 of Powershell, I set out to find a way to get around this problem using .NET’s DynamicMethod class.

I mostly got it done yesterday, but I was feeling a bit tired and didn’t finish it so I could blog about it. Today I see that the Powershell team has posted a workaround of their own, which is remarkably similar to mine. Although, theirs probably gets extra coolness points since it’s written in Powershell itself and not implemented as a cmdlet like mine.

Anyway, I have now finished it and decided to post it anyway (I spent quite a lot of time on it, mostly silly mistakes on my part). Here goes:

using System;
using System.Collections.Generic;
using System.Text;
using System.Management.Automation;
using System.Reflection.Emit;
using System.Reflection;
using Fines.Utils.Collections;

namespace Fines.ConvertScriptBlockSnapin
{
    [Cmdlet("Convert", "ScriptBlock")]
    public class ConvertScriptBlockCommand : Cmdlet
    {
        [Parameter(Mandatory=true, Position=0)]
        public string DelegateTypeName
        {
            get { return this.delegateTypeName; }
            set { this.delegateTypeName = value; }
        }

        [Parameter(Mandatory=true, Position=1)]
        public ScriptBlock ScriptBlock
        {
            get { return this.scriptBlock; }
            set { this.scriptBlock = value; }
        }

        [Parameter(Mandatory=false, Position=2)]
        public SwitchParameter ReturnPSObject
        {
            get { return returnPSObject; }
            set { returnPSObject = value; }
        }

        protected override void ProcessRecord()
        {
            // Validate the DelegateTypeName parameter
            // It must be a delegate type
            Type delegateType;
            if ( !LanguagePrimitives.TryConvertTo<Type>( this.DelegateTypeName, out delegateType ) )
            {
                Exception ex = new ArgumentException( "Type not found: " + this.DelegateTypeName, "DelegateTypeName");
                base.ThrowTerminatingError( new ErrorRecord( ex, "TypeNotFound", ErrorCategory.InvalidType, null ) );
            }

            if ( ! typeof(Delegate).IsAssignableFrom(delegateType))
            {
                Exception ex = new ArgumentException( "DelegateTypeName must be of a delegate type. Found " + this.DelegateTypeName,
                    "DelegateTypeName" );
                base.ThrowTerminatingError( new ErrorRecord( ex, "TypeNotDelegate", ErrorCategory.InvalidType, null ) );
            }

            if ( typeof( Delegate ) == delegateType || typeof(MulticastDelegate) == delegateType )
            {
                Exception ex = new ArgumentException( "DelegateTypeName cannot be of type System.Delegate or System.MulticastDelegate",
                    "DelegateTypeName" );
                base.ThrowTerminatingError( new ErrorRecord( ex, "TypeSystemDelegate", ErrorCategory.InvalidType, null ) );
            }

            // Create our trampoline and write it to the pipeline
            ScriptBlockTrampoline tramp = new ScriptBlockTrampoline( this.ScriptBlock, this.ReturnPSObject );
            this.WriteObject( tramp.CreateDelegate( delegateType ) );
        }

        /// <summary>
        /// This class exists to attach state (the scriptblock and the switch parameter) to the dynamic method
        /// we create.
        /// </summary>
        private class ScriptBlockTrampoline
        {
            public ScriptBlockTrampoline( ScriptBlock scriptBlock, SwitchParameter returnPSObject )
            {
                this.scriptBlock = scriptBlock;
                this.returnPSObject = returnPSObject;
            }

            public Delegate CreateDelegate( Type delegateType )
            {
                MethodInfo invokeMethod = delegateType.GetMethod("Invoke");

                // get the parameter types of the delegate
                Type[] baseMethodParmTypes = ListUtils.ToArray(
                    ListUtils.Map<ParameterInfo, Type>( invokeMethod.GetParameters(), delegate( ParameterInfo par )
                    {
                        return par.ParameterType;
                    } )
                );

                // the first arg needs to be of type ScriptBlockTrampoline so we can treat it as an instance method
                Type[] parmTypes = new Type[ baseMethodParmTypes.Length + 1 ];
                parmTypes[ 0 ] = typeof( ScriptBlockTrampoline );
                Array.Copy( baseMethodParmTypes, 0, parmTypes, 1, baseMethodParmTypes.Length );

                // Associate our dynamic method with the ScriptBlockTrampoline class so it can access its privates
                DynamicMethod dm = new DynamicMethod( "Dele", invokeMethod.ReturnType, parmTypes, typeof( ScriptBlockTrampoline ) );
                ILGenerator g = dm.GetILGenerator();

                // Create a local to hold our params array
                LocalBuilder arrayLocal = g.DeclareLocal(typeof(object[]));

                // create the array with our desired length
                g.Emit( OpCodes.Ldc_I4, baseMethodParmTypes.Length );
                g.Emit(OpCodes.Newarr, typeof(object));

                // store it in our local
                g.Emit(OpCodes.Stloc, arrayLocal);

                // load our params into it
                for (int i = 0; i < baseMethodParmTypes.Length; i++)
                {
                    g.Emit( OpCodes.Ldloc, arrayLocal );

                    // the index in the array in which we want to store the element
                    g.Emit( OpCodes.Ldc_I4, i );

                    // load the arg onto the stack
                    g.Emit( OpCodes.Ldarg, i + 1 ); // 'this' is arg 0

                    // if it's a value type, we need to box
                    if ( baseMethodParmTypes[ i ].IsValueType )
                    {
                        g.Emit( OpCodes.Box, baseMethodParmTypes[ i ] );
                    }

                    // and store it in the array
                    g.Emit( OpCodes.Stelem, typeof( object ) );
                }

                 //get the scriptblock field
                g.Emit( OpCodes.Ldarg, 0 );
                g.Emit( OpCodes.Ldfld, typeof( ScriptBlockTrampoline ).GetField( "scriptBlock",
                    BindingFlags.NonPublic | BindingFlags.Instance ) );

                //// now emit the call to the scriptblock
                g.Emit( OpCodes.Ldloc, arrayLocal );
                g.EmitCall( OpCodes.Callvirt, typeof( ScriptBlock ).GetMethod( "InvokeReturnAsIs" ), null );

                // can't have an object on the stack if we're returning void
                if ( invokeMethod.ReturnType == typeof( void ) )
                {
                    g.Emit( OpCodes.Pop );
                }
                else
                {
                    // does the user want the raw object or the PSObject instance?
                    if ( !this.returnPSObject.IsPresent )
                    {
                        // what's returned from the ScriptBlock is a PSObject, get the underlying object instead
                        MethodInfo getMethod = typeof( PSObject ).GetProperty( "BaseObject" ).GetGetMethod();
                        g.EmitCall( OpCodes.Call, getMethod, null );
                    }
                }

                // ret is not optional
                g.Emit( OpCodes.Ret );

                return dm.CreateDelegate( delegateType, this );
            }

            private ScriptBlock scriptBlock;
            private SwitchParameter returnPSObject;
        }

        private ScriptBlock scriptBlock;
        private string delegateTypeName;
        private SwitchParameter returnPSObject;

    }

}

The ScriptBlockTrampoline.CreateDelegate method is where most of the fun stuff happens. I create a DynamicMethod with the same signature as that of the target delegate. Since I also pass in the ScriptBlockTrampoline type to the DynamicMethod constructor, the created method will be treated as a member method on ScriptBlockTrampoline with regards to member access; in other words, it can access private members on this type. The first parameter to the DynamicMethod is a ScriptBlockTrampoline object, which means it can be treated as an instance method and have access to the scriptBlock and returnPSObject private member variables.

Code is then emitted to create an array of type Object[], which is then loaded with the arguments passed to the delegate at the invocation point. This array is passed to the ScriptBlock’s InvokeReturnAsIs() method, which actually executes the scriptblock.

One particular thing that caused me a lot of grief (not really sure why, I’ve encountered and solved the same problem before) was that InvokeReturnAsIs() will return an object and place it on the IL evaluation stack even if there is no actual return value and the delegate return type is void. If you don’t pop this object off the stack (the if statement) before the RET instruction, executing the delegate will cause the .NET runtime to bail out and die since the program is invalid. I spent a lot of hours on this one.

One other thing that bears mentioning is that InvokeReturnAsIs will actually return a PSObject, not the raw underlying object. If the code calling the method expects the actual type (i.e., if it’s called from most managed languages), things will get ugly pretty fast if it gets a PSObject instead. Code is therefore emitted to call the PSObject.BaseMethod property.

With this cmdlet, implementing a Powershell script for doing an “svn ls” with support for authentication becomes rather simple:

# get-svndir.ps1

param ([string]$svnurl       = $(read-host "Please specify the SVN url for the directory you want to list")
    )

[System.Reflection.Assembly]::LoadFrom((join-Path "$env:PROGRAMFILES\AnkhSVN" -childPath NSvn.Common.dll)) | Out-Null
[System.Reflection.Assembly]::LoadFrom((join-Path "$env:PROGRAMFILES\AnkhSVN" -childPath NSvn.Core.dll)) | Out-Null

$PRIVATE:svnclient = new-object NSvn.Core.Client 

$PRIVATE:simplepromptdelegate = convert-scriptblock NSvn.Core.SimplePromptDelegate {
    Write-Host "Authenticating against realm " $args[0]
    $user = Read-Host ("Input username (press Enter to accept default of {0})"  -f $args[1])
    if ( $user.Trim() -eq "" ) 
    {
        $user = $args[1]
    }
    
    $password = Read-Host -AsSecureString "Input password"
    
    # Convert it to a plain string
    $BSTR = [System.Runtime.InteropServices.marshal]::SecureStringToBSTR($password)
    $password = [System.Runtime.InteropServices.marshal]::PtrToStringAuto($BSTR)
    [System.Runtime.InteropServices.Marshal]::ZeroFreeBSTR($BSTR)
    
    $cred = New-Object NSvn.Core.SimpleCredential($user, $password, $False)
    $cred
}

$PRIVATE:svnclient.AuthBaton.Add( [NSvn.Core.AuthenticationProvider]::GetWindowsSimpleProvider() )
$PRIVATE:svnclient.AuthBaton.Add( [NSvn.Core.AuthenticationProvider]::GetSimplePromptProvider($PRIVATE:simplepromptdelegate, 3) )
  

$items = $PRIVATE:svnclient.List($svnurl, [NSvn.Core.Revision]::Head, [NSvn.Core.Revision]::Head, [NSvn.Common.Recurse]::None)
$items

Thanks to /\/\o\/\/ for pointing me to his blog post about how to convert a SecureString to a regular .NET string, which is what the SimpleCredential constructor expects.

A sample run of that script:

PS D:\mydocs\powershell> D:\mydocs\powershell\get-svndir.ps1 http://ankhsvn.com/svn/draco/ | ft Path,LastAuthor,Time
Authenticating against realm <http://ankhsvn.com:80> Subversion repository draco
Input username (press Enter to accept default of Arild):
Input password: ****
Path LastAuthor Time
---- ---------- ----
trunk Arild 16.01.2006 03:33:30
branches Arild 18.10.2005 21:03:32
tags Arild 18.10.2005 21:03:32

The snapin can be found in SVN at http://ankhsvn.com/svn/powershell/trunk/ConvertScriptBlockCommand/. It is also downloadable as a .zip here. To install it, run install.ps1 from the root folder in a shell with the .NET environment set. By default, it will add the command “Add-PsSnapin ConvertScriptBlockSnapin” to your $PROFILE file after compiling and installing it if it’s not already there.