Monday, July 16, 2007

Validation Application Block - The very start of a new journey

Although LINQ made a huge impression on me at the Tampa Code Camp I do not have Visual Studio 2008 yet so today I worked on the other groundbreaking concept I saw at Code Camp, the Enterprise Library Validation Application Block (VAB). The concept of this project is to provide a consistent way to validate data and relocate all of the validation off of Windows or ASPX screens and place it in the business logic layer. This is accomplished through declarative programming and the addition of validation attributes on properties within the business objects. The implementation of this technique is codeless for all of the validations provided within the VAB because all of the validation is performed by the VAB framework. Of course the VAB is extensible so you can write your own validations as well. Additionally, the inclusion of "rule sets" means that data can be validated using multiple rule sets. Thus, for a company like mine where we write 1 product for many customers we can have a core validation set and each of our customers can add their own custom validators. The amazing part is that validators can be written within code (obviously by developers) but also in config files (potentially by our clients). Obviously, a lot of smart people put this application block together.

Check out this quick video by C# MVP David Hayden and hopefully you will be as excited about the promise of this technology as I am.

Sunday, July 15, 2007

Tampa Code Camp

I went to the Tampa Code Camp this Saturday and it was terrific! If you have not experienced a code camp before I would urge you to find out about one in your area. Code camps are free and the speakers are all volunteers. There are sponsors of course which means you may even get food and door prizes but best of all you will get knowledge. The 2 highlights for me were seeing 2 in depth presentations on LINQ and I saw more information about some of the options available to me from the Enterprise Library. Within the Enterprise Library, I am most excited about the validation block but I imagine I will end up using the configuration, exception and caching blocks. Data access and security sound tempting too.

LINQ just blew me away. This is truly the most amazing innovation to hit programming that I have ever seen. Here are a few links to LINQ:
Scott Guthries Blog
LINQ project homepage - Jim Wooley I and Jim Wooley II

Thanks so much to Jim Wooley (who is writing a book on LINQ) for putting on a great demo that opened my eyes to the possibilities of LINQ. Jonathan Carter did a great job to show many excellent examples of how to use LINQ to query, update and delete data.

If you do not know how to find a code camp try to find a user group near you (check and the user group can probably help you find a code camp. A nearly complete list of code camps is also located here.

With these topics I believe I will have many new adventures!

Debug.Assert + Custom Exceptions = NICE!

Thanks to chapter 3 of Debugging Microsoft .NET 2.0 Applications by John Robbins I have been working to implement debug.assert statements throughout my code. If you are new to debug.assert (as I was) here's a short summary of the benefit they provide.

A debug.assert statement typically looks like this:

Public Sub LoadData(Byval employeeId as integer)
  debug.assert(employeeId > 0)
End Sub

First of all, a debug.assert statement will only be executed in debug mode. So when you compile your project in release mode these lines will be omitted by the compiler. You can verify this setting by going to Project Properties --> Compile tab --> Advanced Compiler Options button and look at the "Define DEBUG constant" setting. You should find that this checkbox is checked for a debug builds and unchecked for release builds. For an ASP.Net project this setting is maintained in web.config in the <compilation> section definition. Just search web.config for the word "debug" and you will find the setting.

Assertion pop up a message box that allows you to go into break mode at the point in code where the debug.assert statement failed. Using the example above, this can be very helpful because if the calling method passed in 0 or a negative number for the employeeId this method is not going to truly succeed and you may get some failure way down the line in your code as some screen or method tries to use the results of the LoadData method. Those routines may not have a direct call stack back to the routine that passed the 0 to the LoadData function and this can slow your debugging. With assertions properly placed and each argument tested for validity you will break into break mode in Visual Studio as soon as something unexpected occurs and not further down the line where you may end up with a blank screen (becaues no data was found) or an unhandled exception. In ASP.Net an unhandled exception can be a real time killer because you may get redirected to an error page, you may have to stop the project, start the project, retrace all of your steps (and you may not remember them!), etc. So, breaking before an error happens can be a huge time saver.

One important thing to note about assertions is that they will dump you into break mode only if they fail. So, when you are writing an assertion you need to write the assertion in such a way as to say, "The valid values for this variable are...". For example, debug.assert(employeeId > 0) is saying, "The valid values for the employeeId are only numbers greater than zero." If the value does not meet that criteria the assertion fails and you get the opportunity to go into break mode.

The last important point to reiterate is that although assertions help you in debug mode they do not replace exceptions. Because of this, I found myself writing code that looked like the following. I had to write 1 line for a debug statement and 1 line for an exception. Then I had to do it for every parameter for every method in every class of every assembly for the rest of my life. This was quite repetitious and not very adventurous!

Public Sub LoadData(Byval employeeId as integer)
  If employeeId <= 0 then

    dim errMsg as string = "employeeId <= 0" 'I used .fail here because I want it to fail everytime since I already know my argument is out of range.
    throw new ArgumentOutOfRangeException(errMsg)
  end if
End Sub

Having the debug statement was nice because in debug mode I would hit the debug statement before the exception was thrown so Visual Studio would give me a chance to look over my callstack and find the flaw without having to suffer through hitting the exception line. However, I found this to be very verbose. Especially for methods that received several arguments. Nevertheless, I continued to implement it because almost immediately after adding debug statements in a few methods I was already experiencing the benefits.

Here is what qualifies this as an adventure.
Despite having written a macro to generate this pattern for me, I decided I needed a more succinct method that would ease the maintenance on all of this code that I was auto generating.  Here is my final implementation:

Public Sub LoadData(Byval employeeId as integer)  
  If employeeId <= 0 then
    throw new ArgException("employeeId <= 0")
  end if

End Sub

Now, you must be wondering, "This guy just said he loved asserts but he just deleted the assertion! What is he thinking?" Here's how I did it.

1) I created a base exception class called ExceptionBase that inherits from System.ApplicationException.

Public Class ExceptionBase
Inherits System.ApplicationException

  Public Sub New(ByVal message As String), Nothing)
  End Sub

  Public Sub New(ByVal message As String, ByVal InnerException As System.Exception) & DebugUtils.Assert(message), InnerException)
  End Sub
End Class

2) I created other custom exception classes that inherit from ExceptionBase. For example, I created a ArgsException class and a DataEOFException class. Each class has different constructors based on their special needs. Here are abbreviated samples:

Public NotInheritable Class ArgException
Inherits ExceptionBase
Private Const ERROR_MSG As String = "Invalid argument value: "

  Public Sub New(ByVal message As String)
    MyBase.New(ERROR_MSG & message)
  End Sub
End Class

Public NotInheritable Class DataEOFException
Inherits ExceptionBase
Private Const ERROR_MSG As String = "Unexpected EOF: "

  Public Sub New(ByVal cmd As DbCommand)
    MyBase.New(ERROR_MSG & DBUtils.GetCommandInfo(cmd))
  End Sub

  Public Sub New(ByVal sql As String)
    MyBase.New(ERROR_MSG & sql)
  End Sub
End Class

3) Here's the tricky part in case you missed it. If you look back at the BaseException class in the Sub New constuctor that actually calls the System.ApplicationException base class you will see this line of code: & DebugUtils.Assert(message), InnerException)

Because a constructor calling its base constructor must do so on the first line of code I cannot write the code like this:

    DebugUtils.Assert(message), InnerException)

So I had to add the call to my custom Assert method right inside of the call to the base constructor. Now, all that was left to do was write the simple Assert method in my DebugUtils module and I was all set.

Friend Module DebugUtils
  Friend Function Assert(ByVal message As String) As String
    TracingUtils.TraceData(TraceEventType.Error, 0, message)
    Return String.Empty 'always nothing
  End Function
End Module

Note that I had to make the Assert method a function in order to call it inline with the call to the base constructor so I simply made Assert return an empty string.

Also note that I implemented a call to my TracingUtils class inside of my Assert method so that if an error happens in a release build and the debug.assert statement is ignored at least I will get something written to my trace logs.

Also note that I declared my DebugUtils class with a "Friend" scope. This was because I did not want to make this method public and accessible to code outside of this assembly. The reason for that is because if the 2 assemblies were compiled in different modes (1 in debug and 1 in release) these debug statements would not behave as expected. So, if a different assembly wants to use any of the functionality within the DebugUtils module that assembly should contain its own copy (using a "Share") of the DebugUtils module.

In summary, I have decreased the lines of code that I have to write for each argument that I test within each method but I have not lost my beloved debug assertions.

Another nice thing to note is that I was able to use the regular expressions feature within VS 2005's Find & Replace screen to do a multiline find & replace. This allowed me to strip out all of the "" from all my methods and take out the newline after them (using \n for newline) so that I was not left with a bunch of blank lines in my source code. So, if you haven't been working on your regular expressions skills there is just one more reason to improve.

Thanks for joining me on this adventure.

Wednesday, July 11, 2007

How to move a Typed Dataset to its own Project and Namespace - Part 2 - Deployment and ConnectionStrings

Just in case you missed part 1 please read that adventure before proceeding.

I thought my adventure was over with these ridiculous typed datasets but today I got a rude surprise. It reminded me of a baseball game that I once played. I was playing shortstop and the go ahead run was at 3rd base so we brought our infielders in on the grass. This guy hit a rocket right at me and the ball bounced up and hit me right in the mouth. That's about how fun this adventure was.

So, what was today's baseball in the choppers you ask? Well, it appears that the geniuses at Microsoft (and there are many I am sure) happened to overlook the fact that the database connectionstring information that is used when you are designing a typed dataset just might not be the connectionstring that you will want to use when you deploy your application. I know in a perfect world (for hackers at least) every database would have the same name, user name and password but I live in an imperfect world and typed datasets are just another shining example.

First of all, thank heavens for a posting on Channel 9 or else I would have never gotten the ideas that were required to workaround this huge oversight by Microsoft.

Basically, the situation is that typed datasets hard code connection string information in them. It can either be burned into the code via app.config or via the settings.settings file underneath the "My Project" folder in the Solution Explorer (you may need to do a "Show All Files" to see this file).

App.Config is great in general but if you have many projects within a solution and each of those project have an app.config each of those files contains their own connectionstring then maintaining & synchronizing the same database connectionstring information in all of the different .config files can really be a nuisance. There appears to be no easy way to convince typed dataset projects to read from the main EXE's app.config or web.config. Keep in mind that if you decide to go with this hard to maintain "many .config file" approach the .config file will not be created on "Debug" builds of your solution. You will need to go to Project Properties Compile Tab and select "Release" from the "Configuration" combobox at the top of the screen. This approach is not bad if you have a small project and a single database but it probably is not an acceptable solution for an enterprise software product. Then again, I am starting to believe that typed datasets are not acceptable for enterprise software products.

The other approach is to place your database connectionstring in the settings.settings file. This was ultimately my preferred approach although at first glance it looked like it was an even worse idea than app.config. Here's why settings seemed like a dead end. The setting that Visual Studio maintains for a connectionstring is generated as readonly. If you try to manually change this by editing the contents of Settings.Designer.vb Visual Studio will reset the property to readonly. There seemed to be no way to fool Visual Studio.

Here's how I solved the problem:
1) I added a new Module to my project and named it DBConnectionString.vb.

Option Strict On
Option Explicit On
'''<summary>Shared module that allows for setting the database connection at runtime.
''' <history>Created by kmccaa on 07.11.07</history>

Public Module DBConnectionString
Private m_RunTimeConnectionString As String = String.Empty
''' <summary>Sets the connection string for all of the XSDs in the project at runtime.
''' Because this is a shared class (module) setting this value once when the application
''' starts (e.g. in global.asax Application_OnStart for web projects) and then all the
''' XSDs will use this runtime connectionstring rather than the connectionstring that was
''' burned into the source files at design time.
''' </summary>
''' <history>Created by kmccaa on 07.11.07</history>

Public Property RunTimeConnectionString() As String
    Return m_RunTimeConnectionString
  End Get
  Set(ByVal value As String)
    m_RunTimeConnectionString = value
  End Set
End Property
End Module

2) If you do not have a connectionstring property setup in your DAL project yet do the following. Go to Project Properties Settings tab.
Add a new setting.
Name it "ConnectionString"
Choose ConnectionString as the "Type"
Choose Application as the "Scope"
Type in a valid database connectionstring for the "Value".
Save your settings and close the property screen.

3) I manually edited the contents of Settings.Designer.vb. These changes will get overwritten any time you change the settings.settings in your project but unlike when I tried to change the property to a read/write property I was able to fool Visual Studio so that it would not change the internals of the property at compile time.

Public ReadOnly Property ConnectionString() As String
    If DBConnectionString.RunTimeConnectionString.Length > 0 Then
      Return DBConnectionString.RunTimeConnectionString
      Return CType(Me("ConnectionString"), String)
    End If
  End Get
End Property

Keep in mind that if you ever add/edit/delete any settings from with Visual Studio the ConnectionString property will probably get overwritten and reverted back to the following.

Public ReadOnly Property ConnectionString() As String
    Return CType(Me("ConnectionString"), String)
  End Get
End Property

For that reason, you should save the snippet above somewhere handy. I would recommend as a code snippet in your toolbox. If you have never added a code snippet to your toolbox before just select the text you like and drag it to your toolbox. Then right click on it and choose "Rename Item". By having it in your toolbox you can restore it to the "custom" way quickly by dragging it out of the toolbox and into the code window.

4) Then I had to make sure that none of my XSDs were reading their connectionstring from app.config. So I had to replace this:

Me._connection.ConnectionString = System.Configuration.ConfigurationManager.ConnectionStrings("ConnectionString").ConnectionString

With this (insert your project name where it says <ProjectName>):

Me._connection.ConnectionString = Global.<ProjectName>.My.MySettings.Default.ConnectionString

For example, if your project name was ABCInc.Employees.DAL then you would use this:

Me._connection.ConnectionString = Global.ABCInc.Employees.DAL.My.MySettings.Default.ConnectionString

5) Delete any connectionstring information that you may have in app.config for your DAL project. Perhaps you can delete the whole app.config file. This just helps to make sure you don't have a stray reference to app.config in your code.

6) Compile your DAL project.

7) Within your main EXE of website you now have to set the DBConnectionString.RunTimeConnectionString value in your DAL project. My work was for a web project so I opened global.asax and added this (of course you have to put your own value in for the <ProjectName>:

Sub Application_Start(ByVal sender As Object, ByVal e As EventArgs)
  <ProjectName>.DBConnectionString.RunTimeConnectionString = ConfigurationManager.ConnectionStrings("ConnectionString").ToString
End Sub

By the way for those that are not used to using modules instead of classes here is the advantage. Because I created the DBConnectionString.vb as a module rather than a class that allows me to set the RunTimeConnectionString once at application startup and not have to worry about setting it each time I try to hit the database. This can have a downside too though if I wanted each user to connect using a different connectionstring then by having a shared property that would mean that all users within the website would all be sharing that single value. In Oracle there is a workaround for this called "Proxy Users" but that is a topic for another day.

8) Set your main web.config or app.config connectionstring to something other than the connectionstring that you used while developing your XSD.

9) Run your project to verify that the queries are indeed running against this alternate database.

10) Send me a Thank You note if it works. Send me a question if it doesn't.

Thanks for sharing this adventure with me and keep your eyes (and not your mouth) on the ball.

Sunday, July 8, 2007

Using Regular Expressions in Find & Replace within Visual Studio

This is not much of an adventure but more of a quick tip. As you may know from my other adventures I have realized that one of the key features in .Net to utilize is regular expressions. So, to help me develop in this area I am trying to use them appropriately wherever possible. One place that you may not be aware of is in the "Find & Replace" screen in Visual Studio.

Here is the tip. Suppose you have a whole line you want to delete from many locations. For example:
'Deletes a record

You could do a find and replace on 'Deletes a record but then you'd be left with an extra blank line in all of those locations. Extra blank lines can drive you crazy when you are trying to maximize screen real estate. So, instead of a simple replace of 'Deletes a record do this:
Replace this: \n+\s*\'deletes a record\n

Then make sure to select "Regular Expressions" in the "Use:" selection of the "Find & Replace" screen.

Here's a quick explanation of the syntax of \n+\s*\'deletes a record\n
\n stands for "New Line"\
+ stands for "1 or more matches" (of new line in this case)
\s stands for a space (or tab or other whitespace characters)
* stands for "zero or more matches" (of blank spaces in this case)
I had to "escape" the apostrophe with a \ because ' has special meaning in the regular expressions engine in VS so by typing \' I am telling the search engine that I really want an apostrophe.
Then of course my text to search for (deletes a record)
\n for the trailing new line character that I want deleted.

A simpler choice could have been
\'deletes a record\n
But without placing the requirement of at least 1 new line before and optionally some spaces there is a chance that I would match something like this:
Some Code 'deletes a record
return true

And turn it into this:
Some Code return true

Which would not compile.

So, using regular expressions in the "Find & Replace" window can be tricky but without daily (or at least weekly) practice on regular expressions I probably won't see real improvement. Because I do not always have a need for writing regular expressions using them within the "Find & Replace" window will allow me to stay sharp.

Thanks for sharing this adventure with me.

Saturday, July 7, 2007

error MSB4018: The "Vbc" task failed unexpectedly.

Today I got this error after moving about a dozen .vb files out of an App_Code directory of an ASP.Net project and into a separate class library project.

error MSB4018: The "Vbc" task failed unexpectedly.

I knew that when I moved all of that code over to this different location the project would not compile so I wasn't surprised to get a few error messages. Well, the nasty thing about this error is that you lose all intellisense and basically Visual Studio becomes Notepad.

I know an aspiring storyteller should always make the sorry really exciting and maybe even include a few car crashes but I think I'll cut to the chase on this one.

After removing all of the .vb files from my project and still not being able to compile I forwarded my problem to the very helpful MSBuild team at Microsoft. It turned out that the problem was that I had manually overridden the "Base Address" setting (Project Properties Compile tab Advanced Compile Options button) with a hex number that was too large for the compiler. So, if you dare to change the base address setting they recommend that set it to a relatively small number. My number was &H98512474.

Here is there full recommendation:

In general, you want to keep base addresses lower than 2gig; although each process that you create runs in a virtual process space that on 32 bits is 4gig, you may be causing a fair amount of memory page swapping with such a large address space. What you are trying to avoid by setting the base address is loading into a space someone else is loading into. Collisions occur, and it just slows down the load time of your program. A small amount, but some. By default, VS builds programs at 0x10000000. Now since Microsoft knows that, we move all our dll's to higher ranges. For example, most MSFT components load in the 0x60000000 to 0x80000000 range (Use the tool below and open System.dll from the .Net Framework to see an example).

A good tool to use to help you set an appropriate address is dependency walker. This used to ship as part of Visual Studio but is now open source.

Build your executable, and open it with dependency walker. This will show every dll you are dependent on. Look at the column 'Preferred Base' and 'Virtual Size'. Set your dll to a base that is not taken, and add your virtual size to ensure that you are not tramping over someone else's preferred base.

A good rule of thumb is to use the space between 0x10000000 and 0x40000000. The only time you really need to worry about this is when you're are building a program with several libraries. Since your libraries will all want to load into the same space, they will clash and collide.

Let's just say this was not one of my most joyous adventures but it sure was exciting when I couldn't get a project with no .vb files to compile.

Thanks for joining me on this adventure.

Friday, July 6, 2007

How to move a Typed Dataset to its own Project and Namespace

This is "Part 1" of a 2 part adventure. Please do not alter any of your code before reading part 1 & part 2.

In today’s adventure I dared to travel into an area that I have feared greatly for many months. For the betterment of our software I decided today was the day that I finally broke all of the typed datasets out of our web project’s App_Code directory and into their own project. After all, nobody wants to read boring “adventures!”

First of all, I have seen many people asking, “How do you add a namespace to a typed dataset?” The answer is “Don’t bother.” It can be done quite simply by just renaming the .xsd file but like I said, “Don’t bother.” Here is why, having all these XSDs embedded in their web projects makes them completely useless to any other application (unless you create web services to expose them but web services can add an extra layer to an application that most applications do not need). So, all that namespacing work is just going to need to be removed when you finally realize that these files should have been in a totally separate project from the beginning. Once the XSDs have been moved to a separate project then they will fall within the root namespace of the project. If you need to add further namespacing in addition to the project root namespace you can accomplish that by renaming the .xsd file.

So, rather than asking, “How do you add a namespace to an XSD” you should be asking “How do I get all of these XSDs out of my ASP.Net project?”

Here is how I did it.

1) Make a copy of all of your XSD files to a separate directory outside of your web project. Let’s say c:\Temp\XSDs.

2) Make all of the .xsd files editable by removing their Read-Only property.

3) Create a new class library project wherever it is appropriate, e.g. c:\CompanyName\SolutionName\SolutionName.DAL\

4) Add a reference to System.Configuration in your project.

5) Create an app.config in your new project. Based on my experience in part 2 this is not a necessary step in the long run but it may be needed to get your project to initially compile. Make sure to read part 2 to see how to make the DAL project run against a runtime database that may differ from your development database.

6) Make sure to set the root namespace of your project (Project Properties Application tab). At my company, we name our projects “SolutionName.DAL” but the root namespace for our projects is “CompanyName.SolutionName.DAL”. You want to get this set correctly before continuing.

7) Create a new XSD in your new project and create some simple query this will accomplish getting a valid connectionString into app.config. This connectionString ends up being a little different than what you may have expected so I think it is better to let the XSD wizards create it for you. Here is approximately what mine looked like. As you can see by the providerName I use Oracle so your connectionString may differ quite a bit. Therefore, I recommend letting the wizard do the work for you. Note this connectionstring is only used by the designer at design time. At runtime it will use the connectionstring named “ConnectionString” from the app.config of the main executable (or ASP.Net site).

<add name="CompanyName.ProjectName.DAL.My.MySettings.ConnectionString" connectionString="Data Source=Something;Persist Security Info=True;User ID=SomeUser;Password=SomePassword; " providerName="System.Data.OracleClient" />

8) Download Notepad++ if you have not already done so. This is a great free text editor.

9) Because Notepad++ is an MDI application it will allow you to open all of the .xsd files at once. So, select all the .xsd files, right click and choose “Edit with Notepad++.”

10) Do a find and replace (ctrl+h) using the following:
a. Find this: web.config
b. Replace it with this: MySettings
c. Hit the “Replace all in all opened documents” button. That’s nice!

11) Save all your files.

12) Next you will need to replace the following section from each .xsd file.
a. Find this:
b. Replace it with the following. Make sure to change where it says “CompanyName.ProjectName.DAL” to the root namespace of your project. This value should also match the name of the connectionString in your app.config file:

c. Hit the “Replace all in all opened documents” button.

13) Save the file.

14) Copy the file (and its child files with the same name and different extensions, e.g. any .vb, .xss files) to your new DAL project. You can simply choose “copy” in Windows Explorer and select the project in Visual Studio’s Solution Explore and choose “paste” and the file will be added to the project and placed in the project’s working directory.

15) Cross your fingers. Stand back from your computer. Try to build your DAL project. If it builds you might be in the clear!

16) Before you get too excited and edit and import all the other XSD files (you adventurer!) take a moment to create another project, perhaps a unit test project, within your solution. Within this other project add a reference to your DAL project.

17) See if you can write some functioning code against the datasets exposed by your DAL project. This will give you a chance to make sure your DAL project is truly working and that you have the namespaces right. Remember, getting the correct namespaces is the whole reason you wanted to do this in the first place! Believe it or not your DAL project can compile but not expose any of the datasets to outside projects if you have an error in your app.config within the DAL project.

18) If you do not like your namespaces change them by changing the project root namespace (Project Properties Application tab) and/or change the file name of the .xsd file. Continue tweaking the namespace and rebuilding the project and testing the project from your test project. Keep in mind that if you make changes now to the root namespace, you may have to synchronize these changes with the settings in the App.Config and the XSD files.

19) Check everything into your source control software before you break something in the next step. It is only good to be adventurous if you live to tell about your adventure.

20) Once you are comfortable with your namespace strategy go back and edit all of the remaining .xsd files in Notepad++ and copy all of the remaining files into your DAL project. Although I love adventure as much as the next person I would recommend that you do this 1 file at a time just to make sure if something breaks it is easy to identify what file won’t compile. After all, you are manually editing the complex .xsd files in a text editor. Something is bound to go wrong sooner or later. Be scientific and limit your variables by being a little patient and importing one file at a time. Don’t forget to write a unit test for each XSD (or at least doing a “Preview Data” within each XSD) too. You really never know if these things are working until you have a working test. Be careful. If you make a mistake you may end up getting this message:
Unable to find connection 'ConnectionString (MySettings)1' for object 'MySettings'. The connection string could not be found in application settings, or the data provider associated with the connection string could not be loaded.

Which will probably lead you here

I hope this has helped someone. I know I will sleep better at night knowing that all of these datasets are now available for use by other projects and the business logic layer that I also had to painstakingly pull out of App_Code today.

Thanks for joining me on this adventure. I hope you did not get too many bumps and bruises.

Monday, July 2, 2007

Adding text to the Clipboard from a Macro - DTE.ExecuteCommand("Edit.Cut")

On 2 occasions now my adventures have required that I write a macro that placed some text into the clipboard. The first adventure involved having the user type "check-in comments" into an inputbox, writing those comments into the code and then placing the comments into the clipboard so they could use them as check-in comments in the check-in window (which I launched as part of the macro). That first adventure proved treacherous because no matter how hard I pounded the same nail with the same hammer it didn't want to go in.

Whenever my macro code tried to execute this line:

Clipboard.SetText(originalComment, TextDataFormat.Text)

I would get this error:
"Current thread must be set to single thread apartment (STA) mode before OLE calls can be made. Ensure that your Main function has STAThreadAttribute marked on it."

Finally, I came across the workaround of placing the text into the code editor in VS and then executing this line of code:


This worked wonders because I had tricked Visual Studio into thinking the user had clicked Cut in the Edit menu.

Today brought a new challenge. I needed to insert many lines into the user's clipboard. In my single line situation I had used the following to select my single line.


It turns out the .LineUp() method supports 2 optional parameters. The first (extend as boolean) tells the editor whether or not to make a selection of the lines as it moves up. The second parameter [count as integer] tells the editor how many lines to go up.

Here's my final solution for copying text to the user's clipboard:
This requires the following imports
Imports System.Text.RegularExpressions
Imports EnvDTE
Imports EnvDTE80

'copies the given text to the IDE clipboard
Friend Sub CopyTextToClipboard(ByVal textToCopy As String)
  Dim sel As TextSelection = DTE.ActiveDocument.Selection
  With sel
    DTE.UndoContext.Open("ToClipboard") 'an arbitrary name

    Dim mc As MatchCollection = Regex.Matches(textToCopy, "\n")
    Dim lineCount As Integer = mc.Count + 1 'add one for the NewLine (below)

    'add the text into the VS code editor
    .Text = textToCopy & ControlChars.NewLine

    'now select the text that we pasted
    .LineUp(True, lineCount) 'True creates a selection region

    'cut it to put it into THEIR clipboard
    DTE.ExecuteCommand("Edit.Cut") 'by using Edit.Cut it accesses the IDE Clipboard!

  End With
End Sub

In this particular situation I frequently paste over 100 lines into the clipboard and watching Visual Studio load those lines into the IDE and then cut them right back out is a unnecessarily time consuming. If anyone knows a faster method for rendering text into the IDE or the clipboard I would love to hear from you.

Thanks for joining me on this adventure.

Sunday, July 1, 2007

Macros That Display Windows Screens. A journey to the GAC and beyond.

In today's adventure Kennon will try to implement a macro for adding regions into his source code. This is a fairly simple macro to write but hold onto your seats because this is the most dangerous adventure to date. Today's journey will travel to the GAC and beyond! First of all, if you have not yet discovered macros consider reading p. 171+ in Programming Microsoft Visual Basic 2005: The Language by Francesco Balena. Here is a very brief introduction if you don't happen to have Balena's excellent book handy. A macro allows you to record repetitive keystrokes or other actions and then replay those strokes again and again as needed. Macros can be created, edited and executed using the "Macro Explorer" window which can be launched from Tools-->Macros. Here is an example that inserts a code region into a source file (for whatever reason I have the hardest time typing #Region ""):

Public Sub CreateRegion(Optional ByVal regionName As String = "")
  If String.IsNullOrEmpty(regionName) Then
    regionName = InputBox("Region Name:", "Region Name:", "Imports")

  End If

  If regionName.Length > 0 Then
    Dim sel As TextSelection = DTE.ActiveDocument.Selection
    Dim textToWorkWith As String = sel.Text
    If textToWorkWith.EndsWith(vbCrLf) Then
      textToWorkWith = textToWorkWith.Substring(0, textToWorkWith.Length - 2)
    End If
    textToWorkWith = "#Region """ & regionName &amp;amp;amp;amp;amp;amp; """" & vbCrLf & _ textToWorkWith & vbCrLf & "#End Region" & vbCrLf
    sel.Text = textToWorkWith
  End If
End Sub

This seems like a lot of trouble to go through just to avoid typing #Region but one recommendation I received on Macro writing was, "Start small." Since writing this macro I have expanded my horizons to write a macro that writes my name, the date and check-in comments in my source code, then optionally checks the file into my version control system and then optionally closes the source file. It is a real timesaver and means that I'll never have to wonder what the date is again! Because macros allow you to pass 1 (only) optional string (no other data types are supported) parameter as well as display input boxes and message boxes they offer a great deal of flexibility. Furthermore, macros are written in VB.Net so you have broad access to the .Net Framework. Macros can also be executed from the command window by typing ">Macros.MyMacros.. " where ModuleName is the module name where your macro method was written. The module name is usually "RecordingModule" when you are first starting out but once you start to write many more macros you might start to group different macros into different modules. If your macro accepts an optional argument you can pass that argument as the value of . Note that this value does not need to be enclosed in quotation marks. One workaround for the single parameter limitation is to pass multiple values concatenated together all as a single value and do some parsing to break apart the separate values. Here is an example of how I can use the command window to call my "CreateRegion" macro and pass "Imports" as the optional parameter. >Macros.MyMacros.RecordingModule.CreateRegion Imports When you look at that syntax you have to scratch your head and say, "That takes too long. Why doesn't this guy just type #Region "Imports" and get on with his life?" That is a pretty valid question but because the command window allows for aliases to be assigned to macros (or any other IDE function) I can type the following to quickly assign "imp" as my alias for "Macros.MyMacros.RecordingModule.CreateRegion Imports". Here is how: >alias imp Macros.MyMacros.RecordingModule.CreateRegion Imports Now you might be saying, "That's pretty fast but accessing the command window is awkward and slow. Is there a faster way?" Luckily, there is. Macros can be assigned to keyboard shortcuts (Tools-->Options then the "Keyboard" option). On this screen you can quickly find the command you want by typing a filter into the "Show commands containing:" textbox and then assign a shortcut by putting the focus in the "Press shortcut keys:" textbox and typing your desired combination and clicking "Assign." Well, this is great but there is no way for me to pass a parameter (e.g. Imports) to my CreateRegion method so in this case this is a step backward although in most cases these keyboard shortcuts are very helpful. By the way, the keyboard shortcut assignment screen is a great screen for searching for IDE command names that you will need to reference when writing macros. For example, within the VS Macro Editor you can execute any IDE command (I think?) by coding DTE.ExecuteCommand(""). For example, DTE.ExecuteCommand("File.Close") will close a file. If you are not sure whether it should be "File.Exit" or "File.Close" the keyboard shortcut screen is a great way to find out!

OK, back to our adventure. Although I like the ability to simply type "imp" in the command window and get my imports region the fact is there are about 20 common regions that I need to create regularly. I could create command window aliases for each but then I'd have 20 aliases to remember. I could write 20 different macros and assign 20 different shortcut keys (ouch!). I could give up on using macros and create 20 code snippets (messy). Then the obvious occurred to me, "I could create a combo box so that I could just select the desired region name!" That was so obvious that you must be thinking, "Duh, there is no way this guy should be a programmer!" Well, here's the catch. Apparently (perhaps for security reasons) the entire macros team at Microsoft never saw this as a possibility either because there is no documented way to add a windows screen to a macro project. The work around is simple, or so I thought. I can just write my combo box screen in a different assembly and then reference it from my Macro project and this where this adventure heads into uncharted waters.

First, I wrote my other application (MacroHelpers.dll) that contains one public dialog screen that has a combo box with all of my region names and compiled my assembly. Now, all I needed to do was add the reference to MacroHelpers.dll from within my macros project and I would be all set. WRONG! It turns out macro projects only allow the addition of references that show up in the .NET reference tab and my MacroHelpers.dll definitely did not qualify as something special enough to be listed in the as a .Net reference.

OK, it was time for plan B. I'll sign MacroHelpers.dll using a .snk file (see the "Signing" tab in the "My Project" screen) and register it in the GAC using GacUtil.exe. That should do it! Plan B sounded more fun anyway because it would give me a chance to add post build events (see the "Build Events" button on the "Compile" tab of the "My Project" screen). So, this was my first task. How could I add a build event that would register my DLL using GacUtil.exe? It turned out to be fairly simple based on everything I read online here was the suggestion:

gacutil /i $(TargetPath)

Well, this blew up with a startlingly loud bang and returned this message:
The command "gacutil /i "C:\Developer Resources\MacroHelpers\bin\Debug\MacroHelpers.dll"" exited with code 9009.

Part of the reason this blew up was my "TargetPath" included spaces (e.g. "Developer Resources") so I had to enclose the keyword $(TargetPath) in quotes. Like so, gacutil /i "$(TargetPath)". Well, this still blew up! I thought, maybe it needs to say gacutil.exe /i "$(TargetPath)" but that still failed. Luckily, this page provided some good advice. Basically, it stated that gacutil may not be accessible without path information and it recommended the following:
"$(DevEnvDir)..\..\SDK\v2.0\Bin\gacutil.exe" /i "$(TargetPath)"

Ah, sweet success! Now, that I had registered by strongly named assembly and restarted VS and the VS Macro Editor I should finally see MacroHelpers.dll in the .Net references list! WRONG! My assembly was still not there but at this point I was not going to give up and face extinction. No matter how simple it is to type #Region!

It was time for plan C! OK, I didn't start with a plan C but now I needed to make one up. I got some good advice from the codeproject which sent me to KB 306149 which summarized to: To display your assembly in the Add Reference dialog box, you can add a registry key, such as the following, which points to the location of the assembly:

That sounded great and the fact that is was a Microsoft knowledgebase item really got my hopes up. My optimism was soon squashed because this did not work either. I made the entry and restarted VS and restarted VS and maybe even restarted it a 3rd time but it looked like plan C was not going to work either.

It was time for plan D! I was starting to worry that I might run out of letters for my plans and have to resort to the greek alphabet when finally I took this one drastic step and changed my build events to this (3 lines, ignore any line breaks that were forced by your browser):
"$(DevEnvDir)..\..\SDK\v2.0\Bin\gacutil.exe" /u

COPY "$(TargetPath)" "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\$(TargetFileName)"

"$(DevEnvDir)..\..\SDK\v2.0\Bin\gacutil.exe" /i "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\$(TargetFileName)"

OK, for those of you that didn't read that carefully, I decided to copy my lowly little MacroHelpers.dll into the C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727 directory! Now, I'm not necessarily recommending that you try that at work unless you cannot be fired but it worked for me! Now my macro project could reference my MacroHelpers.dll and I could start launching with windows screens from within my macros! Ah, sweet success.

I guess when you are battling extinction it is good to have a "Never say 'Die' attitude." Today, it paid off.

Update on 07/16/07 - After reading page 105 of Inside Microsoft Visual Studio .Net (2003 edition) by Johnson, Skibo and Young I found the correct way to make an assembly available as a reference within a Macro project. The correct thing to do (and avoid the headaches I experienced) is to copy the DLL to the C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\PublicAssemblies\ directory. So the correct build event syntax would be:

copy "$(TargetPath)" "C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\PublicAssemblies\$(TargetFileName)"

There are a couple of follow ups to add as well:
1) It is important to mention that the first time I ran the macro the dialog screen displayed behind VS05 so I had to use Alt+Tab to find it. VS gave me the nasty "Keep Waiting" message but eventually I found the screen. To fix this, simply change the dialog screen's "TopMost" property to "True" and rebuilt the helper application.
2) It is possible that my usage of a macro for this entire experiment is inappropriate. In addition to writing macros you can also write Add-Ins and perhaps those were designed with more user interaction in mind but the resources I had on Adds-Ins stated that they were more difficult and involved registration in the GAC and when I started this little project I had no idea that my adventure would eventually lead to the GAC and beyond.

3) Part of the reason for being so stubborn on this was that I wanted to standardize region names across my development team. So the combo box implementation will help with this goal.

Thanks for sharing this adventure with me.