Fear and Loathing

As part of the Security on the Brain series of webcasts, I'm presenting an MSDN webcast this Wednesday called "Security - End to End in SharePoint". This is a 1 hour webcast on all aspects of security with SharePoint 2007 covering:

• How to configure authentication
• How to manage permissions bottom to top
• How to securely configure a web farm

This session describes the end-to-end security architecture in SharePoint, including Windows authentication, pluggable forms and Web SSO authentication, groups using SharePoint, system-wide and fine-grained authorization control, developer control over identity and permissions, administrative security options, and upgrade considerations. This session will also cover integration with Active Directory® Federation Services.

You can still register for the webcast which starts at 1PM EST here. See you there!

Quite some time ago I found out MSDN was moving towards a community content driven approach. A wiki style approach to the vast tome of knowledge known to us as the MSDN Library. This was a good move and I wanted to see it blast off as soon as possible. However the implementation has been somewhat lackluster.

When I found out about MSDNWiki I was excited. The PHP and mySQL community have had a feature with their online documentation where users contribute information about each of the topics. Items like bugs, workarounds, examples, and further explanation of the topic are scattered throughout the original documentation. It's an absolutely wonderful environment because not only do you get the syntactical expressions of how to call say ctype_alnum in PHP, but you also get a tip that it could behave strangely depending on the type you pass it. Brilliant.

Here's an example of the user defined types in mySQL from the community:

User Comments

Posted by Joey Bartlett on April 14 2006 7:54am
These are very useful. You can use them for ranks.
Example:
SET @pos=0;
SELECT @pos:=@pos+1,name FROM players ORDER BY score DESC;

This is the way community content should be. Only last week I was reading someone's blog that they went to the SharePoint SDK online only to find 3 or 4 properties with the bare bones, auto-generated documentation that provided nothing for them (or others).

However the implementation of MSDN community content falls a little short. Sure, users can go online and add content to some areas in the documentation. Some. Community content in MSDN documentation isn't about items that have content, it's about the content itself. Don't give me a mechanism to find entries that have content added to them because that's not how we use documentation. We use it by going to the class that we need help on. If there's content for that particular topic, great. So my suggestion is just open up all topics to community content. I've read through various junctions that "many other developer sets will add this feature over the next few weeks and months". What are we waiting for? If Microsoft is serious about engaging the community and have us supplement their documentation with real-world experience, tips and tricks, and information that's better than a skim of the XML comments in their source code, then we need to be engaged; fully, madly, deeply.

In addition, I personally prefer to use my local help as the primary source but in this day and age surely Microsoft can figure a way so that when I use my local help it can, in addition, supplement each help topic with comments from the online community? Am I asking for too much here? I don't think so. Each help topic, local or otherwise, needs to have a section at the bottom of it where anyone (maybe via your Windows Live login) can author content. A simple snippet of code, a simple tip or advice on the method or property in question. If I'm disconnected then just display the help content for me sans community content. This is neither rocket nor science. 

Why can't my world be like this someday:

We could get this today without Microsoft's help although I'm not sure I want to go down that path (or if I legally could do it). Someone could setup a web site with a simple app where you create an account (for identification purposes, basic stuff like email and name) and use that account to author content. Someone else could decompile the CHM files into their separate bits and writing a small utility, inject a link to the online site in order to pull down user contributed content and insert it into a new CHM. Additionaly, using IFrames, the online site could combine the MSDN and contributed content to form an experience like what the mySQL and PHP documentation sites do now. Like I said, this is just off the top of my head and not a solution by any means and there may be issues with republishing Microsoft content; not all documentation is in CHM format; blah; blah; blah; blah.

Wouldn't it be a wonderful world if we looked up the documentation for Microsoft.SharePoint.Help.HCLockAction and actually found contributed content from users who have done something with this enumeration? Or an example of what you would use it for? Or something other than the auto-generated docs spit out from code comments that tells us nothing because the product team was focused on delivering the product (which is what they should be doing).

Now that is real community content. I can only wish we move towards that model someday. Nudge, nudge, wink, wink, say no more, say no more.

If you're looking for the best SharePoint info then look no further. The Newsgator folks have put together an OPML with all of us SharePoint MVPs in it. Very nice to simply subscribe to this and you'll get the entire gang in one feed. Check it out here.

Just saw that the Visual Studio for Database Professionals is on MSDN now. It RTMed earlier this week but you can download the DVD image now. Also there is the MSDN Library to accompany it and for those without a Team Suite license, a trial version is available. I'll be doing some more posts on this later as I'm doing a demo next week of it and it's cool capabilities for the DataGuy (or DataGirl) in you.

Also I found a little tip that might help you in avoiding to download the 3.2GB version. If you have a copy of Team Suite installed (not a specific flavour like Developer, Test, etc.) then you can download the file marked "Trial" (the 20MB one). When this version is installed over top of a copy of Team Suite, it becomes a full copy. Why you can do this and avoid the 3.2GB download of a DVD image is beyond me, but then I'm not into the whole marketing thing.

Oh bother. Visual Studio 2003 and Cruise Control.NET. Simple and elegant. A basic NAnt script to build the solution and you're good to go. Run NUnit, output goes to a format CC can understand and Bob's yer uncle. Let me quantify this. Our cruise server has a subversion client (command line) and the .NET 1.1 SDK. Visual Studio isn't installed because, duh, it's a server and cruise just needs something to build the system with.

Enter Visual Studio 2005. I just recently setup CI for our 2005 projects but it's just plain ugly, in so many ways. First there was trying to get the system to build using MSBuild. That was fine because you can simply enter this:

msbuild /t:rebuild solutionname.sln

(or whatever target you want like Debug or Release)

Like I said, if that's all it was no problem but it gets real ugly real fast.

First there's VSTS unit test projects. The team is all equipped with Visual Studio Team Suite. An expensive proposition, but one made long ago by someone wiser than me. No problem though. We're not really using the Test Manager much, there are no automated web tests, so we just write unit tests (and run them with Jamie Cansdales excellent TestDriven.NET). However when MSBuild gets ahold of a solution that contains a VS unit test project it needs some additional set of assemblies (assemblies buried in the GAC_MSIL and other places). The snippet in the ccnet.config file to get MSBuild going was pretty straight forward:

<msbuild>
    <executable>C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\MSBuild.exe</executable>
    <workingDirectory>D:\ccnet\projects\ProjectName</workingDirectory>
    <projectFile>SolutionName.sln</projectFile>
    <buildArgs>/noconsolelogger /p:Configuration=AutomatedBuild /v:diag</buildArgs>
    <targets>Build</targets>
    <timeout>600</timeout>
    <logger>C:\Program Files\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MsBuild.dll</logger>
</msbuild>

Through brute force (run MSBuild, figure out what assembly it needs, go find it, lather, rinse, repeat) I was able to get a 2005 solution (with unit test projects) to compile. However actually running the tests is another story. Again brute force reigned supreme here as I trodded through an hour or two of running MSTest.exe to try to coax a couple hundred unit tests to run. The cc.config entry for getting some unit tests looks something like this:

<exec>
    <executable>C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\MSTest.exe</executable>
    <baseDirectory>d:\ccnet\projects\ProjectName</baseDirectory>
    <buildArgs>/testcontainer:Source\Tests\UnitTests\bin\AutomatedBuild\UnitTests.dll /runconfig:localtestrun.Testrunconfig /resultsfile:testResults.trx</buildArgs>
    <buildTimeoutSeconds>600</buildTimeoutSeconds>
</exec>

Remember that I said this sever did not have Visual Studio installed. For the VS2005 solutions, I just installed the .NET SDK 2.0 and that was good enough. Although MSTest.exe isn't included, I snagged the file (and it's crazy set of additional DLLs scattered all over the hard drive) from another system and stuck it where the EXE was so it would be happy. 

No dice on running MSTest against a unit test project. It started down the path of running the test, but then hit a crazy COM error (CLSID not registered) and that was enough for me. No way I'm going to track down all the stupid COM registry settings for Visual Studio 2005. And what the hell was this? COM? I thought we got rid of that about 3 compilers ago?

So I was stuck to installing a full Team Suite on the server. Okay, I'll bite. I mean, I don't need everything so it'll be okay (I keep telling myself as I watch a million registry entries fill up). A few hours later and I'm staring at my command prompt again and type in my MSTest.exe command. And a dialog box pops up. Yup, a modal dialog box that tells me something is wrong (I don't remember the specifics but it wasn't very informative).

Visual Studio was installed fine and I could compile and run and build the solution I was trying to, but testing from the command line with MSTest.exe was a no-go. No matter how hard I tried, how much I cleaned up, or how many virgins I sacrificed it just won't go. And there's another problem. With 2003 and our NUnit tests, we go some nice stats. Timings, informative messages, all that good stuff. With MSTest we get nothing (other than x number of tests ran and maybe a failure, but I couldn't get that far to see if it would give the details of the failure). On top of that, the MSBuild logger bites and produces about 10,000 lines of gobbly-gook that isn't very useful. Yes, I could spend my time writing new xslt to make it pretty, trim out the "Everything was okay" lines that fill up the MSBuild task logger but I think that's not value-added at my rates.

Here's a sample email I got from the build which gives you an idea of how useless a CC.NET/MSBuild/MSTest combo is:

It's ugly. Really ugly. MSBuild produces a ton of crazy output. Writing a new MSBuild file is no walk in the park and even for someone like me that knows NAnt quite well, I'm still wrapping my head around how MSBuild does stuff. I had to create a special configuration inside of Visual Studio to omit our Report project because MSBuild can't build them (hence the target AutomatedBuild above which is not the same configuration that our developers use and something I don't like doing because that's one point of a CI server, consistent builds for everyone).

From the output above, Cruise said the build was okay but there's an error message that came out of the MSBuild logger (our report project). This is a 2005 project so the information makes no sense (I belive it's a bug that's logged but who knows when it might get fixed). And I really can't integrate the MSTest output into the email because, well, there is none. There's a hundred tests or so in this project but the logger doesn't produce anything. 

Additionaly, there are some project types MSBuild can't handle and again, it takes a rocket scientist to create an MSBuild file (even using something cool like MSBuild Sidekick) that can call another MSBuild task and exclude certain projects. This certainly isn't as easy as it was in 2003 where you just created an exclusion list (we excluded our client app as there was no extra license for the grid control and a modal dialog came up when the trial version was even looked at by the compiler).

Okay, this is mostly a rant but there's some wisdom here. Continuous Integration does not need to be this hard. CruiseControl.NET is an excellent tool and very flexible with new tools and integrating output from those tools. However when those tools require a million registry settings and even more DLLs (put in very specific places, trust me, you can't just toss them in the GAC and call it a day) and dump gobs of XML that no mere mortal (well maybe DonXml could) would be able to translate, it's just wrong. And as for the built-in Team Build that you could us, that's equally as useless as a) you can't schedule builds to trigger off of code checkins and b) again it requires a full Team Suite client to be installed on the server. Worst case scenario when I first started setting up CC.NET servers it took 4 hours. Now I can get it done in an hour (which includes testing the build of the first project). I've already spent a good day on just trying to get something to compile. It's compiling now but the output is crap and no running of tests and thats just not good enough for me. You can get MSBuild and (maybe) MSTest running with CruiseControl.NET. It's not impossible. If you install the full client and your projects are "just right" it'll work but the output isn't that hot and you'll watch your server CPU slam when compiles happen.

My advice, avoid trying to combine MSBuild, MSTest, and CruiseControl and stick to NAnt, and NUnit (using MSBuild if you have to when building 2005 solutions).

Needless to say I'm looking at one option right now. Dumping the install of VS2005 on the server, changing all our unit tests back to NUnit and using NAnt to do everything (and just calling MSBuild as an exec task for VS2005 projects). I still need to run 2003 projects so our CI server is going to look like a Frankenstein monster by the time I'm done, building VS2003 and VS2005 projects, running NUnit, MSBuild, NAnt, Simian, FxCop and whatever else we have in our mix. Even given that, using NAnt the output will be simpler (and integrate well with CC.NET), test output will be useful and informative, and I don't need to install a $15,000 piece of software on a server that has the distinct possibility to pop-up a modal dialog someday when it can't find some registry key.

Grrr. Argh.

And finally getting caught up!!!

Oh man, it's been a roller-coaster ride the past while but I think the worst is over (and the best is yet to come). Vista launches, SharePoint RTMs, new VMs, two sprints completed, new ScrumMaster duties on YAP [yet another project], new laptop, dogs and cats living together... mass hysteria! Tonight I'm finally taking a breather, just to get back on track this weekend with a bunch of stuff like posting crazy entries around everything that's been building in my own backlog.

A couple of things as I'm sifting through the rubble. I broke down and bought a new laptop. My Presario 4000 was good (it was aimed as a desktop replacement) and a year and a half ago it was. Fast, heavy, but impressive. Most impressive. Now it's all about Core 2 Duo and that so I snagged a Inspiron 9400 from Dell and couldn't be happier. It's an inch larger all around, but half as thin; 3lbs lighter; 100db quieter; and about the same price as I paid for my Compaq. 4-5 days of setting up and installing software and tools and I think it's good to go. I've already given one presentation on using NetTiers and CodeSmith with it (which was a bit of a joke trying to get used to the new keyboard so lots of typo's in the demo).

Secondly I saw that JetBrains has released their Omea Pro product for free now. I was using the free Reader but now have updated to the Pro. It's everything (mail, rss, newsgroups, tasks, etc.) all in one package and very well done. I gave up on RSS Bandit as it was just chewing up memory like there's no tommorow and Omea lets me flag things say in newsgroup threads so I don't have to read every thread out there. The uber cool thing? They're going open source so we'll get to see how the guys who created ReSharper write code. Very cool learning experience and a very cool tool that will hopefully grow when the community gets its hands on it (I would like to see a WPF version come out of that). Also there's a 2.5 EAP of ReSharper out you can grab, although it's a little buggy (as all EAPs have been with these guys) but looks like it's going to be killer when it releases.

BTW, if you really only want to subscribe to one blog to find cool (geek) stuff out there check out Jason Haley's Interesting Finds. He's recently (the last month or two) added short comments to the blog entries so it's a great feed to look at and keep on top of cool stuff he finds (most of which I consider cool as well) without having to subscribe to everyone on the planet.

Just wanted to let you know I haven't dropped off the face of the planet, it's just been a crazy couple of weeks after returning from DevConnections in Vegas. I'm on and offline so posts are spotty. I'm also up here in Edmonton, Alberta for the first Vista/Office 2007 launch event. This is the first launch event for Vista, Office 2007 (client and server) and Exchange 2007. Funny how the first event is Canadian (go Canada eh!) as the U.S. launch is in January.

Anyways, I'm at the experts booth today at the event so if you're there, stop by and ask questions and make me work for my lunch. We'll also have some computers setup so we'll see if we can get a demo or two going. It should be a blast.

Last night we had the VIP event at Brewsters here in Edmonton. A great turnout from the various UG leaders, local MVPs, and softies. I met up with local Access MVPs Tony Toews (and finally found out how to pronounce his last name) and Albert Kallal, a MVP that shares a last name with Superman and bears an uncanny physical resemblance to BillG.

My fellow plumber John "The Pimp" Bristowe (sporting the latest cool swag, the Developer Night in Canada t-shirt) is giving most of the dev sessions today and is on the road with the event. He was there last night (along with my other Plumber/MVP James Kovacs). John was emcee for the night, handing out the usual t-shirt swag and other goodies from the MS Warehouse and telling us again how big the one he almost caught was. It's going to be a long day today with a full day of sessions, a reception tonight and a 3 hour drive home to Calgary in the snow.

Catchup blogs in the queue:

• Code and resource links from DevConnections. Just have to zip up the source and get it online, blog post is done.
• SharePoint Forums and Knowledgebase web parts updates including 2007 (RTM) compatibility. Aiming to get this out in the next week
• Scrum retrospectives from two iterations on a few projects, should be fun for some discussion and learning
• Launch event (with pics) and last nights VIP event (with more pics). I'll also be at the launch event and VIP reception the night before in Calgary next week.
• A couple of other small (new) SharePoint tools for developers

So much time and so little to do. Wait a minute. Strike that. Reverse it.

Joel Oleson, a senior product manager on Office SharePoint Server, announced on the SharePoint Team blog the RTW (Release to Web) version of WSS and Office SharePoint Server 2007 (Standard and Enteprise editions). These are available for download so you can start using WSS immediately in a production environment (this is the final version) and install eval versions of MOSS if you so desire. Joel even posted the product keys for installation if you need them and where to download the files. Check out his post here.

Jeff Teper, Mr. BigWig himself (the corporate vice president of SharePoint Server), also has a post here describing the journey and how important MOSS is to the even bigger guy, BillG. Congrats guys, it's been a long and winding road but we're finally there (or here, depending on which way you're looking at it).

Now back to my DevConnections post and other SharePoint-y stuff.