Fear and Loathing

Scott Hanselman is, hands down, simply my hero today. A couple of years ago he posted a pretty massive list of useful tools, utilities, and must have stuff a any self-respecting geek should have on his hard drive. Now he's updated the list and it's massive (and broken down into catagories to boot). I find that I use about 1/2 of the tools listed and now I'm off to investigate the other half. If there's one blog entry you need to bookmark, clip, or copy and paste into OneNote (if that's your thing) this is it. Check out his 2005 updated list here. Everyone else is.

Here's my personal top 10 from his list (plus some that are not there) that I faithfully use every day:

• ReSharper - Basically live and die by this. I can do all the refactorings by hand, but it's just so sweet to rename a class and have 30 files update automatically.
• WinMerge - Fantastic util and very handy for trying to diff SharePoint xml files between environments. Open source to boot.
• NAnt - Build, build, build (and do a whole lot more)
• TestDriven.Net - Test, test, test (and then refactor and test some more)
• OneNote - I clip tons of stuff everyday. It's from my old days as a graphic artist where I would clip pictures and put them in what's called a "graveyard". OneNote lets me do this, search it, organize and much more. I also keep a OneNote section on blog ideas and whatnot. Very handy!
• Notepad++ - Scott pushes Notepad2, but I prefer this Notepad replacement. YMMV.
• Midnight Commander - Scott doesn't have this listed so call me a command line guy, but I prefer this util over Explorer anyday. Reminds me of the odl XTree tool (which I used all the time as well)
• CopySourceAsHtml - Great for blog postings!
• Reflector - I'm poking inside Microsofts assemblies everyday trying to see what makes things tick.
• Web Services Studio 2.0 - This is an invaluable tool when trying to debug Web Services from SharePoint. Get it now!

What a week of living H-E-double-hockey-sticks as I've had about 10 hours of sleep for the entire week (which is a bit low even for me) and basically got very little done. Two things that I wanted to mention though.

Yesterday I had the pleasure of working with Kit George, the Program Manager for the CLR team at Microsoft. The CLR team was in Calgary these past few days (and have been in Canada for the past week) helping out customers with some compatibility testing with the 2.0 framework. It was great talking to Kit and found a few intersting facts out I didn't know. There are about 66 developers on the CLR team alone which gives you a real idea of how big the entire team is (so add on project/program managers, architects, testers, etc.). Oh yeah, and if you're looking forward to the GZipStream class coming in System.IO.Compression like I am (goodbye SharpZipLib!), you can thank Kit. That's his baby.

If you're looking for a migration path then I suggest you install the framework. Really. Just install it. You can get the redist here which, like it's 1.1 cousin, is just a single 20mb setup.exe that you run. Install it on a development server and in your Web Site properties in IIS just retarget it for 2.0. No iisreset needed. No reboot required. Your 1.1 ASP.NET apps should work fine (they'll be a short JIT when someone accesses them for the first time). We only did a small test (3 or 4 apps) but the 30 or so others running on that server didn't show any issues and nobody reported any problems. Try it out for awhile like that. Then you can think about the recompiling of apps, which is a different issue than just retargeting for the platform. The retargeting and installation of the 2.0 framework is pretty simple. We did find a silly problem sometimes because the ViewState was cached, so just relaunch your browser to fix it. This was when we were in the middle of a task, retargeted the system, then just tried carrying on from there. Not something you would normally do. If you do deploy and try testing but still find any issues you might want to drop a note off to Kit and his team.

If you're looking to recompile (and you will at some point) then you'll need of course Visual Studio and all that jazz. While the retargeting is simple and the team says it'll probably work fine 99% of the time (there will be those off things that are exceptions, again if you come across them let Microsoft know!) it's the recompiling that's going to bite you. I've been recompiling apps here at work from time to time (what else does a geek like me do on his lunchbreak?) to see what kind of issues there are (not testing the apps, just recompiling the solutions) and there are some deprecated things that you'll need to rework your code at some point (one of the biggest ones is system.configuration). The other thing is splitting up of assemblies. Let's say you have a common assembly for logging or something that 10 ASP.NET apps use. You can't be expected to upgrade all 10 apps at the same time as the testing effort might be too much to bear. So you'll be in a mixed mode scenario where your environment might be targetted for 2.0 and a few of the apps are native 2.0 (migrated and recompiled) but others (like common assemblies) are still 1.1. This is the scenario that needs the most testing and you'll probably stumble over issues (DateTime serialization being one of the biggest ones). So just something to watch out for and that there's going to be some work migrating to 2.0.

Finally Serge posted a great example in the comments from Thursdays post of how he accesses the SQL directly for retrieving things from a command line (via bcp but you can do it with osql as well). He did say that he tried my Wrapper classes but they didn't provide enough depth of functionality and were a bit of work. This resonates with me that I need to shift a bunch of focus on the library and make it useful instead of a bit of a toy which is what it kind of is right now. If I'm going to go around preaching for people to write remote apps I feel that I should provide you with an easy to use and rich tool to do this. So I will be getting back to putting more time into the wrappers and hopefully provide a complete set of objects that anyone can use for any purpose. Web Services are still slower than the Object Model but the advantage is that you don't have to run it directly on the server and some caching might be able to speed things up a bit. I think it'll be a long time before Microsoft does something like make the SharePoint object model remotable (if ever) so hopefully this will help in the meantime.

As I get down off my soapbox from yesterday about overpriced third-party SharePoint tools and components I have a new gripe (seems like a good week for this). Time and time again I'm seeing two things become apparent in tools that are coming onto the market. First, some are just bypassing the rules of the road and doing stuff directly against the SQL databases. Second, about 9 out of 10 tools must be run directly on the server and require a local administrator.

Okay, first on the SQL stuff. I don't understand why developers can't get over this. You've seen numerous posts (including Fitz going on a few times about it) to get out of the database. So why do you keep doing it? My only reasoning is that you're frustrated by the lack of access to things that we think should be apparant and well, we can all write T-SQL so why not talk to it directly? What's the harm? I'm not going to get into the issues of what the database is doing as Fitz had a good explaination around that. Maybe that position should change though as more and more products are just blatently going against the database to do things like counting the number of sites in a portal. Rather than looping through a property in the Object Model, a quick "SELECT COUNT(1) FROM WEBS" will do the trick. So the $10,000 question might be should we be doing this? If you're using the database directly can you do simple selects, counts, etc. as long as you're not doing updates?

Second is the ever growing number of tools that keep requiring two things. One to be run directly on the server and two to be run by a local administrator. It's great for some tools because they are administrator tools, but I think the one big thing with SharePoint that people are having trouble with is the admisistration. More and more companies are either outsourcing their infrastructure or trying to get everyone to do their jobs from a single location. At our company, a huge effort just finished on stopping the free love access that users had to systems. The other thing is that a SharePoint administrator shouldn't have to have local admin to the SharePoint box. In a clustered or scaled out scenario, that means they really need access to several boxes (including the database servers). I think what we need is more tools that users can use off their desktop (via Web Services) to do the mundane stuff that SharePoint doesn't necessarily expose through the Web UI, or for custom tasks that are specific to your business needs. Again, we're given lots of tools but most of them we might be in a position to not be able to use in our environment due to restrictions on our infrastructure. I don't think the answer is to keep opening up the infrastructure to allow a SharePoint admin local root power but that's just me.

A long time ago in a technology far, far, away I used to write these things called Doors that ran on community-based, dial-up, text-based systems called BBSes (yes, before the big bad "Internet"). A BBS was like a website (but not really) where communities got together, posted messages, swaped files and email and played games (sounds like a website huh?). Doors were all the rage and I ran many systems and wrote a bunch of Door games and all was well in the universe.

Along game a man named Tim Stryker who was a bit of a radical, but basically a genius. He (and others) put together a system called MajorBBS (MBBS for short). This was a unique system over traditional BBS systems of the time because rather than spawning off separate processes to run all these Door games available, MBBS would run them as Dynamic Link Librairies (DLLs) and all modules in a system would run in real-time (at first they were actually linked into the system and you had to recompile the system when you bought a new module, crazy huh?). Things like real-time chatting, real-time updates from people playing in games, were now possible with the approach and technology Mr. Stryker and Galacticomm built.

However here's the catch. Building a regular (non-MBBS) Door game was pretty much a no-brainer for a good programmer. There were some frameworks out there that handled the Serial IO for you (remember this is all dial-up, async, 2400 baud and all that) but for the most part you wrote a single player application and only worried about it while it was running. The Door would launch, the user would play, the executable would end and the user was shuffled back to the BBS system from whence he (or she) came. Then Major BBS modules appeared but there was a bit of a catch on building these as you had to now deal with shared memory, be wary of heavy processing and not taking down all the other lines (up to 255 on a single PC!) and stuff like that. Additionally you had to outfit yourself with some pretty specialized tools. Borland C was the only compiler you could use (later Microsoft Visual C was able to do the job), a special memory extender library (PharLap) had to be used, etc.

On top of all that, you also had to buy your MBBS system just to test your code. All in all, I personally invested about $2000-$3000 bucks to build my development environment which was a lot for back then. To offset this, Major BBS developers could charge an arm and leg for their products. Even a simple game like Tic-Tac-Toe (with a chat feature) would sell for hundreds of dollars. I built small RPG games that I sold for over $1000. That's each. And you could get away with selling licenses for more lines. The more lines (users) a BBS wanted, the more your could charge. The short of it was that we could get away with this because it was difficult to build the software, an investment for the environment, and then there was support and the fact that any guy off the street couldn't just sit down one night and say "I'm going to build a MBBS module and make a boatload of cash". A Door developer on the other hand could knock off a pretty decent game in a few days/weeks and sell it for $20 a pop and laugh all the way to the bank.

There are parallels with BBS Door Development that can be done on a shoestring and say writing ASP.NET web apps using free tools today. I can download the .NET SDK for free and compile my system. Web Matrix isn't a bad tool. I can deploy or sell my web solution or I could build .NET client applications using something like SharpDevelop. In other words, I can build the Doors of yesterday today with .NET and some cheap (free) tools. With Web Parts and SharePoint the experience is very much like what we went through with Major BBS development. You need a specialized environment (Windows Server 2003). You need a specific IDE (Visual Studio .NET although you *might* be able to use something like SharpDevelop). You need to develop right on the server or else put up with very complicated ways of remote debugging (trust me, debugging with PharLap and protected memory dumps was no picnic either). You need to know a lot of about how SharePoint works, what works (and what doesn't) and how to twist it to do your bidding. Development in SharePoint is not a walk in the park and can be expensive to setup and work with.

Why am I telling you all this? I'm seeing the MBBS trend happen again but this time in the SharePoint space. No, people are not writing Door games for SharePoint (although that does entice my already overtaxed project list). There are however lots of niche products coming out and while some are great, some not so great, they're (for the most part) expensive. I'm also not talking about expensive as in comparing them to 1980s prices or through inflation but just the fact that a small widget (or set of widgets) that does something useful (say backup files, deploys changes to your SharePoint sites, provides better than basic workflow, etc.) is pretty costly for what it does.

Don't get me wrong. I'm all for free enterprise and making money however I just look across the SharePoint tool and Web Part space and feel a little deja vu coming on. Again, there is an investment here and a return that companies want to see on that investment. However it just *feels* costly for a few Web Parts. Maybe it's me, maybe times have changed, but is it really worth thousands of dollars to a company to purchase an out of the box Recycle Bin for SharePoint? Sure there are free tools and low(er) cost Tools, Web Parts, and Solutions coming out as well but my gut feel is that when you direct someone to a commercial product more often than not it's going to cost a few bucks to get that tool. I look at just a sampling of a lot of tools and we're talking $5k, $10k, and up for many of them.

Again, I'm not trying to paint everyone with a single brush and I guess cost is relative and in the grand scheme of things, some companies have that kind of money to invest and don't see it as a large outlay. I just feel like I'm back in 1980 where we, the MBBS developers, could charge 10 times more for a module than a BBS door just because we knew how to build them. I might be alone in my thoughts here but hey, it's my blog and I'll ramble if I want to.

I'm playing catch up today as things in the universe are just a little off kilter for me.

The weekend was a bit of a bust with regards to getting my Remote SharePoint Explorer out as I was bogged down in re-installing (yet again) Windows XP. For the life of me, I couldn't get my SATA drive to boot as the master device so couldn't get my OS moved from a small 40gb drive to where it should have been in the first place, on a nice happy 200gb one. So I had to run out and grab a 200gb IDE which the system would recognize. Then I spent the better part of the day screwing around trying to figure out why it kept complaining that it couldn't find hal.dll (when it was clearly there). Finally I yanked the SATA connection out and all was well. Seems Windows XP got confuzzled over where it was looking for files. At boot time it was talking to my IDE, but then at install time it figured my SATA drive was in charge and thus couldn't continue. Installed XP on the IDE, replugged the SATA drive back in and that was that. Sometime around Sunday night I finally finished setting up Visual Studio for the umpteenth time. You can read the brochures and rally around products like Norton Ghost and Acronis Disk Image, but moving installed programs under Windows around just never works. There are always left over droppings of registry settings, files in common places, and configurations in Documents and Settings that screw up moving anything from drive to drive.

On top of that, some kind of weird electrical oddity occured in our building at work so Monday was Snow Day for me and we were turned away at the doors (well, the few that didn't read the email on Sunday night saying the building was closed). So I tried getting some work done but just couldn't and ended up killing some brain cells with Halo 2 most of the day.

Okay, with that out of the way I did find a very cool page on Channel 9 that handles all the grunt work of CTP madness for you. You know, when you try to install a Community Technology Preview of something but end up messing up your environment so badly (thank the maker for Virtual PC) that you don't know what to do. CTP Build Finder is a slick tool that lets you select a product (from the list of all the current CTPs) and it will show you what other CTP products it's compatible with. Very cool and very useful if you're planning to try out something like Indigo and Longhorn at the same time. Check it out here.

Thanks to everyone who supported me (or felt sorry for me) and clicked on the PDC link. As far as rankings go, I'm tied in first place with 41 referrals (there are three different urls it's tracking, the original one, the original one with a "/" and the archived entry). Of course, referrals are not just the criteria for the contest as they'll be reading the blogs and deciding based on content and all that jazz. My pathetic attempts at humor will of course put me in last place here but hey, it's all fun until someone loses an eye.

I do have a dilemma with the PDC contest though. The contest ends mid August while the early bird registration ends in July. The early registration will save you $500USD for the conference, which in Canadian money is like $10,000. So if I wait for the contest and lose, I'm costing my company an additional $500USD for the registration. What's a girl to do?

Finally on a side note, there was an article on Slashdot about the rise and fall of blogs and how it's degraded into a personal rant fest and most people would like all bloggers to curl up and die. This post just contributes to that dribble so enjoy the wasted bandwidth and 2 minutes of your life you took reading this as you're not going to get it back.

This morning is special. It's special because I'm on my knees to the SharePoint blogging community (and anyone else who stumbled on this blog after googling for "contest"). The good geeks over at Channel 9 are sending some lucky nerd to this years Professional Developers Conference through a simple contest and I want to be that nerd. This blog post is my plea for you to send me to PDC. Why? Why send this lowly MVP to PDC and not say someone else who doesn't gripe about not going to TechEd every day of the event?

It's not like I'm asking to be sent to Tibet for an all-expense paid vacation as a yak herder. This years PDC is in Los Angeles. No, PDC is no TechEd. It does not have the glitz, the traffic, the smog, the shopping malls or the proximity of Disney World, not to mention a comparable number of television channels. Yet PDC is like the Willy Wonka of IT. With it's hands on experience with the next version of SharePoint (hopefully), in-depth talks on the future of portals, and long all-night drinking sessions that touch on the very nature of Information Technology. How can you not send me there, at least to live vicariously through my blog entries that I'll surely post.

Think of the precendent if you don't send Bil to PDC. The next time some Fortune 500 mega-conglomerate denies a geeks desire to seek knowledge and share wisdom, the IT organization will be left with no appeal. "Remember Bil" they'll say.

My promise to the world, should you send me to PDC, is simple.

• Daily updated photos of what's happening including anything so embarassing it will be blogged about for years to come
• At least one morning where I'm hung over and have to give a presentation or speak publically and don't come off looking like a complete idiot
• An in-depth interview in the PDC garage where you provide the questions with someone so close to SharePoint that we can only call him "Deep Throat"
• A blog-by-blog posting several times throughout the day on the magical goings on and what I step in when it comes to SharePoint, development, and IT
• A live demonstration of my Whack-A-Fitz concept
• An answer to the age old question. Boxers or Briefs?

So please, for the love of all that is managed code, send me to PDC. Click on the link below to do this. Really. Just one click. That's all.

No SharePointy stuff today as I'm just spending most of my day re-configuring our VMs so calls to SPUtility.GuessLoginNameFromEmail will work. I do have some goodies coming up this weekend as a server move is going to keep me away from the office so I'm going to be VPCing at home on non-project work and have a few things to post on my SharePoint Wrappers project which provide an OO way to access SharePoint services from remote locations (like desktops). I'll be posting a blog and source update to a new tool called Remote SharePoint Explorer (for lack of a better name) which mimics what SharePoint Explorer from Navigo does, but you can run it from your desktop (doesn't have as much functionality as SharePoint Explorer does, but gets you as close as you can via Web Services). So keep an eye out for that this weekend.

Fitz resurfaced from drinking and carrying on at TechEd, an event that I was unable to make it to (grumble, grumble, grumble, gripe, gripe, grip) and posted a few notes from the event. He's got a nice little block on Navigo Systems that have a solution for boolean type searches and, as Fitz puts it, is a much more powerful and flexible search option that can be a drop-in replacement for the standard search. Also check out SharePad and the FrontPage RPC stuff that the Interlink Group put together. It's an excellent toolkit and for those that want to build custom file handlers in their portals, it's the way to go. I took a look at it a couple of weeks ago and am just trying to see how to leverage it to do some SharePoint wizardry that we can't get out of the box. Great stuff.

The WebPartPages namespace and the various Web Parts that Microsoft has in them keeps puzzling me. While everyone is off at TechEd, some of us still have work to do (yeah, that was today's grumbling).

I had previously tried to instantiate my own copy of a ListFormWebPart (because it's sealed so I can't inherit from it) and wrapping it up in my own custom Web Part. This failed miserably so here we are again. Now I'm bugged by the XmlWebPart. It's a great Web Part for rendering Xml without having to build your own XmlDocument and do transformations and all that jazz. Just set the Xml (or XmlLink) property and optionally the Xsl (or XslLink) property and voila, insto no-nonsense Xml rendering. Trouble is that I can't seem to do this on the fly. The Xml links have to be static.

The best compromise I can find is to:

1. Grab the SPWebPartCollection from the current SPWeb (through a url to a Web Part Page I create which has an XmlWebPart on it)
2. Find the XmlWebPart on the page by walking through the SPWebPartCollection and matching up on GetType or something
3. Set the Xml (or XmlLink) property once the XmlWebPart is found
4. Redirect the user to the custom Web Part Page for happy viewing

Or maybe I'm just trying to beat a dead horse and should just do my own dang transformation? I know Microsoft has to use the WebPartPages for their own Web Parts but why is it so difficult for anyone else to just tap in and leverage them?

Man, a week without TechEd (and the bright Orlando sunshine) and it's been raining for a few days here in Cowtown. Sucks to be me.