2

Background

When adding custom development functionality to MOSS/SharePoint 2007, making changes to Web.Configs is not as straightforward as a regular ASP.NET application. The reason for this is that MOSS uses a multiple-server model, and even if you have only one server, it’s still considered a Web Farm.

So, in the same way that MOSS automatically configures IIS sites, host names, authentication, and more, MOSS also has a centralized concept for modifying Web.Config settings.

Why Web.Configs?

A SharePoint site, as glamorous as it may make itself out to be, is nothing more than an ASP.NET web site running on top of IIS. Sure, there’s tons of functionality in it, but at the end of the day, it’s just an ASP.NET web site. With that in mind, it too will have a web.config file, where certain settings can be adjusted, custom application settings can be added, and so on. To give one example, assembly bindings can be specified (and even re-mapped to alternate versions of assemblies) through web.config file to make your application work.

The Problem

Let’s say you have 5 front-end web servers that serve up your SharePoint site. That means you have 5 copies of IIS, and 5 copies of the SharePoint web application. In turn, you have 5 web.config files to keep up with. If you were to manage this manually, you would have to make sure that whenever you made a change, you manually copy web.config files to each server.

Now, let’s say you have 10 SharePoint web applications running on each of those servers. You then have 10 web.configs per server, so a total of 50 web.configs to manage across your farm.

And finally, let’s say you add a 6th server to your farm. You would have to make sure your 10 web.configs were added to the 6th server, on top of now having 60 web.configs to manage and keep in sync.

Microsoft has solved this problem with the centralized concept of web.config changes.

General  Concept

The general idea of this concept revolves around the fact that a Web.Config is nothing more than XML, and that with XML XPath support, certain blocks of HTML can be found with accuracy and reliability, and then changed.

So, if there were just one place where you could store your “changes”, the rest could be automated and kept in sync with the other servers. And when a new server was provisioned, it could automatically be sync’d with this copy of the changeset. If we only had a SharePoint concept, called, oh, I don’t know…

SPWebConfigModification

The SPWebConfigModification class in SharePoint (as of WSS version 3.0) allows you to specify changes with the following information:

  1. What “ownership” group should this settings change go under? (To group certain settings together for easy management.)
  2. What is the XML Node path you want to start at? (for example, /configuration/system.web)
  3. What is the XPath to the specific node you want to match?
  4. What is the block of XML that you wish to go under the matched node?

These changes are stored in a centralized repository per-web application, and are automatically sync’d with all SharePoint web applications in the farm at regular intervals.

Below is a screenshot of what this “central repository” looks like when viewed through an internal tool I developed for this purpose (blurred to exclude possible sensitive info):

image

How Changes are Applied

Changes are applied one of two ways:

  1. Manually (on-demand), through the SharePoint object model.
  2. Automatically, at a regular interval.

When a web.config changeset synchronization occurs, the following things happen:

  1. A timer job is created to synchronize web.config settings, specified to run on all farm servers.
  2. Within 30 seconds, each server picks up its timer job to synchronize its own web.config.
  3. Each server reads its own web.config, compares the XML nodes with the central modification repository entries, and ensures that all values match up. If a repository entry doesn’t exist in the web.config, it will be created, and if it does exist in the web.config with a different value, the web.config value will be updated.
  4. Depending on what was changed, this can cause a recycle of the SharePoint web application.

WARNING: SharePoint, by default, schedules this to happen automatically at regular intervals (at least once a day). This is why it is important NOT to make “manual” changes to web.config files (i.e. by editing them in a text editor). If your manual changes happen to match a repository entry’s XPath, your manual change will be overwritten as soon as this synchronization occurs.

Additionally, if the XPath is only ‘halfway’ correct, it can cause multiple entries of the same node. For an example of this, please read the previous post MOSS Web.Config Modification – Beware the XPath.

Managing Web.Config Changes for Structured Deployments

Unfortunately, there is not a good way within SharePoint’s out of the box Solution Deployment to handle web.config change deployments. As far as I know, you will need to roll your own. However, the team working on this particular project came up with a pretty clever way of managing these changes, and pushing them out in a structured manner.

First, a XML file that is more or less referred to as a web.config Changeset File:

<?xml version="1.0" encoding="utf-8" ?>
<WebConfigChanges xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <!-- This change set is for unit test web applications -->
  <ChangeSet Name="UnitTesting" Status="Active" CanUpdate="True">
    <ChangeItem Name="trust" Path="configuration/system.web" Sequence="0" Environment="Development">
      <trust level="Full" originUrl="" />
    </ChangeItem>
  </ChangeSet>
 
  <!-- This change set is for development web applications -->
  <ChangeSet Name="Development" Status="Active" CanUpdate="True">
    <ChangeItem Name="compilation" Path="configuration/system.web" Sequence="2" Environment="Development">
      <compilation batch="false" debug="true">
        <assemblies>
          <add assembly="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
        </assemblies>
        <expressionBuilders>
          <remove expressionPrefix="Resources" />
          <add expressionPrefix="Resources" type="Microsoft.SharePoint.SPResourceExpressionBuilder, Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
          <add expressionPrefix="SPHtmlEncodedResources" type="Microsoft.SharePoint.SPHtmlEncodedResourceExpressionBuilder, Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
          <add expressionPrefix="SPSimpleFormattingEncodedResources" type="Microsoft.SharePoint.SPSimpleFormattingEncodedResourceExpressionBuilder, Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
          <add expressionPrefix="SPUrl" type="Microsoft.SharePoint.Publishing.WebControls.SPUrlExpressionBuilder, Microsoft.SharePoint.Publishing, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
        </expressionBuilders>
      </compilation>
    </ChangeItem>
  </ChangeSet>
 
  <!-- Used to configure error log page-->
  <ChangeSet Name="ErrorLogging" Status="Active" CanUpdate="True">
    <ChangeItem Name="SafeMode" Path="configuration/SharePoint" Sequence="0">
      <SafeMode MaxControls="300" CallStack="true" DirectFileDependencies="10" TotalFileDependencies="50" AllowPageLevelTrace="false">
        <PageParserPaths>
        </PageParserPaths>
      </SafeMode>
    </ChangeItem>
    <ChangeItem Name="customErrors" Path="configuration/system.web" Sequence="1">
      <customErrors mode="On" defaultRedirect="~/_layouts/error.aspx" />
    </ChangeItem>
  </ChangeSet>
</WebConfigChanges>

This file encapsulates what will go into a SPWebModification repository entry, and specifies a couple critical things:

  1. The changeset name (translated into the “Owner” in SPWebConfigModification)
  2. The environment that should be applied to each item (this provides a way to specify different settings for Development, Staging, Production, All, etc, for example, connection strings).
  3. The XPath and Sequence of the modification entry, directly translated into the SPWebModification entry.

A straightforward deployment tool parses this XML, takes a Web Application as an argument, identifies the current environment (via a MOSS Farm-level Property Bag setting), and translates relevant entries into SPWebModification entries for the specified web application.

This provides a structured way to deploy and manage web.config changes, specific to the environment. Although you can still get in a great deal of trouble if XPaths are not absolutely perfect, this provides a great way to eliminate human error to deploy web.config changes in a similar fashion as normal MOSS .WSP package deployments.

1

(This is a follow-up to the previous post: Propagating Content Type Changes in MOSS 2007)

So far, I have propagated a few Content Type fields in a live production environment of ~65,000 webs using this method. Mind you, this does not mean that I have propagated a content type to 65,000 lists or items, just that the environment is large.

The whole game here revolves around Field Links (see the SPFieldLink class on MSDN). And to recap, the goal here is to be able to maintain your Content Types in in a Solution (WSP) feature. MOSS does not support updating content types "declaratively via XML" (i.e. through a solution package).

Here's what MOSS will do: In most cases, MOSS will add new fields to the root / site collection content type (make sure you do an RDAD deployment - Retract / Delete / Add / Deploy). What this actually means (and this is important) - is that MOSS will add the field and create a SPFieldLink which links the field to the root content type. This is why when you create a new list item based on the content type, that it will show up, but there are no field links automatically created for existing items.

So here's the work we need to do to propagate the field:

  1. If working with a large site collection (more than 1,000 webs), determine the usage count of the content type you're updating. This will give you an approximate idea of how long this is going to take to propagate (more on this below).
  2. Break the SPFieldLink at the root level, so that the field is no longer linked to the content type, and update the content type.
  3. Add the SPFieldLink to the content type, but this time, call .Update(true) on the content type, so that SharePoint will propagate the changes to all inherited items (including lists items).

Let's step through these...

1. Determine Usage Count - This determines where the content type is actually used, and therefore, the # of updates that will take place. A content type usage is defined as the # of items that actually use / inherit the content type.

This is simple - if you already have a SPWeb object called "web" and the Content Type name string called "contentTypeName", just use:

int contentTypeUsages = Microsoft.SharePoint.SPContentTypeUsage.GetUsages(web.ContentTypes[contentTypeName]).Count;

In testing so far, here are my performance findings as to how long it will take to propagate. Bear in mind that for smaller usage counts there is still some overhead time in establishing a context, retrieving web, etc. However, the results are not very linear, and I couldn't tell you why. This will just give you a very general and vague idea to set your expectations.

  • 1 usage - 0.66 seconds per web (Total Time: 28 seconds)
  • 6 usages - 0.01 seconds per web (Total Time: 19 seconds)
  • 1,876 usages - 0.64 seconds per web (Total Time: 1192 seconds / 19 min, 52 sec)
  • 13,138 usages - 0.53 seconds per web (Total Time: 1003 seconds / 16 min, 43 sec)
  • 15,031 usages - 0.66 seconds per web (Total Time: 1234 seconds / 20 min, 34 sec)

2. Break the SPFieldLink at the root level - If you already see the field when you go to Site Collection Settings > Content Types, then you will need to break the SPFieldLink. Find the field link by looking in the SPContentType.FieldLinks collection (in the below example, this is the selectedFieldLink object, and selectedSiteColl is the SPSite site collection object).

SPFieldCollection fieldColl = selectedSiteColl.RootWeb.Fields;

this.selectedContentType.FieldLinks.Delete(selectedFieldLink.Id);
this.selectedContentType.Update(true);

fieldColl.Delete(selectedFieldLink.Name);
selectedSiteColl.RootWeb.Update();

3. Propagate the field - Example below - In this case I have created a FieldLinkInstance class that just holds some properties like the Site Collection, Content Type name, etc, but you should be able to see the general idea here. The most critical thing is the last line where you call c.Update(true).

public void PropagateFieldToSiteCollection(FieldLinkInstance fieldLink)
{
SPContentTypeCollection contTypes = fieldLink.SiteCollection.RootWeb.ContentTypes;
SPFieldCollection fieldColl = fieldLink.SiteCollection.RootWeb.Fields;
SPContentType c = contTypes[fieldLink.FieldLink.ContentType.ContentTypeName];
SPField newField = new SPField(fieldColl, fieldLink.FieldLink.Field.FieldType, fieldLink.FieldLink.Field.Name);
newField.Title = fieldLink.FieldLink.Field.DisplayName;
newField.StaticName = fieldLink.FieldLink.Field.StaticName;
newField.PushChangesToLists = true;

string strNewColumn = fieldColl.Add(newField);
SPField targetField = fieldColl.GetFieldByInternalName(strNewColumn);

SPFieldLink oFieldLink = new SPFieldLink(targetField);
c.FieldLinks.Add(oFieldLink);

c.Update(true);
}

Hopefully this will give you an idea and a little code to propagate content types using this method. I have created a class library that actually parses WSP solution packages to auto-detect unpropagated content types between a Solution and live MOSS farm and propagate them, but can't share the source code for that at the moment. In general, you can extract the WSP package as a CAB, then parse the XML files to extract the Content Types.

4

Today I finally got around to diagnosing an interesting and simple (but critical) problem with web.config settings. Here's the scenario:

Every time web.config modifications are done, multiple entries get added to the web.config file. In this case, to retrieve data from a web service, the MOSS farm uses a <proxy> entry to specify the outbound proxy server for the application.

However, every time SPWeb.ApplyWebConfigModifications() was called, an additional <proxy> entry is added to the web.config file. You can't have more than one proxy entry, and therefore causes these components (or the entire requests) to fail because of an invalid web.config.

(For more information about Web.Config Modifications in MOSS - http://sharepointsolutions.blogspot.com/2006/12/using-spwebconfigmodificat_116736917110571614.html)

Well, the MOSS web.config modification entries looked something like this:

Owner: ProxySettings
Name: proxy[@autoDetect='true']
Value: <proxy proxyaddress="http://someproxy:8080" />

The problem lies in the fact that MOSS does not do any validation to make sure that the value you're putting in matches the name's XML Xpath. So, MOSS says "OK, got a new modification for me? Alright... I'll put it in. No problem." and it works fine.

The next time SPWeb.ApplyWebConfigModifications() is called, MOSS does the following:

  1. Looks at the existing web.config file, and makes an XML DOM inventory of what is already in it.
  2. Enumerates through the Web.Config Modifications (which, by the way, are persistently stored in the SharepointConfig database) that should be applied.
  3. Gets to this proxy modification entry, and tries to match the name - proxy[@autoDetect='true'] - which in XML-ese is slang for: "Let me see if I already have a tag (element) that starts with <proxy and has an attribute in it of autoDetect='true'.

    So, if our modification value looked like this: <proxy autoDetect='true' /> then MOSS would have found this entry and not added a new one. The problem is, our value has no autoDetect='true' attribute inside it, so every time MOSS tries to match the value, it never will match.

  4. Match fails, so MOSS assumes that the entry doesn't exist yet, and adds another one.
  5. MOSS site goes down in flames due to multiple proxy entries, users scream, riots happen, etc.

Really, the setting should have looked like this:

Owner: ProxySettings
Name: proxy
Value: <proxy proxyaddress="http://someproxy:8080" />

That way, any element/tag starting with <proxy... would be matched and correctly handled.

Unfortunately, even before or after this is corrected, there is one more important thing: The original never matched, so it's now orphaned and MOSS will not remove it. Once the web.config modifications are removed from the persisted MOSS object model, the entries are still there in the web.config. The proper way to do this is as follows:

  1. Delete the SPWebConfigModification entry in the object model.
  2. Apply web config modification settings with SPWeb.ApplyWebConfigModifications().
  3. Manually remove all entries (in this case, all entries starting with <proxy...) from the web.config file and save.
  4. Add the correct SPWebConfigModification entry and apply modifications again.

Now, the web.config modifications should be matched by MOSS.