Working with CAML.Net – Part 1

Well, there’s a bit of a buzz growing around this CAML.Net idea, so I thought it might help to dive a little deeper into the technique to show what can be done.  It’s really kind of interesting what happens when you go from a purely xml-based language to an object-oriented one.

This post introduces the notion of reusable CAML queries and leverages the power of inheritance to build a library of custom queries.  Then we extend the concept a little further to handle automatic data-binding of query results to arbitrary C# classes.

Note: To use the techniques described in this article, you’ll need to download the latest beta version (0.2.0.0) of the JohnHolliday.Caml.Net assembly from CodePlex.

First, let’s review the basic concept.  Caml.Net provides a static wrapper for the CAML query schema, which adds strong data types and variable substitution for Caml queries.  This produces an XML fragment that you can pass directly to any API method that accepts a CAML query string.  For example, you can get a data table containing all list items that match a given content type by setting the query string into a SPSiteDataQuery object like so:

   string typeName = "My Content Type";
   string queryByContentType =
       CAML.Where(CAML.Eq(CAML.FieldRef("ContentType"), CAML.Value(typeName)));

   SPWeb web = SPContext.Current.Web;
   SPSiteDataQuery query = new SPSiteDataQuery();
   query.Query = queryByContentType;
   DataTable table = web.GetSiteData(query);

Creating Reusable CAML Queries

This is fine for one-off queries, but what we really want is the ability to define a query once and reuse it many times, ultimately creating a library of compiled queries that we don’t have to rebuild.  To enable this capability, I’ve added a CamlQuery object to the Caml.Net assembly that provides two essential services.  First, it allows you to instantiate the query object separately from the underlying query string.  This means we can add persistence and other non-string operations.  Second, it provides default implementations for most of the query execution scenarios you would otherwise have to write yourself.

Here is the preliminary interface definition:

    public interface ICamlQuery
    {
        /// <summary>
        /// Gets or sets the CAML query string.
        /// </summary>
        string QueryXml { get; set; }
        /// <summary>
        /// Gets or sets the CAML Lists specifier.
        /// </summary>
        string ListsXml { get; set; }
        /// <summary>
        /// Gets or sets the CAML ViewFields specifier.
        /// </summary>
        string ViewFieldsXml { get; set; }
        /// <summary>
        /// Retrieves matching list items from all lists in all webs of a site collection.
        /// </summary>
        IList<SPListItem> Fetch(SPSite site);
        /// <summary>
        /// Retrieves matching list items from within a single web.
        /// </summary>
        IList<SPListItem> Fetch(SPWeb web);
        /// <summary>
        /// Retrieves matching list items from within a specific query scope.
        /// </summary>
        IList<SPListItem> Fetch(SPWeb web, CAML.QueryScope scope);
        /// <summary>
        /// Retrieves matching list items from a single list.
        /// </summary>
        IList<SPListItem> Fetch(SPList list);
        /// <summary>
        /// Retrieves matching list items and binds them to objects of any type.
        /// </summary>
        IList<T> Fetch<T>(SPSite site);
        IList<T> Fetch<T>(SPWeb web);
        IList<T> Fetch<T>(SPWeb web, CAML.QueryScope scope) where T : new();
        IList<T> Fetch<T>(SPList list);
    }

The ICamlQuery interface includes several overloads of the Fetch method and the CamlQuery class is smart enough to determine the appropriate query scope based on the parameters passed to it.  To get only those items found in a specific web, you can pass an SPWeb object.  To get all list items anywhere within the site collection, simply pass in an SPSite object instead.  Similarly, to limit the search to a single list, pass an SPList object, etc.  You can also specify a particular scope by using the strongly typed CAML.QueryScope enumeration.

With these methods in place, we can now extend the default CamlQuery implementation to create a reusable query that will retrieve list items of a given content type.   Notice that we simply pass in the content type name as an argument to the constructor.  The base class accepts a simple XML string as its argument, which we construct using the static methods provided by the CAML class.

    /// <summary>
    /// A reusable CAML query that performs auto-substitution of the
    /// supplied content type name to find matching list items.
    /// </summary>
    public class QueryByContentType : CamlQuery
    {
        public QueryByContentType(string typeName) : base(
            CAML.Where(CAML.Eq(CAML.FieldRef("ContentType"),CAML.Value(typeName)))
            )
        {
        }
    }

Now we can compile the object into an assembly and use it whenever we need to work with objects of a given type.   For instance, say we have a content type named “News Article” in a publishing website, and we have some items in one or more lists.  We need to write some code that manipulates the list items.  In this example, we’ll just display the article title to the console.

Using the default implementation provided by the CamlQuery base class and our custom QueryByContentType derived class, we can easily retrieve a collection of news articles, given an SPWeb object.  First, we’ll display the raw XML just to see what the query looks like before it is processed.  Then we’ll get a generic collection of SPListItem objects and display the title of each one.

    // Use the QueryByContentType class to retrieve a collection of News Articles

    using (SPSite publishingSite = new SPSite("http://moss"))
    {
        using (SPWeb publishingWeb = publishingSite.OpenWeb())
        {
            QueryByContentType qt = new QueryByContentType("News Article");
            Console.WriteLine(crlf + "Query list items by content type (raw CAML):");
            Console.WriteLine(crlf + qt.QueryXml);

            IList<SPListItem> items = qt.Fetch(publishingWeb);
            Console.WriteLine(crlf + "News Articles in Publishing Web:");
            foreach (SPListItem item in items)
                Console.WriteLine("{0}", item.Title);
        }
    }

Automatic Data Binding

So far, so good.  We can get generic collections of SPListItem objects.  But wouldn’t it be nice if we could also translate the resulting list items into arbitrary data structures automatically?  Then we would be able to model our solutions more intuitively.  Fortunately, with a little extra effort, the .Net Framework gives us everything we need.  We’ll use a generic method to accommodate any kind of data structure, and we’ll use reflection to bind the list items to the individual fields.  Notice the following set of method declarations in the ICamlQuery interface:

        /// <summary>
        /// Retrieves matching list items and binds them to objects of any type.
        /// </summary>
        IList<T> Fetch<T>(SPSite site) where T : new();
        IList<T> Fetch<T>(SPWeb web) where T : new();
        IList<T> Fetch<T>(SPWeb web, CAML.QueryScope scope) where T : new();
        IList<T> Fetch<T>(SPList list) where T : new();

This is where the power of C# really comes to our aid. With these generic methods, we can write truly reusable code, but we need some additional tools to handle the data binding.  That’s where reflection comes in.  For this to work, we need to markup the fields of a class with the information needed to locate the corresponding metadata in a list item.  In our example, we want to declare a NewsArticle class and then mark it up with the appropriate field metadata, as follows:

        /// <summary>
        /// A custom class that represents a news article.
        /// </summary>
        public class NewsArticle
        {
            [CamlField("Title")]
            private string ArticleTitle=string.Empty;

            [CamlField("Article Date")]
            private DateTime ArticleDate=DateTime.Now;

            [CamlField("Content")]
            private String ArticleContent=string.Empty;

            /// <summary>
            /// Default constructor needed for reflection.
            /// </summary>
            public NewsArticle() { }

            /// <summary>
            /// Display the article title and date.
            /// </summary>
            /// <param name="writer"></param>
            public void Dump(TextWriter writer)
            {
                writer.WriteLine("Title: {0}rnDate: {1}rn",
                    this.ArticleTitle, this.ArticleDate.ToLongDateString());
            }
        }

Now, we have everything in place.  Let’s write a simple program that illustrates all the concepts we’ve covered.

        static void Main(string[] args)
        {
            // Define the query.
            QueryByContentType qt = new QueryByContentType("News Article");

            // Display the raw CAML just to see what it looks like.
            Console.WriteLine("rnQuery list items by content type (raw CAML):rn");
            Console.WriteLine(qt.QueryXml);

            // Open a SharePoint site.
            using (SPSite site = new SPSite("http://moss"))
            {
                // Start at the top level web.
                using (SPWeb web = site.OpenWeb())
                {
                    // Display articles as SPListItem instances.
                    ListNewsArticleItems(qt, web);

                    // Display articles as NewsArticle instances.
                    ListNewsArticleObjects(qt, web);
                }
            }

            // Pause for effect.
            Console.Write("rnPress any key...");
            Console.Read();
        }

        // Fetch news articles from a specific web site.
        private static void ListNewsArticleItems(QueryByContentType query, SPWeb web)
        {
            Console.WriteLine("rnNews Articles in {0}:rn",web.Url);
            IList<SPListItem> articles = query.Fetch(web, CAML.QueryScope.WebOnly);
            foreach (SPListItem item in articles)
                Console.WriteLine("{0}", item.Title);
        }

        // Fetch news articles as NewsArticle objects using automatic data-binding.
        private static void ListNewsArticleObjects(QueryByContentType query, SPWeb web)
        {
            Console.WriteLine("rnUsing automatic data-binding:rn");
            IList<NewsArticle> articles = query.Fetch<NewsArticle>(web, CAML.QueryScope.SiteCollection);
            foreach (NewsArticle article in articles)
                article.Dump(Console.Out);
        }

And the final result:

Summary

The power of CAML lies in its flexibility, but that flexibility comes at a cost.  Typically, the cost is measured in lost productivity because it forces developers to work without the tools they need to ensure their code meets solution requirements.  It also makes it difficult to effectively extend the efforts of others through the encapsulation and data abstraction that would be possible with a stronly-typed managed language like C#.

CAML.NET addresses this problem by providing a rich set of static methods for easily expressing CAML queries and a corresponding set of wrapper classes for executing queries and then binding the results to any .Net object or data structure.  Using CAML.NET, developers can build reusable query libraries and derive new queries from existing ones, adding value to their teams with each new assignment.

In other words, they can work faster.  They can work better.  They can work like (say it with me) – dromedaries!

The latest beta version of the JohnHolliday.Caml.Net assembly is available for immediate download from http://www.codeplex.com/camldotnet/Release/ProjectReleases.aspx?ReleaseId=3975.

Please Note: The software is provided under the Microsoft Community License (Ms-CL).  This is a change from the Creative Commons License which was applied to the previous version.  Source code is not available at this time.

Enjoy.  And stay tuned.