Check out this postby Lee Dallason why he thinks Microsoft has been so successful in redefining ECM around the capabilities of the SharePoint platform.  To put it simply, it's because of a brilliant marketing strategy and not because of SharePoint itself.

Dallas boils it down to three key points:

1. Microsoft focused on solving the most pervasive problem first: self-management of workgroup file sharing (off-loading the burden from IT).
2. Microsoft setoff a viral promotion campaign by letting end-users believe they could control their own data.
3. Microsoft marketed the product through IT operations channels.

Dallas observes that because SharePoint deployments are so pervasive and are expanding so rapidly, the entire ECM food chain has been thrown out of balance to the point that traditional ECM vendors must now justify their products against mundane collaboration features instead of the more specialized features they tend to promote.  But perhaps the most interesting observation is revealed by the following statement:

"There is no single department in any IT organization with greater potential reach than those that manage access and identity (i.e. Active Directory)."

Dallas argues that by marketing and promoting SharePoint through IT operations channels, Microsoft essentially achieved preferential status for SharePoint, even though the technical challenges in infrastructure and security were no less prevalent than with other products. In other words, the people evaluating SharePoint and who would ultimately bless its deployment had already invested their time and energy learning how to resolve the problems they would face after the product was put into production.  Brilliant!

All in all, a good read for anyone interested in ECM.

Mattias Wollnik from Microsoft has created a nifty PowerShell script that makes it much easier to move content from file shares into SharePoint 2007 and SharePoint 2010 beta sites.   The FCI SharePoint Uploadscript is based on the File Classification Infrastructure(FCI) provided by Windows Server 2008 R2.  The FCI allows administrators to setup policies that control the metadata associated with individual files.  Using Mattias’ script, you can upload files and it will automatically set any matching metadata properties that are present in the target document library.

This is similar to the way that properties are promoted for Office 2007 documents, except that the script works for any file type.  The File Classification Infrastructure provides a snap-in that lets you setup custom file management tasks.  To use the script, you setup a new task that invokes PowerShell with the appropriate parameters.

For example, to upload files to a document library called “Company Files” in the Litware demo portal, you would specify an action type of “Custom” with the executable set to “C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe” and the arguments set to the following:

-noninteractive –file c:\scripts\FciSharePointUpload.ps1 –file “[Source File Path]” –url “http://litwareinc.com” –libPath “Company Files” –sourceAction url –user litwareinc\administrator –password pass@word1

The “[Source File Path]” string tells FCI to substitute the actual file path when the source files are processed.  One cool feature is the ability to upload files directly to a SharePoint 2010 Records Center site, specifying the target content type on the command line.  Using this feature, if any Content Organizer rules exist for the specified content type, they will be applied and processed automatically.

To do this, simply include the –useOfficialFileWebService parameter and then specify the content type using the –contentType parameter instead of -libPath, as in the following example:

-noninteractive –file c:\scripts\FciSharePointUpload.ps1 –useOfficialFileWebService –file “[Source File Path]” –url “http://litwareinc.com” –contentType “My Content Type” –sourceAction url –user litwareinc\administrator –password pass@word1

To learn more about the script with examples and a complete source code listing, visit http://gallery.technet.microsoft.com/ScriptCenter/en-us/f538c34c-4f74-4645-9649-fd25e49805d6.

It's kind of surreal, but the day has finally arrived when I can say "it's done"! My new book, Professional SharePoint 2007 Records Management Development:
Managing Official Records with Microsoft Office SharePoint Server 2007 is on its way to store shelves and is available directly from Wiley.com, Amazon.com and other online retailers.

  This book is for developers who need to build records management solutions, and who need to understand the ins and outs of the records management framework that is built into the SharePoint platform.  For a more complete synopsis and to review the TOC, just click the image to the left to visit the Wiley information page.

One thing you'll notice is that Wrox is now using a new cover format.  If you expected to see my face on the cover, you won't.  Those shrewd marketing folks at Wiley decided to give the entire Wrox line a "makeover".  I have to say I was a little disappointed at first, but I actually like the new cover design.  Who am I to argue with the marketing guys, right?

Once you get the book, you’ll also notice that I took a slightly different approach than some of the other development books out there, in that I tried to structure the chapters so that the code in later chapters builds on and extends the code presented early in the book.  On the one hand, it adds consistency because you end up with a set of reusable components that can be applied to many different types of solutions.  On the other hand, it means that in order to understand and use some of the later examples, you have to dig into the earlier ones.  I know this may be a bit frustrating for some readers, but I’m hoping that the end result justifies the approach.  I’d be very interested in getting your feedback, so don’t be shy.

On the up side, there is a LOT of code included with the book, and the kicker is it’s ALL available online NOW.  That’s right, you can download the HUGE set of code examples directly from Wrox RIGHT NOW by just clicking on the “downloads” tab on the book’s information page on the Wiley site.

As always, I'm here to serve, and I'm especially eager to hear what you think.  Wiley has setup a forum just for this book at the Wrox Programmer to Programmer site.  Just log in to the P2P site and search on my name and you’ll find the forum.

Thanks to everyone who helped make this book a reality, especially Eli Robillard, Dan Attis, Aaron Cutlip, Todd Meister, Stacy Draper and all the folks at Wiley.  It was a long and arduous road, but it was also a wonderfully enlightening and strengthening experience thanks to everyone involved.

-John

E=MC^2 is probably the most famous equation in modern science, but little is known about its true origins. At least I didn’t know much about it until very recently. Of course, I understood, like everyone else, that it was formulated as part of professor Einstein's General Theory of Relativity, but I never really thought about it much beyond what we learned in school. It was not until I started digging into my own personal genealogy that I stumbled across something that shed some light on what had remained, until now, a mystery.

I had often wondered why Professor Einstein chose to express his now famous equation using those particular letters, E=MC^2.  Was it just pure coincidence that we’re now using those very same letters to describe a business problem that threatens the very fabric of modern-day society? Indeed, where would we be today without some form of Enterprise Content Management (ECM) system?

Following the war, there wasn’t a lot of work for singers in Harlem, so my distant cousin, Billie (on my father's side, four times removed), would supplement her income by filling in as a part-time assistant for the professor at his office in Princeton. While rummaging around in the attic of our old house, I recently came across a box of her personal items. In it, there was a diary that she kept during the years between 1947 and 1952, after her divorce from Jimmy Monroe.

With the cold war still in full swing, it seems that the professor was literally swamped with letters and packages arriving daily from nearly every corner of the globe containing white papers and pamphlets on a myriad of subjects ranging from Newtonian physics to quantum field theory. Keeping track of all that content often proved too much for the professor as he became increasingly frustrated while developing his unified field theory. In one entry, my cousin writes:

“E. was really upset today. The poor man just can’t keep track of all the information that keeps streaming into the office. I try my best to organize it all, but there’s just too much of it.”

More telling is the entry from March 15^th, 1951 - the day after the professor's 72nd birthday.

“I overheard E. shouting at one of his students just as I was coming into the office today. ‘Why so difficult to separate ze metadata from ze content?! Teilen-Punkt! Teilen-Punkt! If only we had ein Teilen-Punkt!’ I wish I knew what he was talking about, so I could help him find it.  He really is starting to worry me.“

I found this entry particularly intriguing; because it seems to capture the essence of the same problem many of us are facing today. Managing enterprise content is quickly becoming one of the most important fields of study in the information age – especially the problem of dealing with metadata. Could it be that Einstein perceived a solution to this problem way back in the 1950’s?

I kept digging through the box and came across lots of interesting stuff. There were photos of Billie on the road with Duke. Billie back-stage with a guy that looks like Miles and sharing a laugh with other hard-core musician types. But the real shocker was a small photo that was stuck to the side of the box with a piece of masking tape.  I almost missed it.  On the back there was one legible sentence scribbled in pencil along the edge so faint I could barely make it out. “Don’t ignore ze metadata. It is ze content. Without it, zer can be no Teilen-Punkt.”  I was still trying to comprehend what I was reading, and then I turned the picture over.  There, before my eyes, was the answer - scrawled on the chalkboard right behind him.

Was ECM part of Einstein’s grand vision of the unified field? Is metadata somehow connected with the theory of relativity? Are you buying any of this?  We know now that content management is at the heart all business processes.  Is it possible that the father of modern scientific thought was also focused squarely on solving this most perplexing problem of modern day society – namely, how do we deal with content overload without interfering with day-to-day productivity?

One can only imagine what might have happened had SharePoint been available in Einstein’s day. Teilen-Punkt.  Alas, all we can do now is speculate, and pay tribute to a wonderfully creative and brilliant mind. A true visionary – in many ways, the father of ECM.

All content is relative.  Teilen-Punkt, indeed!

Enterprise Content Management - Document Retention

Document retention is an important part of content management.  SharePoint provides out-of-the-box support for managing document retention using the built-in expiration policy feature of the information management policy framework.  The built-in expiration policy feature includes the ability to define expiration formulas based on document metadata, but there are many scenarios in which the expiration date depends on conditions external to a given document.  In this session, we'll explore the information management policy architecture in detail and learn how to extend the expiration policy feature by writing custom document expiration formulas that calculate the expiration date based on data pulled from elsewhere in the SharePoint farm.

Speaker: John Holliday, MVP Office SharePoint Server

There is a new standards effort in the works that promises to unify the disparate ECM platforms currently available into a common set of interfaces.  The Content Management Interoperability Services (CMIS) specification is the result of a collaboration between Microsoft, IBM, EMC, Alfresco, OpenText, SAP and Oracle.  Click here to download a preview copy of the spec.

First impressions:

The specification tries to define the "core" components of any ECM system.  On the one hand, this is good because we at least need to agree on common terms.  For example, the whole "content type" versus "object type" debate is important to get resolved.  On the other hand, with such a young paradigm, it's not yet clear what functionality will be the winning market differentiator for any given vendor, thus thwarting further attempts to refine the specification.  To that end, the group has submitted the draft specification to the OASIS consortium so that others can join the effort.  This is a good move and will hopefully lead to faster refinement.

1. The primary goal of the spec at this stage seems to be to provide a common set of SOA endpoints for the different ECM offerings from the vendors involved.  There are SOAP and REST/Atom protocol bindings for its proposed "domain model", each of which are explicitly required for conformance with the standard.
2. The spec does not cover administration, configuration or security, and includes within its definition of "administration" the modification of Object Types.  This is a bit of a disappointment, but not surprising given the different interpretations of that term.  Yet, the explicit exclusion of such a fundamental set of methods again reflects the newness of the paradigm and the difficulty of defining a common object model.
3. The data model focuses on the ability to create and access a repository, but explicitly excludes anything defined as "transient", which includes such important concepts as compound document, workflow, event and subscription.  Here again, we see what appears to be disagreement on first principles.  The result is that the data model describes only low-level components, such as object, property, document, folder, relationship and policy.
4. CMIS supports versions for document objects, and also includes a standardized query model based on a subset of SQL.  This is very nice, because it means that clients will be able to submit SQL SELECT statements to retrieve objects from multiple ECM vendors.  This will include searches based on object properties and folder membership as well as full text searches.

The spec at this point is pretty low-level, similar to the Open XML Formats packaging API, but offers a strong first step toward cooperation among the ECM vendors in providing a common set of tools for accessing disparate content repositories. 

The query specification is interesting, and includes a few sample queries that I'm anxious to dig into.  There are also some good examples of how versioning, checkin and checkout will work.  Hopefully, I'll get a chance to go into these in some detail in the coming weeks.

Stay tuned.

When it comes to content management in SharePoint, half the battle is figuring out the best angle of attack.  Why?  Well, because there are so many options and combinations of options that come into play when designing a solution.

Take the SharePoint permissions architecture, for example.  There are 33 different permission masks divided into 3 categories: personal, list and site permissions.  These are combined into role definitions (permission levels) which may be assigned to users and groups and then associated with sites, lists and list items.   Often, you need to create a custom role definition for a particular user or group.  When doing so, it is important to understand permission dependencies, because that will determine the effective permissions being granted.

With the exception of the "Open" permission, which grants the ability to open Web sites, lists and folders to access their contents, every SharePoint permission depends on one or more of the other permissions.  This means, for example, that if you grant the "Manage Alerts" site permission, you are also automatically granting the "Create Item Alerts", "View List Items", and "Open List Items" list permissions as well as the "View Site Pages" and "Open" site permissions.

If you're like me, with so much information to digest and process, and so little time to do it in, a simple diagram can go a long way towards sharpening your focus.  To that end, here is a little chart I created for my ECM401 course (all praise to the Visio gods!) that you may find useful.

Enjoy.

Let's face it, SharePoint is a huge development platform no matter what angle you approach it from.   As most of you know, I've spent the past year or so traveling and teaching SharePoint developer courses for the Ted Pattison Group, and my focus has been pretty broad, covering topics ranging from features to content types to workflow and everything in between. 

Now, as the platform continues to mature and as developers become more and more familiar with the basics of SharePoint development, it is increasingly evident that more specialized training is needed, particularly in the area of Enterprise Content Management (ECM).  To that end, I'm pleased to announce the immediate availability of two new courses I've developed that are being offered through the Ted Pattison Group:

• ECM401 - Enterprise Content Management with SharePoint Server 2007 (hands-on)
• WC-ECM401 - Enterprise Content Management with SharePoint Server 2007 (online)

Both versions of the course are available for immediate registration.  The online version is a great option for those unable to attend the hands-on course.

• The hands-on version includes 12 modules instead of 10 with a deeper treatment of content modeling and using workflow to drive ECM solutions.
• The hands-on course is 4 full days with lectures, demos and labs.  The online version is 5 days with about 3 hours of lectures and demos each day plus labs assigned as homework.
• The online version is conducted via Live Meeting and students may submit questions during the lecture which are answered at the end of each session.  Questions may also be submitted directly to the instructor via email or live chat.
• Students attending the hands-on course receive a student workbook with slides and labs.  Online students receive the workbook in electronic format.

Feel free to contact me directly with any questions you may have about either version of the course.  For additional details and to review the course outline, please visit the Ted Pattison Group website using the following links.

Here is the upcoming course schedule, with links to the registration form.

Well, we're just a couple weeks away from TechEd 2008.  This year, I'm giving four presentations - three breakout sessions and one TLC session, so it should be a lot of fun.  Here are the dates and times:

• OFC10 - TLC 6/3/2008 3:00PM - 4:15PM
  Building Custom Routers for Records Management in Microsoft Office SharePoint Server 2007

  This talk discusses developer aspects of records management in Microsoft Office SharePoint Server 2007. Custom routers are an important extensibility point for records management and this talk details the requirements for these components. It provides a demo of building a custom router, including deploying it to a SharePoint Server environment.
  

• OFC318 6/4/2008 8:30AM - 9:45AM
  XOML, XOML, Everywhere: Building Custom Workflow Editors for SharePoint
  

  XOML is the markup language behind Windows Workflow Foundation and is the same language used by Microsoft Office SharePoint Designer 2007 to attach declarative, no-code workflows to SharePoint lists and document libraries. This session takes a deep dive into Workflow Markup and explores how you can use it to generate custom workflows from any client application, including Office clients like Word and Excel. The session also shows how to attach custom workflow markup to SharePoint lists using the websvcWebPartPages Web service provided by Windows SharePoint Services.
  

• OFC309 6/4/2008 4:30PM - 5:45PM
  Building Document Management Solutions Using SharePoint Content Types
  

  Windows SharePoint Services 3.0 introduces a new concept called "Content Types" that will dramatically alter the Enterprise Content Management landscape. On the surface, Content Types make it easier to encapsulate field definitions into a nice neat package, but there is a lot more going on under the covers. In this session, we take a deep dive into the world of content types and see how to declare them using XML and how to create them programmatically. Then we use content types to create a real-world document management solution with custom policies that we can use to control every stage of the document lifecycle. At the end of this session, you will have a thorough understanding of what content types are, with a deeper appreciation of the value they bring to document management solution development.
  

• OFC404 6/5/2008 10:15AM - 11:30AM
  Turning SharePoint Data into Microsoft Office Documents: A Deep Dive into SharePoint Document Assembly Using Open XML
  

  Windows SharePoint Services 3.0 introduces a new concept called "Content Types" that will dramatically alter the Enterprise Content Management landscape. On the surface, Content Types make it easier to encapsulate field definitions into a nice neat package, but there is a lot more going on under the covers. In this session, we take a deep dive into the world of content types and see how to declare them using XML and how to create them programmatically. Then we use content types to create a real-world document management solution with custom policies that we can use to control every stage of the document lifecycle. At the end of this session, you will have a thorough understanding of what content types are, with a deeper appreciation of the value they bring to document management solution development.

See you in Orlando!

As the new year approaches... Wait! Is it 2008 yet? So much is happening - so fast! Do you ever get that feeling that you'll never catch up? I sure do. Time flies...

For the past few weeks, I've been busy writing chapters for a new book (more on that later), developing and experimenting with new toys and project ideas (lots to talk about) and getting ready for my ODC2008 sessions.

I'll be delivering three sessions this year. One in Karthik Ravindran's architecture track, one in Steve Fox's client track and one in Jerome Thiebaud's server track.

• ARC303 - Building Collaborative BPM Solutions

This session will cover best practices for leveraging the Microsoft BPM platform offerings (Windows Workflow Foundation, BizTalk, SharePoint Workflow, and Windows Communications Foundation) to integrate comprehensive Business Process Management (BPM) capabilities into Business Solutions built on the Microsoft Office Platform/System. This session will help the audience gain a broad perspective of the BPM scenarios targeted by each component and will offer prescriptive guidance on choosing the right option(s) to address common solution requirements. This session will also show how the various components can interplay and add value to Office Business Applications.

• CLI307 - Generating Enterprise Content using InfoPath 2007, SharePoint and the Microsoft SDK for Office Open XML Formats

This session will introduce the Microsoft SDK for Open XML formats and show how developers can use the SDK and InfoPath 2007 to generate Microsoft Office documents on the server from data stored in SharePoint form libraries.

• SER401 - Building Document Management Solutions using Windows SharePoint Services 3.0 Content Types

Windows SharePoint Services 3.0 introduces a new concept call "Content Types" that will dramatically alter the Enterprise Content Management (ECM) landscape. On the surface, Content Types make it easier to encapsulate metadata, but there is a lot more going on under the covers. In this session, we'll take a deep dive into the world of content types and see how to declare them using XML and how to create them programmatically. Then we'll use content types to create a real-world document management solution with custom policies that we'll use to control every stage of the document lifecycle. At the end of this session, you will have a thorough understanding of what content types are with a deeper appreciation of the vale they bring to document management solution development.

I'm grateful for the opportunity to present these sessions (thanks to Karthik, Steve and Jerome), and it looks like this is shaping up to be a great Conference. I'll be arriving early with my friend Andrew Connell if you want to just hang out. Otherwise, I'll look forward to seeing you at the Conference!