SharePoint 2010 for Records Management

sharepoint-rm2In October 2011 we presented at a Records Management event in London for Unicom –  SharePoint and Records Management: The Good, Bad and Ugly

The following presentation was created to explore what works and what doesn’t work when using SharePoint for electronic document and record management solutions (EDRMS).

The slides are embedded below. Please note they never behave as well on the web as when presented. Especially so in this case, as I decided to make the slideshow behave like an app

The slides were created to support the talk, not prop up the present so don’t work well on their own. Here are some notes about what was said on the day:

The Good

There are a bunch of new features in SharePoint 2010 to support records management:

  • Document IDs provide permalinks that will work even if you move the documents to different sites, perfect if you want to manage records in a centralised store rather than in-place within collaborative sites. They are configured per site collection, which needs to be taken into consideration when designing your SharePoint deployment.
  • SharePoint includes some new workflow actions that can be used for managing documents and records. ‘Lookup manager for user’ will query the User Profile Service. Provided you have a well populated directory sync’ing with the User Profile service, you can actually look up any properties, but manager is great for automating approval processes. ‘Replace list item permissions’ can be used to automatically set item permissions during the workflow. Means you can set the user’s permission to read-only whilst the manager can still edit during that approval process.  And when the document is finished, you can now automatically declare the document as a record, locking it for everyone.
  • Being able to configure a retention stage was available in SharePoint 2007. But now in 2010 you can configure multiple different stages, which provides a much more realistic solution. The example in the slide: review a document every 2 years, after 12 years send it to the archive.  New in SharePoint 2010 is a Content Organiser as part of the centralised Records Center. It can route incoming items and store them as part of a file plan. Users don’t need to know what retention settings are, provided they describe what the document is, e.g. a contract, or it’s status, e.g. confidential, the Content Organiser can be configured to automatically decide where to put the document. Placing in a certain folder can then automatically inherit additional metadata properties, such as retain for 6 years from date last modified if it’s in the Financial Documents.

New to SharePoint 2010 since July 2011 (requires SharePoint 2010 with Service Pack 1) is a big jump in scale options. Previously, life was happier if all site collections were kept to under 100GB and content databases kept to 200GB. In rare exceptions, you were allowed to grow a single site in a single site collection in a single content database to 1TB.  Now, the 100GB/200GB limit is still recommended for highly active sites and deployments with only basic disaster recovery/backup capabilities. But the new limits are up to 4TB for all scenarios, if you’ve got the right set-up. And you can even go unlimited for archive scenarios (where a tiny amount of content is accessed infrequently). What didn’t make the list? Managed Metadata. I’m being a bit harsh, it’s a great introduction for information management in general. But there are to many limitations currently to get it beyond ‘OK’ for serious records management needs. For more details, see references at the end of this post.

The Bad

Licensing is an easy issue to pick on.  It applies to most Microsoft products deployed in organisations.  As well as choosing which server product to use, you also need to pick the type of CAL, and the options vary if being deployed internally versus externally.  External use has a special Server licence that is different (i.e. a lot more expensive) than the internal Server licence but doesn’t require a CAL whereas internal servers do.  If allowing external use, you’ll be needing ISA Server, ForeFront or other technologies to secure access over the Internet. You may be considering the full FAST product and chances are you will still be using Office for creating and editing documents. Which leads on to Office Web Access – enables Office documents to be opened in a web browser as part of a SharePoint site but isn’t part of the SharePoint licence, it requires an Office volume licensing agreement (no prizes for guessing why). Getting started is the next challenge. SharePoint is a platform and building a records management solution using its features takes a fair bit of planning and configuration.  And comprehensive solutions usually require additional software from specialist partners. Microsoft’s own records management deployment uses partner solutions to manage the full records management process. And, as with many Microsoft products, cross-platform support is currently weak. It’s a step-up from SharePoint 2007 in that you can now open SharePoint 2010 sites using non-Internet Explorer browsers. But try viewing on an iPad, it’s not just Flash that isn’t supported (SharePoint uses a fair bit of Silverlight, Microsoft’s equivalent to Flash) and SharePoint default user interface is definitely not touch-optimised. Almost certainly to improve in future versions, but for now a definite gotcha. And beyond devices, organisations that are primarily on Microsoft products will have a far easier time deploying than those using other servers, operating systems and applications to be integrated.

Options

As to whether or not organisations should be considering alternatives to SharePoint. There are a number of criteria to consider:

  • Platform vs Niche: is records management the primary purpose of your organisation? Then you probably should evaluate niche solutions that specialise. If it needs to be considered as part of a wider remit of information and knowledge-based activities, then a platform is likely to have better value. SharePoint is a platform. It’s not the only one.
  • Proprietary vs Open Source: If you’re anti-proprietary solutions, then you’ll answer this for yourself. From a solution perspective, the main difference is that proprietary solutions tend to have limited lifespans that require upgrades where as open source solutions can be self-maintained for as long as you want to. Whether that’s a good or bad thing depends on if there is a reason for the upgrade, such as changes in security needs.
  • On-premise vs Cloud: Is still a very immature discussion at this stage.  And the cloud-options beyond basic file storage are very limited. SharePoint Online does not have the full feature set of the on-premise server – there is no Records Center, only in-place records management.
The Ugly

This section was less about SharePoint and more about the challenge of records management. First up, are we talking about records management or information governance?  It’s important to define, very specifically, what you mean by records management because that should then drive the technology choice. The hypothetical example I used to demonstrate: (The slide is missing its animation build on Slideshare.) A lorry driver decides to do a friend a favour (agreed via email) and fit in an extra delivery on his route. The tachograph would record this and prove that he drove over hours (as would the unofficial email).  A tachograph must be kept for 28 days. If a court case started before the 28 days are up (hypothetical, remember), the lorry driver would commit an offence if he failed to produce the tachograph, even if the actual day in court is after the 28 days are up. Whilst the email may also become a record as part of the court case, that only happens if it hasn’t been deleted (emails don’t have a defined retention).  Failing to produce the email, if you still have it, would also be an offence.  If the court case starts after the 28 days, neither the tachograph or the email are required to be kept. If they’ve been deleted, they can’t be used in court. But again, it’s an offence if you just say they’ve been deleted when in fact they are still in existence.  And this is all separate to the facts of the case itself. Yes, the tachograph and email would prove the lorry driver drove for too long. But the facts are not the responsibility of records management. If they are, then you don’t have records management, you just keep everything for ever and accept the cost of retrieving all relevant items if and when they are required in a legal matter. Good luck with that. When we started moving from paper to digital records, the volume of the problem began to grow because technically any piece of information can be considered a record during a legal matter.  Social networks have grown the challenge exponentially. You can only manage your own content, you have little or no control beyond your organisation’s walls.  An email always exists in at least 2 places. Are you making sure people are aware of what they can and can’t post on public web sites?  Once something exists on the Internet, you cannot guarantee it is ever removed. Massive data is where we are heading – great globs of different types of the stuff with new tools appearing to mine and manage it.  Where does records management fit in this context?  The same can be asked of SharePoint. Whilst its scale is improving dramatically, can its current architecture handle massive volumes of information? We are seeing new storage methods being used by sites such as Facebook and LinkedIn, moving beyond the traditional hierarchical stores and relational databases. And finally, the Achilles heel of any records management system is not the technology you choose to use, it’s the people you choose to employ.  Most news headlines don’t talk about a system failing to delete something or deleting something it shouldn’t have. They are usually about a human taking information out of the system and sharing it with someone they shouldn’t, or losing the device the information has been stored on. No technology is going to solve that problem without creating a bigger problem in the process.

Summary

Records management should be about two requirements: 1. keeping and disposing of formal documents, i.e. those that have a defined retention; and 2. ensuring any information (formal and informal) used in a legal matter is not deleted whilst that legal matter is in progress. Everything else is about information governance. Few organisations outside of a legal context do this well because it is still an immature solution and technology is only part. of that solution People are likely to cause you more headaches than the technology. Some level of training is essential to remind people about how they manage their information. Still worried? Then invest in automation and autoclassification technologies to eliminate as much of the human element of the process as possible. Decide on your deletion policy (or strategy).  If you want to be able to prove when someone is lying in court, then accept that you are going to spend a lot of money on keeping and classifying everything. For those with a budget, make sure you can manage what matters and get rid of everything else when you can.  Your job is not to try and prove or disprove the legal argument, it is to ensure you are able to provide all the relevant information you have kept.

SharePoint for Records Management?

To close, some tips on using SharePoint for records management.

  • Don’t let people keep old documents in collaborative sites or folders.  If they are no longer being used in an active process but people still want to keep them, move them to a centralised records management site. Because if they are inactive, over time they will be forgotten making them harder to find should they become relevant in a legal matter.
  • Consider using document IDs for formal documents if they are going to be moved to an archive at some point (note: this assumes they stay within SharePoint).
  • Leverage the features to automate as much of the processes as possible. It will take time to configure but the effort will minimise errors and ease records management procedures
  • Linking into the previous slide, if you investing in autoclassification, you need more than just SharePoint. Either the full FAST product, which includes autoclassification, or an alternative solution that integrates with SharePoint.

References

This article was originally published on www.joiningdots.com