When good intentions go awry

So I was reading the Using Registered Tables vs Object Types design patterns discussion a few months back on EMC Developer Network.

I’m a Documentum old timer, so I was more comfortable using registered tables. Those of you who have used registered tables as lookups knows that its pretty straight forward. Create a table and then REGISTER it so that you can query against it using DQL. The hitch is that if you dont have privileges on the database (or the database is managed by a different group), you are highly dependent on them getting the table schema created with the appropriate permissions. Also, you are dependent on them to move the table and its values from one environment to the next OR you have to have strict version management of table population scripts. Anyways, I have followed this tedious process for many, many years.

After reading the design patterns discussion, I decided for my new project, I would try using object types for lookups. I was not too concerned about security on the objects given that the users were using Taskspace and would not have the ability to delete/manipulate the lookup objects.  I created custom type will null supertype, since I wasnt planning to have any content associated with them.  See Laurence Hart’s posting on using Contentless Objects as Lookups.

So I created quite a few custom types and created several hundred object instances of my custom lookup types.  It was very easy to change the value of the lookups and create new ones using DA or any other Documentum application.  Not only did I not have to worry about managing table population scripts, but I did not have to get DBA team involved, since creation of custom object types can be done using DA or DAB.  Everything work as expected.

Fast forward a couple of months and now we are ready to deploy our Taskspace application from DEV environment to testing environment.  Taskspace was designed to use the docapp archive feature of DAB to move form templates (components), processes, tabs, roles, and presets from one environment to the next.

Lo and behold, DAB does not support the archiving of contentless objects with null supertype.  There is no way to include object instances of this kind in docapp archive.  This makes sense at object type definition level; if custom type is not derived from dm_sysobject, then it doesnt have i_folder_id attribute, which means the object lives in what I like to call “La La Land”.   Application Builder supports the inserting of objects, but it assumes that the object resides in some folder.  Unfortunately, there is no “Insert Persistent Objects.”

I havent tried Composer to see if this is supported, but it really was a surprise to me that this feature was not in DAB.  In the end I had to remap the custom lookup types to be derived from dm_sysobject.  I hope my experience with this will prevent someone from going down the wrong path.

A year later - where is Documentum and Sharepoint today?

Someone just recently posted that its been almost a year since I have initiated the discussion about Documentum vs Sharepoint. Robin East’s comments were pretty insightful and a year later, it seems to be right on the mark.

At EMC World 2007, I felt that EMC was promoting its Content Services for Sharepoint as a way to respond to existing Documentum customers when they started to ask about Sharepoint.  As of today, there seems to be only 1 or 2 dedicated sessions to Sharepoint.  Hopefully, there will be more sessions as we get closer to the conference date.

I personally feel that Documentum is in the process of re-defining itself in the market place. Since March Madness is around the corner, I would like to say that this change is analogous to a basketball team losing all of its seniors last year. I believe in the next two years, we will see EMC Documentum re-invent itself by truly embracing the services architecture and spend less time developing specific products.

We have seen the evolution of Webtop and how feature rich the application has become. Like Microsoft Word 2003, there seems to be too many features exposed through the UI that most users do not use -80% of the users use only 20% of the features. With D6, the clutter has been reduced by the introduction of presets functionality. I think this is a stop gap measure.

The reality is that there is no magic bullet application that will work for everyone. EMC Documentum needs to return to its roots and focus on how to best support its developers and make it easier (and thereby cheaper for clients) to produce feature rich applications in a minimal amount of time. I think TaskSpace is the first step, but I feel that EMC Documentum needs to put a lot more R&D into developing this platform and expand its current line of web/product services.

I dont for see revolutionary changes when D6.5 comes out later this year, but I do believe that 2009 will be the test on whether the new EMC Documentum can really challenge Sharepoint’s advances. Till then, Sharepoint will keep on making inroads into ECM market.

Any other predictions for 2009?

Where to put it – a Migration Dilemma

A fellow colleague of mine was recently involved in a migration effort to move documents from a custom application into Documentum. While this custom application did not have a folder structure, Documentum does. The client was insistent of not wanting to spend any time or effort in designing a rudimentary folder (taxonomy) structure to place the migrated documents. Instead, the client wanted to import over 50,000 documents into a single folder. They insisted that the folder structure was unnecessary and that searching would solve all of their needs, as was the case in the old custom application. Is this the right thing to do?

I disagree with this approach if there is some metadata available with the migrated content. Most content management applications usually have metadata like creation date, document type, or department. This kind of metadata can be used to aggregate the migrated content into at least 10-20 folders. This can potentially reduce the number of documents per single folder from 50,000 to maybe 2,000-3,000.

Why is this important? Obviously, you still would not want a user to browse through a few thousand documents in a folder. The benefit of aggregating content into smaller buckets is the ability to process documents in batches. This may not be a current requirement, but it does support future needs.

Imagine having migrated content segregated into yearly/monthly subfolders. This would facilitate archiving of content based on creation month. It would also lend itself to applying retention policy and disposition dates for different groups of documents.

If there is no metadata available with the migrated content, then it does not make any sense to expend time and effort to try to impose a folder structure/taxonomy. So now you know where to put it.