IBM ECM migration and system consolidation

Customer Challenge

Our client – a large international bank – piled up a complex and heterogeneous ECM infrastructure, with multiple systems and document repositories. For years the bank was using an old FileNet Imaging Service (IS) to store documents and a newer IBM FileNet Content Engine (CE) 4.2 to store the related metadata. Later, a newer version (5.1) of IBM FileNet Content Engine was also added to the palette. Due to workflow reasons various application and sub systems were connected with these systems, so data was stored across three systems using different settings and different directory structures, raising numerous IT issues. Maintaining three different systems also meant high expenses and additional work for the IT department. FileNet IS had especially high annual maintenance costs, being already an unsupported system.

Issues with the ECM infrastructure:

  • Business content is stored in 3 systems (IS, CE 4.2 and CE 5.1)
  • Complicated and resource intensive to manage and support
  • Limited scalability and sensitivity to issues
  • FileNet Imaging Service is pretty slow (uses legacy disk based storing logic)
  • Backup issues (system is unable to effectively backup 2.5 TB of data in the given backup window)
  • Software components being used on a variety of hardware

Due to these issues, our client initiated the consolidation, simplification and optimization of this infrastructure – to reduce the risk of negative business impact, by migrating all the content, approximately 21.000.000 documents (2.5 TB of data) from the old FileNet IS and the older version of FileNet CE into the newer version FileNet CE.

Solution

First, MPS did a detailed performance review of the actual issues demonstrated by the users (review of application server logs, FileNet system logs and system parameters, application server parameters, etc..). Parallel to this, the target system’s (the newer version of the FileNet CE) performance was optimized, to be able to successfully handle the large amount of new data. Then using MPS Voyager – our migration tool, and its built-in IBM Filenet system connector (just one of the many built-in ECM system connectors) the two source systems were scanned for business content. After the discovery, the source systems were mapped to the target system by directly mapping source system classes, properties, folders and security settings to those in the target system, defining transformations and derivations where it was needed. Moreover, with MPS Voyager security and mapping rules can be configured to automatically create a folder structure, and set security of folders/documents based on metadata of migrated objects. With an additional lookup identification, source systems, lookup keys and transformations were defined to identify the missing metadata.

Next step was the testing, that included deploying MPS Voyager to test the environment, connecting to source and target systems and testing the migration under various circumstances, correcting errors, trying to identify data quality problems as well as running multiple storage optimizations on the target system. After successful testing, MPS Voyager was deployed to the production environment and executed a scheduled migration (migrating only during non-business hours), during which 21.000.000 documents (2.5TB of data) were moved from the two source systems to the target system. The connected applications were migrated in four cut-overs.The whole live migration process took 3 months, running an average of 100 hours/week with a speed varying between 3 to 8 documents/sec. (slower speed were the result of the instability of the old FileNet IS).

During the migration an extensive, configurable logging was done, making it possible to generate daily reports and keep track of errors (which in turn will be corrected and remigrated).

As a final step, we conducted the backup optimization of the “new” system, by optimizing the daily incremental and weekly full backup and configuring the new settings of the IBM Tivoli storage manager for the FileNet backup, as a result, reducing the time of the backup process, to fit the given backup window.

Results

MPS successfully consolidated three IBM systems into one, by migrating all the business content from an old IBM FileNet Image Services system and an IBM FileNet Content Engine 4.2 into one single, newer IBM FileNet Content Engine 5.1 system. This migration not only made everyday work and development much easier, clearer and faster for our client, since all the necessary data is now stored in one, single ECM system, but also eliminated the high annual maintenance cost of IBM FileNet IS, as well as the the costly burden (in terms of both, money and human resources) of maintaining three various systems for the same tasks. Furthermore, by optimizing and fine-tuning the new IBM FileNet Content Engine system,our client was able to eliminate the previously existing performance and back-up issues.

  • Consolidating 3 various IBM systems into 1
  • Migrating 21.000.000 documents (2.5 TB of data) to the target system
  • Eliminating almost 560.000 USD annual maintenance cost for IBM FileNet IS
  • Instant ROI (cost of the migration is a fraction of annual maintenance fees)
  • Improving system performance with fine-tuning
  • Solving all back-up issues