Data Mapping & Prototyping

Data analysts can bring raw data from any number of sources, and work from the data up, to fully understanding how to move their data to the new target system, and build a working prototype of that data movement.

This not only allows the analyst to get their mapping specifications correct, but also provides business users with the earliest possible visibility to what they are going to have in their final target system, allowing them to ensure the fitness of purpose, not just technically, but also to the business itself.

Data can be joined, aggregated, filtered and transformed with simple point-and-click methods. Data can be filtered with any level of complexity using a simple guided-English rule builder and data can be transformed graphically using the Pandora Expression Editor.

Pandora is a shared and secure multi-user system allowing many users to collaborate concurrently on the same prototype, sharing results in real-time, regardless of their location.

Data Integration design is fundamentally exploratory, requiring incremental refinement - the more you understand about the target system and existing source data, the more you know about what must be done to make existing data work in the target.

Pandora allows users to design and build data movements piece-by-piece as they gain more knowledge, through a series of layered, transformed views of data. No matter how many steps were taken, Pandora optimises the result to the most efficient design, and generates a fully-validated, English Mapping Specification complete with the output test data. This can be implemented with the full confidence that the specification is 100% correct because they have already been seen the final result with full data volumes.

Pandora allows users to do whatever data manipulations they wish, in whatever order and save the results as Dynamic Views, which can be instantly recalled, and then manipulated further to create other views, or edited to further develop the same view.

Pandora allows the user to create new columns in any table or view, derived from any number of existing columns with a feature-rich palette of 200+ functions to choose from. No technical skills are required to do this, just knowledge of the data and simple point-and-click and drag-and-drop.

Pandora provides functions for all of the following areas:

• Aggregation
• String processing
• Data Cleansing
• Decision Rules
• Date & Time
• Maths & Scientific
• Lookups
• Mapping & Constants
• Free-text Processing

No technical skills are required, just simple point-and-click in the Graphical Expression Editor. All transformations are real-time, meaning that you can add and remove columns and see the results instantly - no wait time, regardless of data volumes. Transformations are dynamically compiled to become part of the core server, so are performed very quickly.

You are limited only by your imagination as to how complex the rules are, and they can be easily edited and reused as your understanding of what needs to be done to the data increases over time.

If you have applied multiple expressions over a series of layered views, when looking at the final output Pandora will automatically combine all the expressions for you to create a single, optimised version for the final generated Mapping Specification document.

Pandora provides an intuitive data-driven method for filtering, joining, transforming and aggregating data that does not require the user to mandate any series of processing steps. They simply manipulate the data as they see fit, on-screen, saving their work in as many different steps as they wish. Pandora remembers everything that they do. Eventually, when the user has created the final result that they want, they can simply ask Pandora to do all of the hard work of evaluating the optimum mapping specification to go from the original sources to their new target, irrespective of the number of steps they took. This is entirely dynamic, and at the click of the button. Should they change anything at any level, the specification will automatically reflect these changes.

This truly automates the process of building a fully validated mapping specification for data movement. Indeed, for smaller Data Migrations or for migration of static reference data, the output of Pandora is often sufficient avoiding the need to build anything outside of Pandora.

Part of Pandora's box of tricks includes the ability to map any data view to another automatically by using Semantic Metadata matching. Pandora will attempt to map fields from source to target automatically, using exact matches in the first instance, and then a series of loose matching rules to ascertain the most likely target fields for available source fields. Where there are multiple opportunities for mapping, it provides the user with a list of potential candidates from which to choose. The auto-mapper validates all of the mappings to ensure that the source and target are compatible in terms of datatype, length, nullability and syntax rules with inconsistencies brought to your attention visually. Where applicable, the metadata for the target can be manipulated within the auto-mapper itself.

This is an incredible accelerator to the Data Mapping process. The result can be saved as a Dynamic View with no wait time required.

One of the hardest things to do for any Data Integration project is to actually test the end result, especially in cases where the target system is still under development. Generally, a series of hand-crafted test data is used, that is unlikely to be truly representative of the data in the real source systems. This typically rears its ugly head right at the end during system testing, or even during pre-go-live rehearsals. A fantastic free benefit from using the prototyping capabilities of Pandora with full volumes of data is that, in addition to the valid Mapping Specification, you also get a data set which can be used to validate the final results out of build, and also to provide the much-needed array of test cases. Pandora provides full export capabilities to get this data out into a usable format for your target system.

You can also use Pandora to analyse the output of the final build processes to ensure that they match your prototype results exactly - which they should. This is fantastic for people who have outsourced their Data Integration build to a 3rd party, to ensure that they are getting and accurate and high-quality result.