Cleanse, Transform, and Enrich Data for Successful SaaS Deployments

Deploying SaaS (Software-as-a-Service) invariably involves converting data from existing systems into new applications. While data conversion presents an opportunity to preserve historical information for reporting and transactional processing within the new SaaS environment, it also poses challenges for many deployments. 

In our discussion, we will consider the potential advantages and benefits of migrating data into your new SaaS applications, identify key challenges associated with data conversion, and explain how Sierra-Cedar can facilitate a successful conversion process, serving as the foundation for a successful deployment of Oracle SaaS. 

Deciding which data to migrate to your new SaaS Application 

As you adopt a SaaS application, you need to make various decisions about what data to migrate. First, it’s essential to consider data retention and historical data purging policies. Historical transactional data can serve as an asset to the organization, but sheer volume can create a burden. Therefore, your organization must use careful deliberation to balance retaining valuable historical data with excessive data preservation. 

Different types of data require different considerations. For example, you can avoid converting some transactional data into a SaaS application, but non-transactional data, like Workers and Suppliers, require conversion. Additionally, regulatory or organizational reporting requirements may mandate the conversion of specific transactional data.  

Another example is a situation in which the SaaS application does not provide a conversion method. For example, Payroll check data is often used for Retro Pay scenarios. Payroll data not created by the SaaS payroll application could cause the Retro Pay processes to fail. 

Why does data need cleansing?  

Data corruption can occur for various reasons over time. One frequent scenario involves repurposing an extraneous field designated initially for one type of data to store a different data type. This shift in a field’s usage can lead to confusion about the purpose of the data values stored in that field. Another significant factor contributing to data corruption is manipulation, either through a failed batch process, errant programming, or direct SQL updates that inaccurately modify or corrupt the data. Additionally, in relational databases, removing a parent row in a parent-child relationship can result in orphaned child rows, further complicating data integrity. 

Transforming Data as part of a SaaS Implementation 

When data is moved from one application to another, especially into a modern online application, the data often needs to be adapted to fit the new system’s rules about data types and what values are allowed. For example, let’s say the old system only stored numeric codes to represent the Education Level for a job application, but the new SaaS application requires words like “High School Graduate” or “Bachelor’s Degree.” To transform the data, we might use a “crosswalk” table that shows which number should be changed to which words, verifying the data is accurate before it goes into the new application. 

Other data transformations can involve using the values from two or more fields extracted from a dataset to determine the value of another field. This process often employs algorithms or decision trees to generate new data that wasn’t present in the original application but can be inferred from the available inputs. These transformations usually occur after the data extraction and before loading it into the new system. It may also be necessary to have a default value for required fields when a valid value cannot be determined based on the available data. 

How Does Data Enrichment Differ from Data Transformation?   

Sometimes, important data needed for conversion might not be available in the source application. This extra data could be in other applications or in the target SaaS application itself. Data enrichment helps by extracting additional data related to transactions or people from external sources. This level of data enrichment requires an orchestration layer within the transformation process, capable of employing business logic to control when and where data enrichment should occur and where to locate the enriching data values. Integration platforms such as Oracle Integration Cloud (OIC) or MuleSoft can serve as this orchestration layer, enabling connections to various data sources such as applications, spreadsheets, or web services to gather enriching data. Data enrichment makes it easier for users to work with converted data in the SaaS application. 

How can Sierra-Cedar help you prepare for your transition to SaaS? 

Sierra-Cedar’s exclusive DataFactory tool harnesses the capabilities of OIC to provide a powerful platform for programmatic data cleansing, transformation, and enrichment. Our team of functional and technical consultants understand the challenges of converting data into SaaS applications. We are eager to discuss your data conversion needs and share the insights gained from our extensive experience in addressing such challenges. Please feel free to reach out to us to explore how we can assist you further.

Wayne Pinckley brings nearly 30 years of experience in enterprise software leadership to Sierra-Cedar LLC. With a focus on collaboration, Wayne cultivates innovation within the Public Sector, Commercial, and Healthcare, business segments. Wayne’s expertise spans various capacities, including project management and technical leadership. Throughout his career, Wayne has displayed his versatility by serving a wide range of clients, including commercial enterprises, federal agencies, state and local governments, school districts, and higher education institutions. His dedication to harnessing technology for the enhancement of both government and commercial entities has resulted in repeated successful project engagements.

Wayne is a graduate of Freed-Hardeman University, where he cultivated personal values of integrity, kindness, and servant leadership. Based in Middle Tennessee with his wife, Darlene, Wayne derives fulfillment from activities such as public speaking, singing, computing, outdoor grilling, and traveling. He seamlessly integrates his personal values into his professional ethos, embracing a comprehensive approach to achieving success in endeavors greater than himself.