In Part 2 of my interview with Russ Barberio (RB) and Dan Croce (DC), we dig deeper into the PeopleSoft Test Framework (PTF) as we discuss the system and cost benefits associated with automated testing. PTF is a powerful tool, especially when applying variables and using test scripts.
Here’s more on about the value of PTF and how to optimally leverage it.
Q: How do customizations affect the use of PTF?
DC: If a customization is impacted by a PUM change, the test associated with that business process may fail until the customization is reapplied.
RB: When updates are selected for implementation, PTF predicts how the customized process will be affected. PTF’s analytical reporting tools identify where a process will break after the update is applied.
Q: Where have you used PTF, and what did your clients think about it after the project was completed?
DC: Often, when I propose using PTF as part of a testing strategy, clients are sometimes hesitant to create the scripts. These clients focus on the current update vs. considering a broader look at the update strategy for several years. The real payback that clients see is in the second, third, etc. update.
RB: I am currently working with a university where we have used PTF extensively. Initially, PTF and PeopleSoft were being implemented concurrently, which was not ideal. However, the university has embraced the use of PTF and now is planning to apply its first PUM image. As part of this project, Sierra-Cedar’s technical team developed a robust set of analytical tools to analyze the PTF results, along with a PUM WorkCenter.
Q: When Oracle-PeopleSoft changes a page or adds a field, does the PTF need to be re-done in this area?
DC: Rework only occurs if the page has changed structurally to include a different functional feature. For example, if a page is enhanced with a checkbox that is read by a downstream process, the script needs to be updated to take advantage of the new functionality.
RB: Since PTF is part of PeopleTools, when Oracle-PeopleSoft changes an object (such as a page reference of a field), the PTF test will change as well. If a field is added and the client chooses to use the new field, PTF will identify other areas that may be affected.
Q: What do you say to organizations who believe their data or configuration changes preclude them from using PTF?
DC: This goes back to the perceived data limitation issue that I mentioned previously. A smart, thoughtful test design should address most of those issues.
RB: PTF is a reliable way of predicting how data or configuration changes will affect the rest of the system. It reduces the occurrence of “I didn’t expect that” moments.
Q: Are there any supplemental tools available for managing the test process?
DC: Sierra-Cedar built a PUM WorkCenter, with series of drilling queries, to simplify and enhance the information provided by delivered maintenance and coverage reports.
RB: Because there was no delivered PUM WorkCenter, we built one to enhance the ability to analyze the outcome of PTF tests against the updated database.
Q: How have these new tools simplified the testing process?
DC: The main and most powerful feature of our PUM WorkCenter is the direct link between a specific PTF test and the PUM objects included in it. In addition, there are new ways to view test gaps and track test runs.
RB: We created reports that make it easier to identify issues with the PTF tests against the updated database and then drill down to the details. These reports allow us to see object types to assess the extent of the issue, as well as identify issues not included in the delivered reports.
Q: Does PTF catch system issues and lower risk via complete/consistent testing, or does it just provide peace of mind?
DC: It does both. Regardless of how you look at it, PTF is an effective way to substantiate that the latest upgrade did not alter any of your organization’s business processes.
RB: PTF does lower the risk of missing unintended results. It will be more comprehensive than manual testing. Yes, it does provide peace of mind when an organization decides to make it a policy to keep current with the system.
Q: Will PTF identify system issues that no one is aware of yet?
DC: During the application of a change, PTF identifies issues with customizations impacted by the change but not yet retrofit. When used properly, the reporting tools expose where these issues will occur prior to ever running a test. Subsequent testing becomes a validation that the customization was retrofit (or removed) correctly.
RB: One example of this is a case where a field changed from “Text” to “Long Text.” Both the fields were on the sub-record, but the update replaced the former field with the latter field on the parent record.
Q: How much rework is needed when a new PUM image is applied?
DC: When a page has a structural change, which isn’t often, then the PTF will need an update.
RB: An example is when a reference on a page to a field on a subpage was changed to a different field on that subpage (with a different field type), the test failed and had to be reworked (a pretty rare occurrence). There may also be some rework if the PUM delivers functionality that enables the user to remove a customization.
Q: When would you expect a payback from test automation vs. the time investment to develop test scripts?
DC: The major upgrades of the past are over if organizations use PeopleSoft Upgrade Manager as it was intended. There are benefits if an organization adapts instead of creating an unneeded upgrade project for itself. To make this possible, however, an organization needs to test frequently. Investing upfront in automated testing will help an organization transition to the new model, become nimble, and never again need to plan and invest in an expensive major upgrade.
RB: The payback depends on the frequency of PUM applications. PTF requires an initial investment because PTF tests need to be created. The need for creating new tests will decrease dramatically as future PUMs are applied.
Q: When does PTF save time or money, or when an organization should expect a payback?
DC: This really depends on how well you prepare up front. By taking the time to clearly document the business processes that your organization uses and then convert them into test scripts upfront, you start saving time and money with the application of your first maintenance package.
RB: PTF starts saving time and money after the initial PUM update. All subsequent PUM updates will require fewer resources. It is important to remember that the ability to keep the PeopleSoft current is the real payback of PTF.
Q: After using PTF at multiple clients, what are your top lessons learned to pass on to our reader?
DC: There must be organizational support. The organization must be ready (know its business processes) and have an upgrade strategy defined that clearly documents how and when PTF will be used.
RB: When implementing PTF, develop a thorough PTF implementation project plan, with a clear consensus of who performs each role; communicate your project plan to appropriate management. You will get the most “bang for the buck” when testing processes that are the most volatile and have customizations. Drive as much data into the tests as you can. It will make it much easier to make changes in the future. Learn and use all the optimization tools provided with the system.
Watch Russ’ and Dan’s WEBINARS Introduction to AI Processes in the PeopleSoft Testing Framework and Mathematical Optimization and Automation of Inventory Replenishment Policies for additional information on PTF.