We are obsessed with continuous improvement @ Focal. Every day, we work hard to come up with fresh ideas to deliver stuff faster, better and cheaper to our customers. We are extremely passionate about building great software and as a Product Development Services firm, one of the key challenges is the time it takes to certify our products on various form factors, offline and online etc. Until some time back, we were focused primarily on UI/App/Mobile testing but in the recent past, we sort of classified our testing areas into three specific areas: 1. Data Layer: Is the data in the environment as expected?2. Service Layer: Are the services working as expected?3. Application Layer: Is the application behaving as expected? In this blog, I want to talk specifically about the steps we are taking to automate the testing of the data layer with a recent example. Last week, we had to make few unexpected changes to the StoryPulse schema, test a lengthy assessment in one environments (QA) and replicate the same in another environment (PROD). The assessment had too many questions and for some whatever reason, we had to move the data manually from QA to PROD. We had very little time to complete the testing and we decided to leverage ETL Validator from DataGaps. The tool has a number of test plans but we decided to try to implement two tests for our scenario. 1. The Metadata Comparison Test Plan: This is to compare the metadata between the two environments. We really wanted to make sure that the metadata is exactly the same between both the environments. 2. The Query Comparison Test Case: This to compare the actual assessment properties and the questions between the two environments. We wanted to make sure that the text of the assessment, the questions and the possible answers are exactly the same in both the environments. It took less than 30 minutes to install ETL Validator and create the above tests. Our QA could leverage the Visual Query Builder functionality and create the queries pretty easily. With basic training from Datagaps team, we could execute the tests and garner insights very quickly. The end result looked something like below for the Query Comparison Test Plan. It showed what records were there in the source but not in the target, records that were in the target but not in the source and differences between the two.
We just leveraged two tests from the list of available options but in no time, our QA could increase the level of confidence and communicate to the customer that the environment was ready for Go Live. The main point I want to make is that tools/platforms such as Jenkins, ETL Validator,Elastic are mandatory to realize the benefits of continuous deployment/integration processes. Everybody is talking about various flavors of Agile but without having proper tools and processes in place, being agile may just make the software development process more frAgile 🙂