HPRath

Innovations in High Performance Computing

GigaTester – Regression Testing Framework

GIGA Tester – Distributed Regression Testing framework

Overview

Giga Tester is a distributed testing framework. It tries to solve many problems in traditional testing frameworks when working with large number tests. Below is a list of major issues with traditional testing framework

  • Tests are executed serially (one after another)
  • No dependency information can be set when tests are run in parallel we cant set some thing like the example below
  • i.e. if A & B depends on C and D depends on B then tests should run C->A&B (parallel)->D (on completion of B)
  • Running a large number of tests on a single computer.. no ways to automatically distribute tests on a set of connected computers, manage execution and re-scheduling the tests on hardware failures
  • Limited mounted disk space – can not leverage fault tolerant file storage like Hadoop Distributed File System
  • Some times esoteric and handicapped testing language
  • No centralized log file

Giga tester is designed to work with modern software projects in mind. With passage of time, complex software products Ex: Operating Systems, Database, EDA (electronics design automation), CAD software become really large with millions of lines of code. Most of these products do tests in console mode, i.e. individual functionality is contained in a tests program, which when invoked with input data and arguments generates out put and if any result data. A typical trivial test would involve capturing the output and comparing it with expected output.

This approach of comparing textually the output and expected output is erroneous. Some text formatting is captured as diffs and test case is marked fail. Giga tester allows using a full fledged Python 3.x interpreter. A test can analyse result and mark the test as pass or fail. A test suit consists of the following sections

  1. Setup – perform any setup related activity, that is prepare data for setsts
  2. RunTests – A number of tests can be written to be run under run-tests
  3. Analyse – Provides a place holder to write code to analyse results of multiple tests
  4. Cleanup – Clean up disk space and other such activity

 

All these steps can be written in a python script file. The python test-case script is executed in the Giga-testers embedded python interpreter. Test case can use any Python3.3 module installed on the machine. It can also interact with GigaTester to perform many other tasks such as

  1. Set status of a test at any point
  2. Set reason and extended reasons for the test – these would appear in the dash board
  3. Centralized configurable logging options – per test logs or a single log or error warning and info logs
  4. Signal other tests
  5. Pass data to other tests
  6. Send emails & alerts

Features

Giga-tester has the following features

  • Designed to scale to extremely large number of test cases, new servers or commodity PCs can be added to the pool of test servers
  • Tests are executed most optimal way in parallel based on test case and test data dependency
  • Aggressive scheduling strategy (based on past test execution times)
  • Test dependencies can be set in the framework. These are maintained in a graph inside Giga Tester, with each vertex representing tests and edge is represented by set of tasks
  • Tests can be written Python3.x. Any python module compatible with Python3.x can be used
  • Centralized configurable logging options – per test logs, single log or error-warning-info logs
  • Runs on Linux/Windows/Mac

 

 

 

Leave a Reply

Required fields are marked *.