Vimpelcom (www.vimpelcom.com) is the leading Russian national mobile operator 6+ millions subscriber database, which operates under Beeline brand. Beeline aggressively grow their customer database and continuously adds new value added services and sophisticated tariff plans in order to maintain it's leading position.

Company prefers to integrate commercial ("out-of-the-box") solutions instead of developing software in-house. This should lead to high quality level, but in real life, significant customization according to Beeline specs which vendor applies to mainstream version could easily introduce even more defects. Augmented by traditional integration glitches and inter-vendor incompatibilities it makes rather complex and uncontrollable environment. Being aware of continuously problems with the testing of new release and localizing (attributing to the software package or component from particular vendor) defects found during the commercial operations, Beeline has initiated independent validation and verifications program. The program goal is to establish one company ultimately responsible for quality assessment of all software solution delivered to Beeline by 3rd party vendors.

Our first goal was to develop and establish the acceptance process for the Beeline, in order to put all activities and communications with vendors under control. Next step was to run QA assessment of the core products already in operations. Using the statistics from the Beeline's monitoring tool, AQT has chosen the first priority candidates: IVR (Interactive Voice response AKA voice menu) platform responsible for balance refill for all pre-paid subscribers. This is centralized solution, used by all regional offices. It was important to emulate some real-life load profile in order to reproduce and localize errors found during commercial operations. In addition, the security audit against potential internal fraud was long time due. As part of assessment, AQT has developed special tool capable to effectively emulate calls and voice menu, which could be effectively scaled from hundreds to thousands of simultaneous subscribers on-line. Content management platform is a gateway between 3rd party information providers and Beeline, supporting all information transfer as well as settlements. AQT has developed automated test model aimed to test functionality and reliability of the system. It took us several iterations to get rid of most severe defects. Balance re-fill voucher management solution - the acceptance test were concentrated on security and reliability, as well as development of the internal fraud control procedures. While fixing the loose ends in the existing software, we've been working on the audit of new solutions. There are two common scenarios on how new candidates for testing could arrive: vendor has issued an upgrade for the implemented system or new system is acquired. We understand that really soon new arrivals will move into the upgrades category, so all test plans is initially designed with continuous regression testing in mind. Having in mind the overall size of company and amount of software solutions in use, our work is far from completion, but both client and AQT agrees that the start was really good.