Environments

It is not uncommon if a tester gets asked, what test environment do you want?, the short and simple answer is, the same as production. Considering the cost of putting a test environment in place, not just the physical or virtual servers, but also the management of that environment and related data, maybe we should be looking at our options?

Listed below are several areas to consider when thinking about the test environment; they are learned from years of working with many clever and talented people including developers, architects, data architects, automation and performance specialists, build and release teams and Dev Ops teams.

Remember that all these people will be there to support you in your testing endeavours, therefore it is good to listen and learn. As with all things, especially with processes, do what it right for your organisation. However, review regularly with a critical eye, keeping what is working and changing what is not.

Here are my points on trying to keep and maintain good test environments.

1 Ownership

Decide who owns the environments and what it means to be the owner of the environment.

This can be a contentious topic and it is worthwhile discussing and documenting your organisations discussions on this point. I used to think that the test team should own the environment. However, I have changed my mind on that point. I now think that either the build and release or operations team should own the environment. Either of those teams should provision the environment for use by the test team and own that environment.

They are the team that should understand what is needed to deliver the environment to the requirements of the test team. Equally, when you have finished testing, they are the team that will reallocate those resources. In my view the test team should only ever be users of the environment. Why do I think that this point is important? In small organisations and applications, I think that it is straight forward for the test team to understand what goes into the environment. However, for large systems, I think this becomes a much more complex issue and is best left to the people that specialise in this area. Plus, they can ensure that the environment is used to its fullest potential.

2 Test Levels and Types

What are the levels and types of testing to be performed?

Everything from code reviews, which probably need the barest of environments, to performance testing which will require something close to production. When it comes to specialist testing such as security, operational or failover and recovery testing, certain network elements may be critical. Put in place the environment you need to meet the objectives of this testing, no more and no less.

If the objective matches the test environment it will be clear what the budget holder is paying for. It is fine in earlier test levels to have the configuration of the test environment to be different from that of production, as long as you are clear of the risk and at some stage plan to test in the production configuration. It is also worth pointing out, and I know most of you know this, the relationship between two solution specifications is often not linear. For example, doubling the CPU’s on a server may not result in CPU usage or response times being halved or even improved.

3 Data

Are you going to generate your own data or take a copy from production?

Clearly, taking and using data from production in these days of data protection and GDPR means following strict protocols. A few points on using production data: Firstly, if you are taking data from production, I would strongly advise that it is anonymised while keeping relation between transactions. All personal information should be hidden in some way. Secondly, try and keep the distribution profile of original data.

For example, in a database of peoples’ names the names are not evenly distributed, so if you are testing the speed of a search algorithm it would make sense to have a similar distribution to the original names. Lastly, remember that production data will often not have the edge conditions that testers are so fond of. Therefore, additional test data may be required to be generated. In addition, if you are using data from production, ensure that refreshing your test environment can be done easily.

4 Configuration Management and Change Control

Everything that needs to go into your environment needs to under configuration management and change control.

That includes data, database schemas, configuration files and all the code. Ideally, you should be able to tell what version of the solution you have in your environment at any time. This is not a trivial matter, in some cases you will have many applications/services delivering a solution. How do you know if you have a configuration management and change control issue, the defects that were fixed reappear, or functionality that was there, goes ‘missing’ when a new release or patch is made to the environment?

If you don’t know what you have in your environment in terms of versioning for code, configurations, database schemas, and data, how can you tell what is going into production and hence it’s quality? As part of configuration management, I would add in the configuration of the environment itself, which would include the setting up of servers, firewalls, load balancers and queues. It is key that an organisation has a configuration management and change control strategy. This should include: what items fall under configuration management and control, how version numbering is going to be defined, approach to branching, how long a branch should stay alive for, and merging strategies.

5 Use of Automation

With the advent of continuous integration, automation for the building and deployment of code is an absolute must.

Even if you are not making use of continuous integration and deployment, it should be a requirement that your organisation can build an environment in hours. Plus, the approach to building your test environment should not be radically different from building production.

Most teams I am sure are now using build tools such as Jenkins; however, using AWS and containerisation can prove invaluable in building and rebuilding environments efficiently. The provisioning of cloud environments can be especially cost effective if those environments are ‘switched off’ when not needed. While these technologies are becoming more common place don’t underestimate the time and effort required to become proficient using them.

Exactest is an independent software test consultancy providing software testing solutions to quality concise and market driven clients utilising experienced test professionals and innovative technical solutions.