Test Development vs Test Execution Environment
There is a conversation I have every couple of weeks that goes something like this:
- Customer – we tried to use eggPlant but our security team won’t let it through our firewall; or we tried to use eggPlant but the connection was really slow.
- TestPlant – why are you trying to communicate across a firewall?
- Customer – well the test team is in India, but we want to have our test environment properly controlled so we have it here in UK/USA.
The mistake at this point is to start talking about protocols, ports, security, policies, IT departments, etc. There is a much better way that is simpler, faster, and gives the customer a better test experience. The key thing to realise is that two things are being combined unnecessarily:
- Test-script creation, i.e. some testers sitting at PCs writing test scripts.
- Test-script execution; i.e. executing the tests against the application-under-test to determine if the application is working correctly.
The “controlled test environment” argument makes a lot of sense for test-script execution because you want to ensure that your results are accurate, but it is irrelevant for test-script creation since no-one is using the results of those test runs. If a tester is irresponsible and makes inappropriate changes to their devices (e.g. jail-breaks the device), then the script will simply fail when it is run in the controlled test environment, and the QA manager can take appropriate action to ensure that tester doesn’t do it again. The key point is that no incorrect results have been used.
So whenever I’m in this conversation I say – “why not create a controlled test lab in the UK/USA that does full reliable test execution, but let the testers do their script creation using local devices and eggPlant”. This immediately makes all firewall, security, and performance issues go away. I’ve had this conversation many times and every time the customer has been happy with the approach; in fact happier than if we had simply solved their firewall issue because the whole setup just makes a lot more sense.
So if you find yourself in a conversation that feels like it’s going this way, avoid getting pulled into a technical rabbit hole, and suggest that they separate controlled centralised test execution and local test creation in this way.