mogri : network

Bugs have feelings, too

Source: cartoontester.blogspot.com

Random Thoughts on Software Test Engineering

I still have relatively a short software testing experience as the first formal software testing experience started with my career at Microsoft four years ago. While working on three projects, owning from a couple of components to multiple teams from Product Search, Live Search and Bing, I have gone through several inner-evolutions of the software test engineering and these are random thoughts that I have come across recently. These, of course, are not perfect and might not apply to some of you who are reading this blog, but I hope to capture a few important things to consider when you test a piece of software product, especially for service products.

Customer and Business

You are writing a software for customers because you want your software to be widely used by people. If you are writing software for yourselves, the same reason applies – you want your software to be useful for you, who is still a customer. This directly relates to business as the business only exists if there are customers.

As a test engineer, it is extremely important that you are testing software for your business thus for customers. What is the P1 test scenario you have to cover in your test when you prepare for test specification or test plan? Customer scenario. Because customers mean business. Simple enough.

Costing and Priority

The time is always the most precious thing and having a good plan will lead half way to the success. As an engineer, you need to be able to estimate the engineering cost. It may not be perfect, but it needs to be your best guess with a good detail in your work items. Do not have a work item costs more than two days – break it down to smaller pieces for better planning (more specific detail in your plan!) and better tracking.

The most of cases, you need to work with the time constraint (again, the time is always the most precious thing) and you need to pick important work items. Which work items are must-have? What needs to be done in order to move forward? If you try to accomplish even ten things at once, you are most unlikely successful. Do you have the right priority in your work items?

Test scenario

One thing I have started asking my team recently is, “Can you come up with a story with your test?” If you cannot explain your test with a very good story, it is most likely that your test is not covering your product scenarios very well.

To be more effective, it is advisable to have product developers deal with component level testing via unit test or other means. Why? Because they understand the white box scenario better. Because they want to and need to test their code before checking in. Our interest is more on the test scenarios measuring the system level understanding. We start with the integration test but it should be just a small part of our job. Capacity test, performance test, and telemetry measurement to fully understand the system behavior and to fail fast, which leads us to immediately work on new scenarios. Just don’t forget your ultimate goal – tell a good story with your test scenarios that makes absolute sense.

software development engineer in test

Let’s set ourselves high bar. Stop thinking yourselves as a tester. You are software development engineers specialized in testing. So why would you settle for less? What have made me frustrated recently is that there isn’t enough emphasis of being “developer” and practical and systematic approach to make testers being developers. One thing I keep on hearing (and I also said this to my reports a few times) is “Just get it done for now.” Why? Because it is not a product code – it is just a test code and who gives a damn. Very frustrating.

For last a few months, I have been driving efficient test development for my team so that they spend less to deal with the framework setup and spend more time with writing test cases in more efficient and effective ways. No more copy and pasting code – it doesn’t scale and we should be ashamed of being software engineer. No more test code written in the way a script language is written. Let’s provide the common functionality through base object so that many can benefit from it without writing again. Let’s write reusable test case code so that you can test different test sets without recompiling your code. Let’s treat test code as the product code simply because it is as important.