Friday, December 19, 2008

Ego of testers - fuel to exploratory testing?

Why do some testers feel that they do not have to follow the test script blindly and they know better than that? Is asking them to stick to the test script, insulting to their intelligence? I am not too sure about the answers as it has many psychological and social layers.

I think there is a need to identify the inherent nature of any individual tester, if some feel that they would do better doing exploratory testing then we need to respect that. At the same time, effort must be made to validate their claim to assess their aptitude. Those testers who wish to be 'free-birds' should be not be forced to stick to pre-decided test script, it will certainly suppress their creativity. While we go about doing this, we got to ensure that we are not producing 'rogue-testers' (apologies if anyone is offended by this term). As saying goes from Spider-man movie, 'With great power comes great responsibility', so we need to be careful that we are not giving power into reckless hands. I would not like to use the term 'adhoc testing' as synonymous to Exploratory Testing, 'adhoc' for me signifies careless, sloppy and chaotic, things we do not relate to testing.

Now who would be ideal candidate for exploratory testing: someone who is well-verse with the application functionality and design, someone who has strong intuition, someone with strong analytical skills, someone who can design and modify the test design on the fly.

Exploratory Testing need not be against the ides of test scripting, rather it should be used in combination with test script based testing. And each of these teams would require testers with specific temperament and ability for successful execution.

Friday, December 5, 2008

Agile Manifesto

The Agile Manifesto reads as follows:

"We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value [the following]:

  • individuals and interactions over processes and tools
  • working software over comprehensive documentation
  • customer collaboration over contract negotiation
  • responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more."

The Agile Manifesto has a list of general principles attached to it.

"We follow these principles:

  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  • We welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
  • We deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  • Business people and developers must work together daily throughout the project.
  • We build projects around motivated individuals. Give them the environment and support they need and trust them to get the job done.
  • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  • Working software is the primary measure of progress.
  • Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  • Continuous attention to technical excellence and good design enhances agility.
  • Simplicity—the art of maximizing the amount of work not done—is essential.
  • The best architectures, requirements, and designs emerge from self-organizing teams.
  • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly."

What's your interpretation of this?...

Tuesday, December 2, 2008

Agile Testing - Only for Agile Development methodology?

Before I introduce the new paradigm of the Agile Testing process for ALL development methodologies (including Agile), I am put across my thoughts over the traditional Agile Testing practice which exists in the realm of Agile development.

Simply put, it is the testing practice that follows the agile manifesto, treating development as the customer of testing. In this light the context-driven manifesto provides a set of principles for agile testing.

First let me list the Agile Development Methodologies that I a aware of:
  • Extreme Programming (XP)
  • Crystal
  • Adaptive Software Development (ASD)
  • Scrum
  • Feature Driven Development (FDD)
  • Dynamic Systems Development Method (DSDM)
  • XBreed
Now let us look at the basic role of testing any project irrespective of the underlying development methodology:
  • Testing is the headlights of the project- Where are you now? Where do you headed?
  • Testing provides information to the team- This allows the team to make informed decisions
  • A “bug” is anything that could bug a user- Testers don’t make the final call
  • Testing does not assure quality- The team does (or doesn’t)
  • Testing is not a game of “gotcha”- Find ways to set goals, rather than focus on mistakes
To focus on one of these, the common XP Practices are:
  • Test-First Programming
  • Pair Programming
  • Short Iterations & ReleasesRefactoring
  • “User Stories" Acceptance Testing
In Test-first programming:
Developers write unit tests before coding. This
- Motivates coding
- Improves design (reducing coupling and improving cohesion)
- Supports refactoring (Refactorting --> “Changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure”.) - Make the simplest design that will work. Add complexity only when needed. Refactor as necessary. Refactoring requires unit tests to ensure that design changes (refactorings) don’t break existing code.
Many open-source test tools have been developed to support this. for e.g. xUnit

Pair Programming
All production software in XP is built by two programmers, sitting side by side, at the same machine. This practice ensures that all production code is reviewed by at least one other programmer, and results in better design, better testing, and better code.

Iterative Test-Driven Development
Extreme Programming is obsessed with feedback, and in software development, good feedback requires good testing. Top XP teams practice "test-driven development", working in very short cycles of adding a test, then making it work. Almost effortlessly, teams produce code with nearly 100 percent test coverage, which is a great step forward in most shops.It isn't enough to write tests: you have to run them. Here, too, Extreme Programming is extreme. These "programmer tests", or "unit tests" are all collected together, and every time any programmer releases any code to the repository (and pairs typically release twice a day or more), every single one of the programmer tests must run correctly. One hundred percent, all the time! This means that programmers get immediate feedback on how they're doing. Additionally, these tests provide invaluable support as the software design is improved.

Now lets look at Acceptance Testing
User stories (same purpose as that of use cases but are not the same) are short descriptions of features that need to be coded. One or more automated acceptance tests must be created to verify the user story has been correctly implemented. Acceptance tests verify the completion of user stories. Ideally they are written before coding.

Should users go along with this?
Some say that XP is an invitation to poor quality and an excuse for hacking. I think that XP is exciting and will improve the practice of testing in the industry.

Even better if we can have an independent Agile Testing process (that is 'independently agile') irrespective of the underlying Development Methodology... please watch this space for more on this.

Saturday, November 29, 2008

Are you a tester?

We all are testers, makes you ponder? Yes, we all test, sometimes more than necessary but we do it all the time... checking if the house door is locked properly, establishing that the vegetables that we are buying are fresh, taking prospective car to test-drive, solving the test-papers from previous years before the exam, pressing the elevator again while it is already pressed and last but not the least, checking to ensure our fly is not open.

All of the above involve verifying and validating our certain expectations or fears (positive and negative testing) associated with the target that we are dealing with. These actions help us reaffirm our belief in the outcome that we expect, makes us tad little more comfortable. Comfortable in the realm of the related risk and cost associated in case of finding a "bug" later after the ownership is transferred to us. Thus the intensity and complexity of the testing varies based on the risk and cost. For example we should be comfortable if told that the sofa-set that we buying is tested to the effect of 99%, we may not feel the same if our airline says same thing for the aircraft that we are going to board (are we going to ask whether this is the 100th flight??!!)

Out need to test arise from the fact that we are humans and we can make mistakes, even machines can go wrong at times. We need to ascertain that the product or the process is as per the defined guidelines and standard, this leads to quality associated with the product. Higher the percentage at which the tests pass, higher is the quality. So essentially we are assuring the quality that we expect from the object under test. Also, to ensure the element of manual defect does not creep in, we tend to automate the tests, especially in case of the multiple repetitions of the test cases under various data sets and scenarios. This leads to test automation and agility in the quality assurance process.

I shall write some more associating the technology of testing and automation concepts with our daily lives. But aren't we all testers here?