Showing posts with label Testing. Show all posts
Showing posts with label Testing. Show all posts

Friday, March 2, 2012

Introducing Selenium IDE, an open source automation testing tool

Selenium, the popular open-source automation test tool for Web-based software, comes in different flavors. This tutorial series started by introducing Selenium Remote Control (Selenium RC). However using Selenium RC requires knowledge of special commands, understanding the elements of an HTML page, and a fair amount of programming. Wouldn't it be nice to be able to record your own actions and convert those to Selenium commands, or, better yet, to record commands, insert some assertions, and just run the script back as a test, without all that complex programming?
There is just such a tool. It's called Selenium IDE.
A simple script recorded with Selenium IDE
IDE is an acronym for Integrated Development Environment. Using Selenium IDE, instead of having a program over here talking to a remote controller over here that drives a browser in a third place, the entire application runs inside the web browser. To do that, Selenium IDE was built as a browser plug-in for FireFox. Let's take a look at the software in-action, after I have recorded a simple script:
This will all be explained in more detail below, but I wanted to give you a quick look at what your Selenium RC screen will look like when we're through. To record this, all I did was start Selenium IDE, type http://techtarget.searchsoftwarequality.com (enter), and use the site to search for my tutorial on Selenium RC. With the test recorded, I can replay that test by clicking the green go button. The 'Base URL' for the test is SearchSoftwareQuality.com so we don't see that mentioned in the open method, but above in the Base URL field. With Web testing, often it can be tricky to obtain an element's id. However, Selenium is able to capture the ids in the target field as the tool is recording actions taking place in the Firefox browser.
One thing the test doesn't do (yet) is actually confirm any text on the screen -- it drives the browser, but doesn't inspect it. We'll talk about that later.
Before we can do that, let's talk about how to install Selenium. Then I'll cover more on how to use it effectively, and a little bit on how to save, re-use tests, and make test suites with the tool.
Installing Selenium IDE
We'll start by going to the openQA download page, which you can find at http://seleniumhq.org/download and clicking on the link to download Selenium IDE. That will take us to the Selenium IDE Add-On Installer:


Now click the "Install Now" button to get started, followed by "Restart FireFox."
You should see a note that the new Add-on was installed, and that's it. You can start Selenium IDE by clicking Tools->Selenium IDE from your Firefox browser.
It's that simple.
Now let's walk through creating that first test together in more detail; you may want to print this page out and follow along.
Recording an Effective Test

  1. Open the IDE through Tools->Selenium IDE.
  2. When the window pops up, mouse over the red button on the top right. It should say "Now recording." If not, click the button to start recording.
  3. In your browser, go to www.searchsoftwarequality.com.
  4. Look at the title of the web page; the words in the blue bar at the very top-left of the screen.
  5. Let's assert that title comes up when we start testing. So click on the white box to add a command:


  • 1.)Then click on the Command textbox and type in "assertTitle" into the command window and "Software quality management, testing and QA resources - SearchSoftwareQuality.com" (which should be showing in the Firefox Title Bar) into the Target textbox.


    2.)Now go back to the browser, click on the search textbox. Type in "Heusser Selenium RC" and click search.
    3.)This brings us to a search results page. In steps 9 and 10, we'll add in checks for two recent Selenium articles that I wrote. 
    4.)
    Click on the next blank command line in the Command Table. This is necessary in order to add a new command. Add an assertElementPresent command for "link=Tutorial: Installing and running Selenium-RC in Perl"
    5.)Add a second assertElementPresent command for "link=Getting started with Selenium Remote Control - Part 2."
    6.)Go back to the main browser window and click that first link to go to "Tutorial: Installing and running Selenium-RC in Perl." The action of clicking the link will both take us to that actual page, and "record" the event in Selenium IDE, adding it automatically to the test case.
    Now we're on a browser page for the first tutorial. Let's make sure at least some actual text from the article appears. To do that: 

    7.)
    In the Selenium IDE window, click on the next blank command line. Then enter the command "assertTextPresent" and add "Used in conjunction with Selenium's online tutorials" added to the Target text box.
  • If you followed these directions, click the red button to stop recording, and you should have something like this:

    The slash at the top is actually a command to open the 'Base URL,' which we can see at the very top is our website. This is a sort of shorthand, so that we can skip the http colon slash slash domain-name dance for every open command. Without the full path, Selenium assumes it is part of the Base URL. In our example, we typed in SearchSoftwareQuality.com and hit enter, causing that Base URL to be set.

    Now that the test is recorded, you can either click one of the green play buttons (we only have one test right now; the test is the suite) or click at the top of the command list and click the step in button to run one test at a time. If you get errors, it's likely that you called assertElementPresent when it should have been assertTextPresent, or you made a typo in the check or command. You can compare your test to this one to look for differences.
    In addition to checking for text and links, Selenium has a host of commands. Most of them, such as clicks, checks, and selects, can be recorded automatically with the IDE. One popular command is waitForElementPresent (element_name) which can be especially important when waiting for Javascript elements to populate. The commands are extensive; this list of common Selenium commands should get you started.
    Also, if you end at a different place than you start, make sure you begin with an "open" command with a location of "/" and that the base URL is the place to start. If the test misses this for some reason, just add the command at the bottom and drag it to the top.
    Saving Tests, creating and re-suing suites
    Sooner or later, you'll want to close your browser, and you'll likely want to get your file back next time you open it. From Selenium IDE use the File Menu to Save Test Case As and specify a filename. This will save to your hard drive for next time. Another option is to create a New Test Case and save that. Having more than one test case open at once shows us the test suite interface. This is just like the test case interface, except it shows the name of all the test cases loaded at left:

    We can now run our tests collectively and save them as a suite. While for smaller apps you may be able to test the entire application in one suite, you'll likely want to break them up for larger applications. In that case, I suggest a test for each minor feature and a suite for each major feature, with each suite saved in a directory. That way, you can test major pieces one at a time. With multiple testers, you can put the entire system under version control, just like software code, and have multiple people working on tests at the same time.
    This is just a quick introduction, but you can find a great deal of detail in the Selenium IDE documentation on-line. I've tried to hit the highlights and get you started quickly. Now I could end the tutorial and ride off into the sunset, but if you'll allow me, I've got a little bit more advice to give.
    Beyond Selenium IDE

    Once the thrill runs off of working with the IDE, you're faced with a few realities. The tool generally takes up your whole browser, and while it is faster, it really isn't that much faster than testing by hand. It always does the same thing, over and over again, and it doesn't have any of the features we look for in a programming language such as variables, looping and abstraction. So if you've got a technical team interested in doing some serious automation testing, I generally suggest moving from Selenium IDE to Selenium Remote Control, which can be programmed. If that tickles your fancy, read the two Selenium RC tutorials that we searched for in our test case. You have the basics. Now it's time to move to the next level.

10 software testing traps

Everyone at some point in their careers faces difficulties. The problem could be not having enough resources or time to complete projects. It could be working with people who don't think your job is important. It could be lack of consideration and respect from managers and those who report to you.
Software testers aren't exempt from this. But as Jon Bach pointed out in his session titled "Top 10 tendencies that trap testers" that he presented at StarEast a couple weeks ago, software testers often do things that affect their work and how co-workers think about them.
Bach, manager for corporate intellect and technical solutions at Quardev Inc., reviewed 10 tendencies he's observed in software testers that often trap them and limit how well they do their job. "If you want to avoid traps because you want to earn credibility, want others to be confident in you, and want respect, then you need to be cautious, be curious and think critically," he said.
Here's a look at what Bach considers the top 10 traps and how to remedy them:

10. Stakeholder trust: This is the tendency to search for or interpret information in a way that confirms your preconceptions. But what if a person's preconceptions are wrong? You can't automatically believe or trust people when they say, "Don't worry about it," "It's fixed," or "I'll take care of it."
Remedies include learning to trust but then verify that what the person says is correct. Testers should also think about the tradeoffs compared with opportunity costs, as well as consider what else might be broken.

9. Compartmental thinking: This means thinking only about what's in front of you. Remedies include thinking about opposite dimensions -- light vs. dark, small vs. big, fast vs. slow, etc. Testers can also exercise a brainstorm tactic called "brute cause analysis" in which one person thinks of an error and then another person thinks of a function.
8. Definition faith: Testers can't assume they know what is being asked of them. For example, if someone says, "Test this," what do you need to test for? The same goes for the term "state." There are many options.
What testers need to do is push back a little and make sure they understand what is expected of them. Is there another interpretation? What is their mission? What is the test meant to find?
7. In-attentional blindness: This is the inability to perceive features in a visual scene when the observer is not attending to them. An example of this is focusing on one thing or being distracted by something while other things go on around you, such as a magic trick.
To remedy this, testers need to increase their situational awareness. Manage the scope and depth of their attention. Look for different things and look at different things in different ways. 

6. Dismissed confusion: If a tester is confused by what he's seeing, he may think, "It's probably working; it's just something I'm doing wrong." He needs to instead have confidence in his confusion. Fresh eyes find bugs, and a tester's confusion is more than likely picking up on something that's wrong.

5. Performance paralysis: This happens when testers are overwhelmed by the number of choices to begin testing. To help get over this, testers can look at the bug database, talk with other testers (paired testing), talk with programmers, look at the design documents, search the Web and review user documentation.
Bach also suggests trying a PIQ (Plunge In/Quit) cycle -- plunge in and just do anything. If it's too hard, then stop and go back to it. Do this several times -- plunge in, quit; plunge in, quit; plunge in, quit. Testers can also try using a test planning checklist and a test plan evaluation.

4. Function fanaticism: Don't get wrapped up in functional testing. Yes, those types of tests are important, but don't forget about structure tests, data tests, platform tests, operations tests and time tests. To get out of that trap, use or invest in your own heuristics.

3. Yourself, untested: Testers tend not to scrutinize their own work. They can become complacent about their testing knowledge, they stop learning more about testing, they have malformed tests and misleading bug titles. Testers need to take a step back and test their testing.

2. Bad oracles: An oracle is a principle or mechanism used to recognize a problem. You could be following a bad one. For example, how do you know a bug is a bug? Testers should file issues as well as bugs, and they should mention in passing to people involved that things might be bugs.

1. Premature celebration: You may think you've found the culprit -- the show-stopping bug. However, another bug may be one step away. To avoid this, testers should "jump to conjecture, not conclusions." They should find the fault, not just the failure.
Testers can also follow the "rumble strip" heuristic. The rumble strip runs along most highways. It's a warning that your car is heading into danger if it continues on its current path. Bach says, "The rumble strip heuristic in testing says that when you're testing and you see the product do strange things (especially when it wasn't doing those strange things just before) that could indicate a big disaster is about to happen."

Tips on performance testing and optimization

The purpose of this document is to explain how to go about performing scalability testing, performance testing, and optimization, in a typical Java 2 Enterprise Edition (J2EE) environment.
Definitions:
Response Time the time it takes between initial request and complete download of response (rendering of entire web page).
Load a measurement of the usage of the system. A server is said to experience high load when its supported application is being heavily trafficked.
Scalability - A scalable application will has a response time that increases linearly as load increases. Such an application, will be able to process more and more volume by adding more hardware resources in a linear (not exponential) fashion.
Automation testing tools Tools (Silk from Segue Software, WebLoad, etc) used to simulate a user by requesting pages or going through pre-programmed workflow on your site.
Load testing tools Most automation testing tools can also be used as load testing software, like WebLoad. These tools will simulate any number of users using your site and provide you with important data like average response times.
Profiler. A profiler is a program that examines your application as it runs. It provides you with useful run time information such as time spent in particular code blocks, memory / heap utilization, number of instances of particular objects in memory, etc.
A Process for performance testing
1) Functional Testing. Most applications begin tests be first completing functional tests. That is, ensuring that all the usecases / workflow in your application work.
2) Load and Scalability Testing. Load and scalabilty testing has too forms:
  • Test Response time as you increase the size of our database
  • Testing response time as you increase concurrent users
3) Interpreting the results.  After measuring response time at varied database sizes and loads, you can now make interpretations based on the average response time of these tests and the resource utilization of the server during the tests. 4) Optimization.  After identifying problems in the last step, you now interpret the results and track down the problem.
Load and Scalability Testing
The purpose of load and scalability testing is to ensure that your application will have a good response time during peak usage. You can also test how your application will behave over time (as your website contains more and more data in your database).  To begin testing, write some testing scripts that will populate your database with an average amount of data. Run your performance tests, measure your response time. Then populate your database with an extreme amount of data (3 to 4 times more data than you can foresee having in 3 years).  Run your performance tests again. If response times are significantly larger for the second test, then something is wrong.
To run your performance tests, you will want to simulate server usage at different loads. As a rule of thumb, I simulate low load (one to 5 concurrent users), medium load (10-50 concurrent users), high load  (100 concurrent users) and extreme load (1000+ concurrent users).  Note that these numbers are arbitrary and depend on your business needs. Also, simulating 10 concurrent users with load testing software isn't representative of  10 people, since each robot in the load test may wait just milliseconds before hitting the server again. Thus, using a load tester to simulate 10 users is probably more representative of the web surfing patterns of 30-40 people.
Once you have tested at all three load levels, you can now compare average response times to see if your system is scales, that is, if the response time increases linearly.

Interpreting the results
 The fun part of this process is interpreting the results of your load testing.  Let us examine some of the different possibilities:
  1.  Response time increases too much when database is over populated
    Response time should not increase too much if you move from a database with 100 rows in its tables to 50,000. Database indexing technology makes finding a row in a table take a matter of milliseconds, even if there are hundreds of thousands of rows. Thus, if your response time increases too much after moving from a moderately populated database to an over populated database, then you probably haven't indexed your appropriate columns yet.
  2.  Response time increases exponentially as load increases
    If your system becomes un-useable as you increase concurrent users, then your system is not scalable. Interpreting these results are difficult, as the problem could be with hardware, deployment configuration, architecture, etc.  Make sure you watch the server resources during the tests:
    1. Watch memory requirements
    2. Watch CPU usage
      If CPU is over used, need faster processor, or more processors. If the CPU is underused, then the problem is probably input/output (I/O) related. Check your database connections, your running thread count, and the network configuration of your test boxes.
If after checking your configuration, verifying that the slowdown is not a hardware bottleneck, and looking over your architecture for code to optimize, its time to run a code profiler.
Optimization
The database, your architecture, configuration and hardware will need to be optimized. As mentioned in the previous section, the easiest way not to scale is to have a database that isn't tuned. A database administrator (DBA) is always a vital person to have on any dev team, but if you don't have one, here is what you can do:
Look though your EJBs and verify that your database isn't doing linear searches for any of the SQL queries that you have encoded. To do this, copy your SQL from your code and in your database SQL window, run an EXPLAIN clause:
Explain select * from table where tablefield = somevalue
Although the explain syntax differs from database to database, there is always something similar. After running this line of code, your DB will tell you if it is searching an index or a linear search. Make sure you verify that every piece of SQL in your application is using your DBs indexes, and if not, create the indexes.
     
After optimizing the database, and optimizing your hardware configuration (as discussed in the previous section), the next step is optimizing your code, and this is done with a profiler.
A profiler is a program that analyzes your application as it runs. A Profiler provides you with information you could not otherwise get access to, such as:
  1. How many objects of each class are in memory and garbage collection behaviour
    • This information can help you identify classes, which should be pooled.
    • Can help you tune your java heap.
  2. How much time your application is spending in particular classes
      This is the most important feature. Your profiler will point its finger and show you which classes are the bottlenecks.
One such program that really helped me is called Optimize-ItOptimize-It can be used with any java program or any java-based application server.  Configuration with Weblogic is easy, and Optimize-It can be used to profile an application on a remote server.
Optimizing your architecture is extremely project specific, but here are some tips:
  1. Make sure you have minimized your network calls, especially database calls
    • It is better to make one large database call rather than many small ones.
    • Make sure ejbStore isn't storing anything for read only operations.
    • Use Details Objects to get entity bean state.
  2. Make sure to take advantage of caching where possible
    Your app. Server probably allows you to cache entity beans in memory, make sure you take advantage of this, as it will dramatically reduce database calls and speed up data access.
  3. Make sure you are using session beans as a façade to your entity beans.You can encapsulate the workflow of one entire use case in one network call to one method on a session bean (and one transaction).

Conclusion
Performance testing and optimizing your application can be pretty challenging. Luckily, there are tools on the market that can make the process easier.  By using these tools and following the simple steps in this paper, you should be able to effectively track down the bottlenecks in your system.

Thursday, March 1, 2012

Learn Selenium Testing Tool

Selenium is a popular automated web based testing tool.Selenium testing tools provides a rich set of testing functions specifically designed to fulfill needs of testing of a web based application.

Benefits of Selenium

1. Selenium tool is an open source

2. Automated black-box web sites testing tool.

3. Used for acceptance / functional testing.

4. Selenium tests run directly in a browser, just like real users do.

5.Works on multiple browsers Mozilla, Chrome, IE, Opera, Safari

5. Running on multiple browsers, multiple Operating Systems.

6. Ajax testing support

7.Tests Flash Applicatons

8.Can do Database Testing

9. Selenium IDE is a plug-in to Firefox to record and playback tests (like WinRunner, QTP).

10. You can then export the recorded test in most language e.g. HTML, Java, .NET , Perl , Ruby etc.

Read more....