How to do Estimating Testing effort ?
Time Estimation method for Testing Process
Note : folloing method is based on use case driven specification.
Step 1 : count number of use cases (NUC) of system
step 2 : Set Avg Time Test Cases(ATTC) as per test plan
step 3 : Estimate total number of test cases (NTC)
Total number of test cases = Number of usecases X Avg testcases per a use case
Step 4 : Set Avg Execution Time (AET) per a test case (idelly 15 min depends on your system)
Step 5 : Calculate Total Execution Time (TET)
TET = Total number of test cases * AET
Step 6 : Calculate Test Case Creation Time (TCCT)
useually we will take 1.5 times of TET as TCCT
TCCT = 1.5 * TET
Step 7 : Time for ReTest Case Execution (RTCE) this is for retesting
useually we take 0.5 times of TET
RTCE = 0.5 * TET
Step 8 : Set Report generation Time (RGT
usually we take 0.2 times of TET
RGT = 0.2 * TET
Step 9 : Set Test Environment Setup Time (TEST)
it also depends on test plan
Step 10 : Total Estimation time = TET + TCCT+ RTCE + RGT + TEST + some buffer...;)
Example
Total No of use cases (NUC) : 227
Average test cases per Use cases(AET) : 10
Estimated Test cases(NTC) : 227 * 10 = 2270
Time estimation execution (TET) : 2270/4 = 567.5 hr
Time for creating testcases (TCCT) : 567.5*4/3 = 756.6 hr
Time for retesting (RTCE) : 567.5/2 = 283.75 hr
Report Generation(RGT) = 100 hr
Test Environment Setup Time(TEST) = 20 hr.
-------------------
Total Hrs 1727.85 + buffer
-------------------
here 4 means Number of testcases executed per hour
i.e 15 min will take for execution of each test case
What is the purpose of test strategy?
Reason number 1: The number one reason of writing a test strategy document is to "have" a signed, sealed, and delivered, FDA (or FAA) approved document, where the document includes a written testing methodology, test plan, and test cases.
Reason number 2: Having a test strategy does satisfy one important step in the software testing process.
Reason number 3: The test strategy document tells us how the software product will be tested.
Reason number 4: The creation of a test strategy document presents an opportunity to review the test plan with the project team.
Reason number 5: The test strategy document describes the roles, responsibilities, and the resources required for the test and schedule constraints.
Reason number 6: When we create a test strategy document, we have to put into writing any testing issues requiring resolution (and usually this means additional negotiation at the project management level).
Reason number 7: The test strategy is decided first, before lower level decisions are made on the test plan, test design, and other testing issues.
What's Quality Approach document? what should be the contents and things like that...
Answer1:
you should start thinking from your company business type, and according to it define different processes for your organization. like procurment, CM etc
Then think over different matrices you will be calculating for each process, and define them with formula, the kind of analysis will be doing and when shall the red flag to be raised,
Decide on your audit policies frequencies etc. Think on the change control board if any process needs modification.
Answer2:
By defining the process i mean the structured collection of practices that describe the characteristics of the work and its quality. writting process means creating a system with which every one will work, the benefits of it are like common language and a shared vision across organization, its will be a frame work for prioritizing actions.
From implementation point of view first you need to break the complete life cycle of your product in diffrent meaningful steps, and setting the goals for each phase.
you can create different document templates which every one shall follow, Define the dependencies among different groups for each project, Define risks for each project and what is mitigation plan for each risk. etc
You can read the CMMI model, customize that as per your organization goal. for a start up company As per my personal opinion, its better to define and reach at the process for Level 3 First and then go for level 5.
What does a test strategy document contain?
The test strategy document contains test cases, conditions, the test environment, a list of related tasks, pass/fail criteria and risk assessment. The test strategy document is a formal description of how a software product will be tested. What is the test strategy document developed for? It is developed for all levels of testing, as required. How is it written, and who writes it? It is the test team that analyzes the requirements, writes the test strategy, and reviews the plan with the project team.
Why Q/A should not report to development?
Based on research from the Quality Assurance Institute, the percent of quality groups in each location is noted,
50% - reports to Senior IT Manager - This is the best positioning because it gives the Quality Manager immediate access to the IT Manager to discuss and promote Quality issues, when the quality manager reports elsewhere, quality issues may not be raised to the appropriate level or receive the necessary action.
25% - reports to Manager of systems/programming
15 % reports to Manger oprerations.
10 % outside IT function.
Which of the following statements about Regression statements are true?
1---Regression testing must consist of a fixed set of tests to create a base line
2---Regression tests should be used to detect defects in new feature
3---Regression testing can be run on every build
4--- Regression testing should be targeted areas of high risk and known code change
5---Regression testing when automated, is highly effective in preventing defects.
Answer1:
1---Regression testing must consist of a fixed set of tests to create a base line
Don't think it is true as a "must" -- it
depends on whether your regression testing style involves repeating identical tests or redoing testing in previously tested areas with similar tests or tests that address the same risks. For example, some people do regression testing with tests whose specific parameters are determined randomly. They broaden the set of values they test while achieving essentially the same testing. Second example--some regression test suites include random stringing together of test cases (they include load testing and duration testing in their regression series, reporting their results as part of the assessment of each build). Depending on your theory of the _point_ of regression testing, these may or may not be entirely valid regression tests.
2---Regression tests should be used to detect defects in new feature
How do you create new regression tests? Should you design new tests as standalone, or should you develop a strategy in which the tests you use for bug-hunting are designed to be reusable as regression tests? If the latter, and I have certainly heard some skilled testers argue that the latter approach worked well in their sistuation, then #2 is sometimes true.
3---Regression testing can be run on every build
This is true, though it might be silly and a big waste of time.
4--- Regression testing should be targeted areas of high risk and known code change
Hmmm, there's a area of computer science called program slicing and one of the objectives of this class of work is to figure out how to restrict the regression test suite to a smaller number of tests, which test only those things that might have been impacted by a change. Bob Glass has criticized the results of some of this work, but if #4 is false, some Ph.D.'s and big research grants should be retracted.
5---Regression testing when automated, is highly effective in preventing defects.
Unit-level automated regression testing is highly effective in preventing defects--read up on test-driven development.
Answer2:
Let me explain why I think 2 & 5 are false
2---Regression tests should be used to detect defects in new feature
Since regression tests only address existing features and functionality, it can't find defects in new features. It can only find where existing features and functionality have been broken by changes.
5---Regression testing when automated, is highly effective in preventing defects.
Since no tests prevent defects, they only find them, it's impossible to prevent defects with a regression test. I will add, however, that if a developer can use an automated regression test to test their own code before submitting it to the code repository (say in the form a series of unit tests coupled to a library, etc.) then you could in some way prevent defects with a regression test.
I also don't like 1- and 4. 1- since a regression test suite grows as the product does. Therefore the tests are not fixed. 4- because a regression test tests the whole application, not just a targeted area. In the past, I have used the concept of test depth (level 1 being the basic regression tests--higher number reflect additional functionality) so you could run a level one regression on the whole program but do level three on the transport layer "because we've updated the library". T
an automated set of tests would be the most likely way to make 3- a possibility. It is unlikely that with daily builds, as many companies run their build process, that anything short of an automated regression test suite would be able to be run daily with any efficacy. if the builds were weekly, then a manual regression test would be likely.
Answer3:
As per the difinition of regression testing and actual workaround if you have to have answer this question then option 3 & 4 is the best choice among all.The reason behind it is :
3---Regression testing can be run on every build It is a normal phenomenon if there is build coming on weekly basis or it is a RC build.Since,there is nothing mention about daily build ,only thing mention is every build so it can be correct.
4---Regression testing should be targeted areas of high risk and known code change This is also true in most of the situation,it is not universally true but in certain condition where there is code change and the related modules are only tested in regression automation rather than whole code.
5 is not true coz in regression we detect the defect not prevent normally.
How do you execute tests?
Execution of tests is completed by following the test documents in a methodical manner. As each test procedure is performed, an entry is recorded in a test execution log to note the execution of the procedure and whether or not the test procedure uncovered any defects. Checkpoint meetings are held throughout the execution phase. Checkpoint meetings are held daily, if required, to address and discuss testing issues, status and activities.
* The output from the execution of test procedures is known as test results. Test results are evaluated by test engineers to determine whether the expected results have been obtained. All discrepancies/anomalies are logged and discussed with the software team lead, hardware test lead, programmers, software engineers and documented for further investigation and resolution. Every company has a different process for logging and reporting bugs/defects uncovered during testing.
* A pass/fail criteria is used to determine the severity of a problem, and results are recorded in a test summary report. The severity of a problem, found during system testing, is defined in accordance to the customer's risk assessment and recorded in their selected tracking tool.
* Proposed fixes are delivered to the testing environment, based on the severity of the problem. Fixes are regression tested and flawless fixes are migrated to a new baseline. Following completion of the test, members of the test team prepare a summary report. The summary report is reviewed by the Project Manager, Software QA Manager and/or Test Team Lead.
* After a particular level of testing has been certified, it is the responsibility of the Configuration Manager to coordinate the migration of the release software components to the next test level, as documented in the Configuration Management Plan. The software is only migrated to the production environment after the Project Manager's formal acceptance.
* The test team reviews test document problems identified during testing, and update documents where appropriate.
Inputs for this process:
* Approved test documents, e.g. Test Plan, Test Cases, Test Procedures.
* Test tools, including automated test tools, if applicable.
* Developed scripts.
* Changes to the design, i.e. Change Request Documents.
* Test data.
* Availability of the test team and project team.
* General and Detailed Design Documents, i.e. Requirements Document, Software Design Document.
* A software that has been migrated to the test environment, i.e. unit tested code, via the Configuration/Build Manager.
* Test Readiness Document.
* Document Updates.
Outputs for this process:
* Log and summary of the test results. Usually this is part of the Test Report. This needs to be approved and signed-off with revised testing deliverables.
* Changes to the code, also known as test fixes.
* Test document problems uncovered as a result of testing. Examples are Requirements document and Design Document problems.
* Reports on software design issues, given to software developers for correction. Examples are bug reports on code issues.
* Formal record of test incidents, usually part of problem tracking.
* Base-lined package, also known as tested source and object code, ready for migration to the next level.
What is a requirements test matrix?
The requirements test matrix is a project management tool for tracking and managing testing efforts, based on requirements, throughout the project's life cycle.
The requirements test matrix is a table, where requirement descriptions are put in the rows of the table, and the descriptions of testing efforts are put in the column headers of the same table.
The requirements test matrix is similar to the requirements traceability matrix, which is a representation of user requirements aligned against system functionality. The requirements traceability matrix ensures that all user requirements are addressed by the system integration team and implemented in the system integration effort.
The requirements test matrix is a representation of user requirements aligned against system testing. Similarly to the requirements traceability matrix, the requirements test matrix ensures that all user requirements are addressed by the system test team and implemented in the system testing effort.
Can you give me a requirements test matrix template?
For a requirements test matrix template, you want to visualize a simple, basic table that you create for cross-referencing purposes.
Step 1: Find out how many requirements you have.
Step 2: Find out how many test cases you have.
Step 3: Based on these numbers, create a basic table. If you have a list of 90 requirements and 360 test cases, you want to create a table of 91 rows and 361 columns.
Step 4: Focus on the the first column of your table. One by one, copy all your 90 requirement numbers, and paste them into rows 2 through 91 of the table.
Step 5: Now switch your attention to the the first row of the table. One by one, copy all your 360 test case numbers, and paste them into columns 2 through 361 of the table.
Step 6: Examine each of your 360 test cases, and, one by one, determine which of the 90 requirements they satisfy. If, for the sake of this example, test case number 64 satisfies requirement number 12, then put a large "X" into cell 13-65 of your table... and then you have it; you have just created a requirements test matrix template that you can use for cross-referencing purposes.
0 comments:
Post a Comment