Bug Hunting Reporting

PRESENTATION
My name is Maria Sobal
Senior Software testing Engineer
7+ years in Software Testing

AGENDA
Bug Definition
Bug Statistics
Bug Life Cycle
Bug Hunting
Troubleshooting

What is bug or defect ?
BUG DEFINITION

BUG DEFINITION
Bugs in requirements
Functional bugs in application
When requirement is incorrect, unambiguous, inconsistent or incomplete
When requirement contradict to business flow in case when it exists
Nonconformance to requirements
Something that does not correspond to valid Customer’s expectations/common sense that are assumed but may be not described in product requirements.

WHY DO WE NEED TO REPORT BUGS?

BUG REPORT: MANDATORY FIELDS
Mandatory bug report fields:
Summary
Severity
Priority
Description
Steps to Reproduce
Actual and Expected Results
Attachment

What additional fields may be present in bug report?
Issue Type/ Testing types like regression issues/New Feature/ Configuration/ Integration/etc.
Product component/ area where defect was found
Environments/ Testing phase like QA, DEV, UAT, PROD
Found/ Fix Version
Sprint/Release where defect was found or should be fixed
…
These fields are very useful for bug analysis!
ADDITIONAL BUG REPORT FIELDS

BUG STATISTICS AND METRICS
Number of defects by priority and status to evaluate product quality and product readiness for delivery

OVERALL NUMBER OF DEFECTS BY PRIORITY AND STATUS
should be 0-1
How many blockers and critical defects in comparison with other defects?
better when it is less
better as less as possible
How many blocker/critical defects are unresolved?
How many unresolved defects in comparison with total number?

NUMBER OF DEFECTS BY PRIORITY AND STATUS BY SPRINT
Summary:
6 unresolved Critical and Major defects are related to user story-XX that was delivered later and testing is still incomplete. We don’t recommend to delivery it for UAT
8 regression defects are not critical and may be fixed during next sprints
4 remaining minor defects are related to user story-YY, ZZ. Please, include these defects into UAT notification for customer.

BUG STATISTICS AND METRICS
Defect distribution by components/ product areas to define most stable components or most “buggy” areas that requires more testing

DEFECT STATISTICS BY COMPONENTS
What components/functional areas are more stable and what still have a lot of issues?

BUG LIFE CYCLE
Who assign defects? – tester? Dev lead? Triage Meeting
What is procedure? To provide screenshots? Close or should be retested by client?
Clarify the reason for each defect!
Try to analyze and define type of defects that deferred

REOPENED AND DECLINED BUGS
It’s not a bug!
Bug isn’t fixed!

REOPENED AND DECLINED BUGS: VALID AND INVALID REASONS
Bug could be reopened because:
Developer’s mistake;
Bug report includes not all details of issue reproducing, some conditions were missed;
Bug is being verified on the environment where bug wasn’t deployed;
Clarify issue with dev and check on right environment.
Bug could be declined because:
Tester’s mistake – incorrect understanding of requirements;
User’s won’t fixed it -> move to deferred state;
Bug was fixed with other bugs/stories and now doesn’t reproduce;
Bug should be closed as Fixed;
Bug was reproduced because of environment issues;
Bug should be closed with resolution/comment ‘Environment issue’.

NOT A BUG
Bug could be converted to another issue type:
Reported bug contains scenario that was not described in requirements;
Bug is converted to story;
Reported bug contains contradicts to requirements - now customer would like to change the behaviour;
Bug is converted to change request.

BUG HUNTING
If you can’t reproduce a bug there is a condition that you are missing.

FACTORS: TIME
Time – delayed
First time
Specific time
Speed

FACTORS: IPHONE TIME EXAMPLE
Preconditions:
Iphone 64-bit iOS 9.3.0 and less
STR:
1. Set date = 01/01/1970 01:00:00 in
UTC +1:30 and more
2. Restart handset
Actual result:
Apple logo is displayed all the time

FACTORS: INVISIBLE FACTORS
Race
Interaction
Load

FACTORS: INVISIBLE FACTORS EXAMPLE

FACTORS: WORLD AROUND
Software Configuration
Hardware Environment
Invalid Data

FACTORS: MEMORY OF THE PAST
Browser cashe
Web server cashe

FACTORS: OPENCART CASH EXAMPLE
Steps to Reproduce:
1. Make any changes
(for ex. Set any module or make any
code changes) by user admin in shop modifier
2. Clear cash by user Admin in shop modifier
3. Try to order any product in shop by Admin and customer users
Actual result:
Product is ordered admin
‘Website is under construction’ message is displayed for customer

BUG HUNTING: RECOMMENDATIONS
General:
Write down all details right away (you do not know, which detail is important right now) and take screenshot
Write down what you tried (even if it did not help)
Create a list of factors that may be important
Write ALL results down
Contact developer
Stop once you tried all ideas and wait for the next time happens
Technical:
Restart
Collect and analyze logs
What will happen to the following bugs:
- 1-st time
- resources taken
- data corrupted
Clean cashe (web server, browser)

TROUBLESHOOTING
You send API request and do not get answer. What will you check?
If there is error response message in console
if server is available
if ports are available
correctness of requested method and parameters
If other requests come to same server
if HTTP operation (GET, PUT, DELETE and POST) is correct

TROUBLESHOOTING
You open web page but got error message.
Based on error message make a decision about the problem source:
404 Not found - requested URL was not found on this server
403 Forbidden - client does not have permission to get URL from requested server
What will you do?
Check if web page address is incorrect (type or copy the correct URL)
check from another networks
check in other browsers,
Ask your teammate to check on his environment
Verify your permissions
Check your certificate

TROUBLESHOOTING
Check the following:
Other dates
Other duration
Other employee
Other type