Black Ops Testing Workshop

Workshop description: http://www.agiletestingdays.com/session/black-ops-testing/

Idea: Test an actual application with help of experienced instructors to learn new tools and techniques

Slides: in google drive

Application under Test: http://www.getontracks.org/ - a tool to help with Getting Things Done. (available via bitnami and github - or these here might still work: http://tpl1.com/atd.htm ) 

Key Learnings:

• Exploratory testing is necessary
  • Even if you have a high coverage by automated (ui) tests, there still can be a lot of bugs not revealed that way
• Exploratory testing seems to be faster than "traditional" approaches (creating test plans, matrixes, equivalence classes etc.)
• Efficiency & inspect and adept are key aspects of exploratory testing
  • In an agile world you do not have much time to test your software
  • inefficient testing means wasting time, which means missing bugs
  • Reflect on your actions on a regular basis and see what you can improve (e.g. make screen recordings and see how much time you spend for what) 
• Pair testing might show to be very useful
  • Testing approaches seem to vary even more than programming approaches - this might be a good opportunity for learning from others
• Mindmaps are useful to record test ideas for a test session
• interesting test aspects / starting points:
  • Date, Time & Time Zone
    • challenge: How to (manually) test time based operations / events?
      • possible solution: modify system / database time
    • Date borders and crossing them by changing time zones
  • Internationalization (which is often not tested well)
  • Security: Injections ( SQL-Injection , XSS )
  • Old Bugs
  • Known unstable areas
  • Specifics of libraries and technologies employed
    • eg. Race conditions in AJAX
  • encoding
• cucumber tests written by somebody else are NOT easy to understand, in case you want to know what's going on on a technical level

Techniques:

• Six Questions:
  • What? Why? When? How? Where? Who?
• Intelligence Cycle (military style)
• Talk - Think - Talk Cycle
• Observe Data
  • Database
  • Cookies
  • Logs
  • Network traffic
  • Queues
• Different approaches:
  • explore existing functionality vs. test expectations
• Do sessions around topics
  • which could be technical, functional, bug-related ...
• Identify test-impediments and find solutions
  • e.g. Google Calendar integration can not be properly tested, because robots.txt is not configured properly

Tools:

• a ruby on rails app uses
  • ruby (language interpreter),
  • gem (package management),
  • rake (make for ruby) and
  • bundler (package management, probably cooler than gem  )
• testing tools
  • Firebug (Firefox)
  • Fiddler
  • JMeter
  • Burpsuite
  • EditThisCookie (Chrome)
  • CookieManager (Firefox)
  • Devtools (from browser)
  • Silenium Webdriver
  • GEB
  • cucumber
  • NetworkLink Conditioner
  • Tail / Grep
  • curl / HTTPie
  • Postman (Chrome, supports http call record & playback!)
  • Bugmagnet
  • Mindmap
    • mindmup
    • xmind
  • Charles
  • wireshark
  • scrivener (note taking)
• Infrastructure
  
  • bitnami (Cloud environment similar to heroku)

Further Thoughts:

• How do we employ exploratory testing in our teams?