Hello there! 👋
Welcome to the 227th edition of Software Testing Notes, a weekly newsletter featuring must-read content on Software Testing. I hope this week has been good for you so far.
This week, we will explore:
The Thinking Toolkit of a Software Tester
Why Smart Systems Need Stupid Tests
Lessons Learned from Fixing Flaky Tests with Claude
Generate Algorithmically Correct Mock Data in JMeter
Prompt Injection: The Test Case Your AI Feature Is Missing
and more…
📚 Testing
More about the testing, less about testers by Jesper Ottosen
Software testing is changing as developers and AI take on more verification work, but that does not make testing less important. Jesper Ottosen looks at how testing responsibilities are moving away from traditional tester roles and towards the wider development team.
The Thinking Toolkit of a Software Tester by Deepak Karn
Good testing depends on asking the right questions when requirements, risks, and system behaviour are unclear. Deepak Karn presents practical heuristics and mental models that help testers explore unfamiliar software and decide where deeper testing is needed.
Mutation Testing Closes the Trust Gap in AI Generated Code by Josphine J.
AI generated tests can produce impressive coverage without proving that they would catch real defects. Josphine J. uses mutation testing to introduce controlled bugs and measure whether the test suite actually detects them.
Test coverage does not tell you whether a change is safe enough by Jitesh Gosai
High test coverage does not tell you how safely a change can reach production. Jitesh Gosai proposes five factors for assessing release risk with controls that can help to get closer to answer.
Why Smart Systems Need Stupid Tests by Franziska
Complex software can still fail on simple inputs and basic user behaviour. Franziska explores how straightforward tests for missing data, repeated actions, failures, and recovery can expose assumptions that more sophisticated testing may overlook.
🔍 Software Testing
⚙️ Automation
Implementing Page Objects in Playwright by T.J. - Maher, Jr.
Page Objects can keep Playwright tests readable. T.J. - Maher shows how to build a TypeScript LoginPage and separate page interactions from test assertions and test data.
Lessons Learned from Fixing Flaky Tests with Claude by Henrik Warne
Henrik Warne managed to run 1,360 integration tests in parallel and cut execution time from about 40 minutes to 4 minutes, but also exposed test interference and timing problems. This article shows how Claude helped diagnose flaky tests while demonstrating why we still need to understand and verify AI generated fixes.
Why Flaky Tests Are Rarely About the Test by Marx Jenes
A flaky test can be the first sign of a real problem rather than a problem with the test itself. This article examines hidden test dependencies, application race conditions, environment assumptions, and outdated mocks that can cause failures or hide defects.
Test-Driven Review: Reading the Tests First by Dennis Martinez
AI coding agents can produce code that works while still implementing the wrong behaviour. Dennis Martinez argues for reading the generated tests first to check the agent's understanding, identify missing scenarios, and find implementation areas that need closer review.
How to Make Playwright Tests Less Flaky: Practical Rules That Actually Work by Svetlana Tretjakova
Playwright tests become fragile when they rely on fixed delays, implementation details, shared data, or unnecessary browser checks. Svetlana Tretjakova shows how to wait for real application state, use resilient locators, synchronize with network activity, isolate test data, and move suitable checks to the API layer.
🔍 Test Automation
💨 Performance
Stop Using Random Strings in JMeter: Generate Algorithmically Correct Mock Data with Mock Jutsu by Altan Sezer Ayan
Random values can fail validation before a JMeter request reaches the business logic you want to load test. Altan Sezer Ayan has created a utility that generates format and checksum valid data such as IBANs, card numbers, identity numbers, and custom identifiers.
🔍 Performance Testing
🌞 Accessibility
Accessibility Bugs Are Often Architecture Bugs by Niharika P Pujari
We testers often find the same accessibility defects because development teams treat accessibility as a final system check. Niharika P Pujari shares how we can prevent manual rework by having accessibility standards set in architecture itself.
🔍 Accessibility Testing
🔒 Security
Prompt Injection: The Test Case Your AI Feature Is Missing by Alex Tokar
LLM applications can treat malicious instructions inside documents, emails, or web content as trusted instructions. Alex Tokar shows how to test indirect prompt injection, including data exfiltration, unsafe actions and more.
🔍 Security Testing
🛠️ Resources & Tools
OpenQA — The open-source test automation harness. Write tests in plain English, let the agent figure out the selectors. Deterministic regression without the brittleness.
reqres-api-test-framework — Enterprise-grade Playwright + TypeScript API test automation framework, built with a feature-based (vertical-slice) architecture.
Mock Jutsu — generate 390+ format & checksum -valid mock data types directly in test plans. TCKN, IBAN, Luhn, VIN, NHS, SWIFT, MRZ, crypto and more. 6 locales (TR/UK/US/DE/FR/RU). Zero external dependencies.
📝 List of Software Testers
Do you create content around Software Testing ? Submit yours blog details here and I will add it to the list.
🎁 Bonus Content
📌 OTHER INTERESTING STUFF
⭐ LAST WEEK’S MOST READ
5 big ideas from "Taking Testing Seriously" book by Oleksandr Romanov
Mobile App Testing: Real-Device Checklist for QA by Julia Pottinger
😂 And Finally,
📨 Send Me Your Articles, Tutorials, Tools!
Wrote something? Send links via Direct Message on Twitter @thetestingkit (details here). If you have any suggestions for improvement or corrections, feel free to reply to this email.
Thanks to everyone for subscribing and reading!
Happy Testing!
Pritesh(@priteshusdadiya)
Reliably useful issue — the recurring theme across these roundups is that most flaky-test pain isn't in the assertions, it's in test data and environment setup. Half the "intermittent failure" tickets I've seen resolved to two suites sharing a database and disagreeing about who cleans up. The unglamorous fix is usually giving each suite its own throwaway store instead of a shared server nobody owns; for Node projects I've used a file-backed JSON db for that (https://github.com/p32929/fjsondb) — delete the file, state is gone, no container to wait on. Not a production datastore, but for fixtures it removes a whole class of "works on my machine".