Issue #212 : Software Testing Notes
When quantity leads to quality
Hello there! š
Welcome to the 212th edition of Software Testing Notes, a weekly newsletter featuring must-read content on Software Testing. I hope this week has been good for you so far.
This week, we will explore:
Optimizing the wrong part of the testing process
Thoughts on āself-healingā in test automation
Your AI-Generated Code Tests Might Be Lying to You
Understanding TPS in Stress Testing
Playwright Accessibility Testing: What axe and Lighthouse Miss
and moreā¦
āØ Featured
The Selenium creator on AI testing, BiDi, and whatās real
If youāve written a Selenium test, youāve used something Jason Huggins built.
At Breakpoint 2026, heās talking about where testing goes from here: BiDi, the shift-left rethink, and the gap between AI demos and what your sprint actually looks like. Itās free, virtual, and runs May 12ā14.
š Testing
Optimizing the wrong part of the testing process by Chris Kenst
Chris Kenst shared a practical story of how inheriting a massive Cypress UI test suite reveals that automating every test case creates slow, low-signal feedback, and why shifting toward risk-based automation is essential. Itās a relatable rethink of what āgoodā test automation actually looks like.
When quantity leads to quality by Jitesh Gosai
This article reframes the quality versus quantity debate through Agile, arguing that faster feedback and iteration matter more than upfront perfection. Jitesh Gosai shares compelling reminder that in complex software environments, quality emerges from learning loops, not careful planning alone.
Why software testers should write documentation even if no one else reads it by Shawn Vernier
An opinionated take by Shawn Vernier on why testers should write documentation even when nobody reads it, arguing itās really for memory, enforcement, and trust rather than communication.
š Software Testing
āļø Automation
My thoughts on 'self-healing' in test automation by Bas Dijkstra
Bas Dijkstra has written a sharp critique of flaky GUI tests and so-called self-healing tools might mask real issues like poor communication and unnoticed changes. Itās a useful reminder that green tests mean little if they hide problems, and Iām inclined to agree.
From CI Failure to Root Cause in Seconds: MCP for QA Engineers by Irfan MujagiÄ
This is an intriguing glimpse at AI doing real investigative work. In this hands-on guide, Irfan MujagiÄ shares how to use MCP to connect AI directly to CI and shows how test triage can shift from slow manual digging to near-instant root cause analysis.
Your AI-Generated Code Tests Might Be Lying to You by Sriram Rajendran
A sharp argument for mutation testing over code coverage shows how AI-generated code can pass tests while still being wrong, exposing dangerous blind spots. Itās a convincing case that green tests can lie, and Iām starting to see coverage as a false sense of safety.
The Hard Part of AI Evals Isnāt the Tooling by Katja Obring
A thoughtful critique of AI eval tooling argues that while platforms automate measurement, they sidestep the harder question of what quality actually means, especially with biased LLM judges.
Furthermore, Akhona Eland has written about how to Test Your LLM Outputs in pytest (15ms, No API Key)
š Test Automation
šØ Performance
Understanding TPS in Stress Testing (And Why It Matters More Than You Think)
A clear explanation of TPS shows why raw throughput alone can be misleading, and why latency and errors matter just as much when judging performance. Itās a helpful reset if youāve ever chased big numbers.
š Performance Testing
š Accessibility
Playwright Accessibility Testing: What axe and Lighthouse Miss by David Mello
David Mello takes a sobering look at accessibility testing and argues that tools like Lighthouse and axe catch only a fraction of real issues, making āgreen checksā misleading.
Testing A11y Using Cypress And wick-a11y Plugin by Gleb Bahmutov
Gleb Bahmutov shares a quick walkthrough showing how adding wick-a11y to Cypress tests can surface accessibility issues in minutes with minimal setup.
š Accessibility Testing
š ļø Resources & Tools
api Spector ā Local-first API testing tool. Inspect, test and mock APIs. Secrets stay on your machine.
pytest-semantix ā Semantic LLM output testing for pytest. Validate that your LLM outputs mean the right thing ā not just that they match a string.
š List of Software Testers
Do you create content around Software Testing ? Submit yours blog details here and I will add it to the list.
š Bonus Content
š OTHER INTERESTING STUFF
ā LAST WEEKāS MOST READ
Playprom: Turn your Playwright test run into a time-series data by Lutfi Fitroh Hadi
Playwright POM (Without Classes): Make Your Tests Clean & Reusable (The Easy Way) by Aparna Mishra
10 Years in QA: The Journey I Never Expected to Have by Vladimir Josifoski
š And Finally,
Test thy code š¤£
šØ Send Me Your Articles, Tutorials, Tools!
Wrote something? Send links via Direct Message on Twitter @thetestingkit (details here). If you have any suggestions for improvement or corrections, feel free to reply to this email.
Thanks to everyone for subscribing and reading!
Happy Testing!
Pritesh(@priteshusdadiya)