Issue #159 : Software Testing Notes
AI for QA: Hype vs Reality
Hello there! 👋
Welcome to the 159th edition of Software Testing Notes, a weekly newsletter featuring must-read content on Software Testing. I hope this week has been good for you so far.
This week, we will explore:
How to systematically evaluate large language models (LLMs)
Some advice for newcomers in the software testing
AI for QA: Hype vs Reality
how to interpret Chrome User Experience Report (CrUX)
Test API for CORS Enablement & Find Issues
🙏 Featured
2138+ new jobs for software test engineers
TestDevJobs is the single place to find all cool jobs in Software Testing. Whether you're looking for a new Automation role, framework specific jobs , Location wise or remote jobs… We got you covered.
📚 Testing
My Advise to a Software Test Engineer by Ravisuriya Eswara
For all newcomers in the software testing, Ravisuriya Eswara advises to integrate programming into your daily practice, applying critical thinking and other essential skills to improve your testing effectiveness.
Exploratory testing: Free thinking by Melissa Fisher
Free-thinking and curiosity-driven testing approach are the key to exploratory testing. As different testers have different styles, Melissa Fisher suggests the key to effective testing is curiosity, continuous learning, and finding a method that works best for the individual testers.
AI for QA: Hype vs Reality by Igor Goldshmidt
AI is neither a magical solution nor a useless trend but a tool that enhances, rather than replaces, human testers. Igor Goldshmidt writes about AI’s strengths in automation, defect prediction, and test data management while cautioning against blindly trusting AI-generated results or using AI without a clear strategy.
Common Mobile App Testing Mistakes to Avoid by Aya Akl
Aya Akl discusses common mobile app testing mistakes, conducting accessibility testing, handling permissions correctly, ensuring proper error handling, and testing push notifications and interruptions.
🔍 Software Testing
⚙️ Automation
Automatically Testing an LLM: Evaluating Response Quality with Semantic Scores by Jean-Christophe Jamet
The article explores how QA engineers can systematically evaluate large language models (LLMs) using automated metrics. Jean-Christophe Jamet introduces an automated evaluation pipeline that compares LLM-generated responses with human references, ensuring scalability, consistency, and rapid iteration in testing.
Collect All URLs Visited During Cypress Test by Gleb Bahmutov
The article explores how to track all URLs visited during a Cypress test, beyond just those explicitly visited using cy.visit(). Gleb Bahmutov demonstrates how to capture navigation events using Cypress hooks and events.
Furthermore, also read how to Run Cypress Tests For The Given URL
There are advantages to using the API to tear down automated end-to-end tests by Mike Harris
Mike Harris argues that using the application's API for teardown, rather than interacting with the UI, can significantly reduce test execution time, improve reliability, make the process less brittle and helps test automation engineers become more familiar with the application's backend, improving overall testing capabilities.
Understanding timeouts in WebdriverIO by Filip Hric
Regardless of framework you use, in UI testing, timeouts play important part in handling randomness when elements appear and interact. Filip Hric writes more about it in this article and also explains different types of timeouts in WebdriverIO, how to configure them, and best practices to avoid unnecessarily long test execution times.
How Terraform Tests Saved a Prod Deployment by Rafaela Azevedo
Testing should be integrated in all part of SDLC, including deployment. Rafaela Azevedo explains key testing features of testing Terraform like plan commands, assertions, variable overrides, and mock providers to catch issues early and avoid production disasters.
🔍 Test Automation
💨 Performance
Optimising for High Latency Environments by Harry Roberts
This is a really fantastic article discusses how to interpret Chrome User Experience Report (CrUX). provides valuable insights into network latency and its impact on user experience. Harry Roberts explains that most web performance issues are latency-bound rather than bandwidth-bound and suggests strategies to mitigate latency.
A Day in the Life of a Performance Engineer: More Than Just Running Tests by Yam Shal-Bar
Performance engineering goes beyond just running load tests. Yam Shal-Bar shares insights from their own journey, advocating for continuous performance improvement.
🔍 Performance Testing
🌞 Accessibility
How to Perform Comprehensive ADA Testing by Caitlin de Rooij
Caitlin de Rooij outlines practical approaches combining automated and manual testing, integrating accessibility into development workflows, and adhering to WCAG standards.
🔍 Accessibility Testing
🔒 Security
Test API for CORS Enablement & Find Issues by Ankit Jain
Developers often find CORS frustrating due to cryptic errors and debugging challenges. Ankit Jain explains Cross-Origin Resource Sharing (CORS) and provides practical guidance on when CORS is needed, how to configure it correctly, and how to test and fix CORS issues effectively.
A Security Testing System for LLMs that Anyone Can Use by Ran Bar-Zik
Ran Bar-Zik explores the vulnerabilities of AI models, specifically LLMs, to adversarial attacks and the necessity of penetration testing. The article discusses real-world attack techniques such as prompt injection and social influence strategies, highlighting how attackers can bypass security mechanisms through sophisticated queries.
Static Application Security Testing (SAST) by Eugene Voskoley
In this article, Eugene Voskoley share two key techniques: Runtime Application Self-Protection (RASP), which safeguards applications during execution, and Static Application Security Testing (SAST), which analyzes code for vulnerabilities before deployment.
🔍 Security Testing
🛠️ Resources & Tools
🎭 playwright-cli-select — is a command-line tool crafted by Dennis B. that simplifies selecting and running specific Playwright tests interactively.
Butch Mayhew has wrote about tool’s usage in TOOL: Playwright-Cli-Select For Quick Targeted Test Runs via CLI
📝 List of Software Testers
Do you create content around Software Testing ? Submit yours blog details here and I will add it to the list.
🎁 Bonus Content
📌 OTHER INTERESTING STUFF
⭐ LAST WEEK'S MOST READ
What is APIs Exploratory Testing and How it works? by Japneet Sachdeva
The Full Software Testing Chain: A Journey, Not Just a Step by Elena Voevodkina
Bruno + GitHub = Ultimate API Test Automation by Giridhar Rajkumar
😂 And Finally,
Just testers messing with Developers 🤣
📨 Send Me Your Articles, Tutorials, Tools!
Wrote something? Send links via Direct Message on Twitter @thetestingkit (details here). If you have any suggestions for improvement or corrections, feel free to reply to this email.
Thanks to everyone for subscribing and reading!
Happy Testing!
Pritesh(@priteshusdadiya)