Deciding what to automate – part 1
Find out what to test first..
Often the discussion about what to automate ends with ‘let’s automate everything’…
Around a year into it the team realizes we might never be able to automate everything
Plus the maintenance cost turns out to be much higher than expected
Eventually not seeing enough progress with automation
While this scenario is the result of multiple mistakes along the way,
The first IMHO is not taking the time to formulate a testing strategy first
We can NEVER test everything, let alone automate it
Find out what to test first, more here:
For my tester friends out there struggling and not seeing results, be it learning automation, learning technology or whatever
“Trust the process” – @John Somnez
The results are not up to us. All we have to think about is the little decisions we make in our daily lives
Every person intends to do great things, but then why only a few make it?
The answer I like the most: “little decisions we make every day without even thinking that is a decision”
A lot of small sloppy decisions every day can take you far off course without us even realizing
So trust the process, have faith, in time it will pay off
Every overnight success has a decade long struggle behind it. Little decisions matter. Hang in there..
Parameterizing Jenkins jobs
If you’re not doing it, or having problems with it read on.
Design every piece of code in or around your automation framework keeping maintainability, reusability, scalability and robustness in mind (Pillars of framework design)
In that spirit your Jenkins job should be parameterized
This will allow you to select different AUT versions, browsers, tests to run from Jenkins itself
Even if you don’t need it right now, you SHOULD in the future
In case your automation tool is having problems reading parameters from Jenkins
You can export them into different file formats (JSON, XML etc.) and read those settings on project initialization of the automation tool
Easier said than done, but with patience and trial and error, you’ll get there
Problem checking connection to your Database?
I recently learned this trick using Universal Data Link files
There are many methods out there, sometimes could be confusing
For windows users I stumbled upon this neat trick:
– Create a new file with the extension “*.UDL”
– Open the file and enter data:
1. Enter the server IP (Sometimes machine names create problems)
2. Add database username and password
3. Select DB name from the list (should show all the DBs hosted on the machine with IP from step 1)
4. Press Test Connection
and Wala.. you can get your answer
Your Spring Boots app might not be connecting to your DB because of using the machine name instead of the IP as well..
For more details:
How to learn automation?
The biggest problem with this question is, it’s not the right question to ask!
We perceive a tester’s career progresses by starting with ‘manual testing’ to ‘automation’ to ‘management’
I don’t think that’s an accurate picture
Mostly we start of in testing as ‘Non-Technical’ (unfortunately)
We deliver greater value once we become ‘Technical’ and learn how the technology stack works
And then when leadership skills are acquired, that gives another boost to our impact, hence a higher pay
It’s not about just automation, it’s about having the capability to create a greater impact on the business’s bottom line
#QsDaily #TestersGoingTechnical #TestAutomation
“Life does not happen to you, it happens for you”
A line that resonated with me immediately and here’s why
Has it ever happened similar events keep happening in your life?
Is it just bad luck?
I’ve felt there is always a reason for why events out of our control take place
Sometimes we’re able to figure out the lesson we were meant to learn, a lot of times the meaning remains hidden
And until we understand why this is here, different events might keep on manifesting for us to learn
To lean and go to the next stage in life
If we choose to believe everything happens for a reason, we have a better chance at improving and being more content
Running tests in Parallel?
It’s not just about what tool to use
1. Why do you need parallel execution?
To reduce execution time yes, but are all the tests running needed? Or can they be executed at different stages of the SDLC instead of one go?
2. Are your tests designed keeping in mind parallel execution?
More here: https://goo.gl/977JEP
3. Parallel execution environment setup
Depends on the tooling you are using and the AUT. Few options I know of with open source:
– Selenium Grid (will run on single machine)
– Jenkins + VM – with Selenium running on multiple VMs / environments
– Jenkins + Docker – Running on a docker server where multiple containers running different tests in parallel. Can manage the parallel execution through Jenkins
– Jenkins + SaaS – services like BrowserStack, SauceLabs etc.
– Zalenium – Docker based Selenium grid. More here: https://goo.gl/pqXoQW
#QsDaily #Docker #Jenkins #TestAutomation
Continuous testing just a buzz word or can it really help you?
To quote my all-time favorite book ‘The teams to take a product from idea to the market will win’
Here’s how I explain it:
Bottom line of a company – Sales
The best way to predict sales and improve them? – Go and sell the product
Unless you don’t have anything to give to your customers, no way to get feedback from them
For decades we tried to ‘model’ markets, trends, buyer personas and so on
Sometimes actual results were around our estimate, mostly way off mark
Then we thought, let’s forget to try and ‘simulate’ the environment, let’s just put it out there instead
The reason for simulations and models was it took too much time to build a deliverable product
With all concepts around Agile, CI, CT, CD, DevOps, we are trying to move from once in a 6 month delivery to multiple times a ‘Day’
Still thinking if continuous testing can help?
A video I did on the subject:
#QsDaily #ContinuousTesting #DevOps #CI #CD
What are they and how are they related
is ‘Artificial’ since the machine will not be ‘aware’ and is only working from a set of mathematical formulas under a specific context
Intelligence – is relevant, a computer can beat the best chess player ever, but has no concept of winning or losing. Can we call VA’s like Siri ‘really’ intelligent?
So why call it intelligent? The same reason we started calling phones ‘smart’!
To develop ‘intelligence’, machines are ‘trained’ to answer specific questions in a given context
There are different ways in which machine learning is done, one is ‘pattern recognition’
To the problem we want to solve, a large sample answer set is ‘programmed’ in the machine
So when an actual problem is given, the machine matches the ‘patterns’ it sees with the patterns it was trained on..
#QsDaily #ArtificialIntelligence #MachineLearning #AI #ML
So these are the basic steps an automation tool does:
Wait, Find, Action – Repeat..
(Still thinking / exploring a cool name for this)
“Wait” for the AUT and automation tool to be in sync, and the object loaded and ready to interact with
“Find” OnLy the ‘Desired’ object “ALWAYS”
“Action” should be compatible with supported browsers / environments and handling the usual errors (delete previous text in text field etc.)
What I really wanted to talk about today were delays (wait). I hope most people reading know this, but it’s too important so I’ll repeat
First we established how important delays are, there 1/3rd of the whole automation! (in a way)
Second, here are my “object delay rules”:
1. Have a delay BEFORE interacting with ANY object “ALWAYS”
2. NEVER use a static delay, ALWAYS a dynamic delay
3. Check for not just if the object is visible, is it READY to interact with? *
If you do these three things with delays, you should be fine
* Exceptions are always there, but they should be the “exception”..
#QsDaily #AutomationFrameworkDesign #TestAutomation
“Within 6 months we’ll automate the testing”..
A goal one of the prospective client’s had for me, you can guess what happened next..
I understand that was a way to sell automation to management for budget approval, but was a horribly wrong statement
I’m glad to announce I never agreed to the crazy plan
The root of all this, IMHO, lies in misunderstanding what testing is
The idea of ‘testing’ being automated is impossible, unless we can create ‘Autobots’ or ‘Desepticons’!
‘Testing’ is a process of thinking, communicating with people, strategizing and continually adapting
We don’t have sophisticated enough systems to do that right now, and not sure when we will reach there if ever
But then how do you convince management that automation benefits everyone?
Here are my thoughts on calculating automation ROI:
#QsDaily #TestAutomation #AutomationRoi #Testing
Deciding what to automate?
Let’s decide what to ‘TEST’ first..
This is a follow up post from earlier where I mentioned METS
The idea of METS is to have a concise look of the important features quickly
And this is not needed for planning testing only , rather far more important for automation
Unless we are automating “what’s important”, we’ll get nowhere..
So first decide what to test, then talk what to automate
In this video I discuss the importance of deciding what to test first:
#QsDaily #testautomation #testing
Automation ‘Wisdom’ reinvented
I’m guilty too, thinking I was the first to discover some automation wisdom
The reality is, thorughout the decades of evolution in automation and automation tools, there have been common misconceptions ‘discovered’ again and again
And probably a lot of folks starting new in that wave of evolution were not aware about these problems seen in the past
In a talk with @Jim Hazen, we touch base on the same topic and Jim shares his experiences around it:
#QsDaily #TestAutomation #AutomationEvolution #TechnologicalExcellence
To write ‘traditional’ test cases or not
Whichever camp you are in, we all agree they are time consuming
There are many methods suggested in the community to document scenarios rather than all the ‘mundane’ steps
It becomes easy to refer and update, and takes less time to write
One such method was presented by @Greg Pascal at the #TestingGuild hosted by @Joe , which I loved instantly
It’s a one liner summary of test scenarios for one module with rows listing feature names and 4 columns signifying the scenario’s importance from critical to low
I like the concept because it’s simple and easy to
With ‘agile’ kicked in, it’s been a big excuse to NOT document ANYTHING, which has made domain knowledge a problem
IMHO, testing strategies like METS could be of great help in such cases
For more on that, here’s Greg’s website on the subject:
When I started learning #docker it was a bit confusing
To ease the learning curve, here’s how I explain docker to testers:
(simplifying things A LOT to make it easy to understand here)
1. Imagine a windows environment ‘X’ running on a desktop machine
2. Now create an image of that environment ‘Y’
3. Run that image (Y) on top of your desktop environment (X)
4. Any app running on Y would not affect X right? So that’s the first thing to appreciate.
5. Now whats common between X and Y? The OS, windows in this case.
6. So let’s take out what’s common, the OS. Remove the duplicate one from Y
And that’s what Docker does, it removes the duplicate OS and related resources, and uses the OS from X for running stuff on Y as well
All this while maintaining the isolation of both environments..
Along with a bunch of other benefits that come with this..
To read on the evolution towards docker and a brief intro on how it works:
As tester’s our job is to identify risks in the AUT,
Come hell or high water we have to get the job done
Wrote an article for #StickyMinds on the subject.If a feature is impossible to test from the outside, it surely can be done through fault injection
All you need is determination to get the job done.
A message from Bill Gates for folks from all walks of life
Then why do testers working in tech shy away?
Coding is like learning any other skill
You don’t have to be extraordinarily intelligent or gifted
Half of coding is about understanding a problem and thinking of a how to solve it
#QsDaily #TestersGoingTechnical #TestAutomation #Testing
– Always prioritize people over process. People are far more valuable than any process implementation. Don’t loose them.
– Management always wants to make decisions based on numbers, learn to give estimates.
– Don’t stop learning, try new stuff all the time. Also every problem you face is not unique. There are surely other people out there who have solved this before.
More to come in future posts.
http://testingguild.com/ and thank you Joe Colantonio for hosting this.
The #TestingGuild2018 conference just kicked off – 100% online filled with testing Awesomeness organized by Joe Colantonio.
Tomorrow at the conference I’ll be sharing a story illustrating the importance for testers to be technical.
In the #TestingGuild2018 conference I’ll be talking about how beneficial it can be to test your product across the complete technology stack.
Teaser of the presentation:
15+ years of coding experience, two dozen testing tools?
While there are a lot of opinions on the subject, I try to go for the essentials
1. Algorithm design aptitude. If our framework is in Java, I’m not focusing on Java developers only. If the candidate can demonstrate reasonable algorithm designing skills, that’s enough for me.
2. Testing Acumen. Not just fundamentals of software testing memorized word for word, rather have the technical depth on various topics around software testing and depth on the topic.
3. And the most important piece, the right attitude. The skills mentioned above can be acquired, but without the right attitude, it’s just impossible. Down the road you’ll have to let go of that person, period.
The Journey of a technical tester
Story of a hypothetical tester learning over his career to become more tech savvy and improving his craft
The first “story-presentation” I did for a conference, #PSQC2018
The presentation illustrates some fundamentals testers can learn over the years and the reasons why it’s important to learn these lessons
With a touch of humor, few core concepts related to the technology stack are explained as well.
The complete presentation can be seen here:
While testing in any shape or form is certainly adding value, but getting the most out of it is something entirely different
More than often I’ve seen teams having no idea where to focus testing on
In the #TestingGuild2018 conference I’ll be presenting a story talking about this subject
How to figure out what’s most important for the product and focus there first
Why we need to prioritize?
One answer would be: 80/20 rule..
(in our context) 80% results come from 20% effort, find that 20%
Teaser of the presentation here:
“Just because something works, doesn’t mean it can’t be improved”
And improve we must
Automation frameworks in an extremely volatile environment
Unless you keep up with the change, big problems are in store
Teaser of a presentation prepared for the #TestingGuild2018
The second in my series of story-telling presentations at conferences.