And it’s implications for automation learners
It goes like this, place a marshmallow on the table, tell the child if they don’t eat this marshmallow for another x minutes, they’ll get a second one, then leave the room.
The jist of it is, kids who can delay gratification will probably be more successful.
And I feel this holds very true for automation learners.
Starting and staying with very basic automation concepts and trying to squeeze every opportunity sitting at that level is not helpful.
Learn to do the grind today and wait for the rewards later on.
Invest in learning how to do automation well instead of doing just the bare minimum and hoping to succeed.
Yes the industry does need automation engineers, but far greater is the need for excellent ones!
Here are the few places I’ve used it in:
Create isolated AUT instances
– We have separate DBs for different AUT versions
– Using docker can instantly create multiple AUT instances
– Different approaches to do it, one can be to just hook up multiple DB versions to the same app server
Multiple execution environments
– Selenium grid(s)?
– If your automation tool allows, can create multiple execution environments
– Can hook them up with Jenkins and everything will be done on the fly
I’m sure there are other uses where the AUT is NOT containerized and we can still leverage docker.
Have you tried something different? would love to hear..
Deciding what to Automate?
First let’s talk about what to test
Often when teams talk about what to automate they jump straight to regression tests
I like to start with ‘Do we have a good set of scenarios to test’ to choose from?
What to test would depend on changes, features inmportant for product positioning, most sought out features by clients, features with most bugs from the field and so on..
Once we have that list, we can talk about what to automate.
Here’s the discussion on the subject:
Does verifying ethical boundaries of a product come under testing, E.g. Data privacy?
If it’s expected by management then off course, if not then?
In this age I feel we’ll have to face more ethical problems than ever before
As technology’s reach increases, more possibilities open, creating more social challenges as a by-product
As a tester, Identifying and factoring end user’s needs and wants is paramount
Keeping a moral high ground has always been tough
The decision would boil down to every person’s core values I guess
Felt this is a question many of us might face in the coming years (if not already)
We all learn in different ways, lessons that stick usually have more effort and emotion involved
This is one such story of debugging an automation framework problem
There were quite a few lessons we learned and have been vital in our success
If I’d have to mention just one, “Whatever can go wrong, will go wrong” (Murhpy has a lot to answer for..)
So try to figure out what can go wrong and prepare for it
For a brief summary of the story:
The complete TechBeacon article here:
Perhaps one of the most common and used among the Java community
For automation engineers, specially working with object oriented languages like Java and Python this is important
These will create clearner, less complex and reduced coupling in you framework
The principles might be a bit complex to understand at first,
But are very useful and worth the time spent
Many time it’s just inadequate planning
Sometime the change is misunderstood
Assumed as a small / localized / easy fix
One thing leads to another and we’ve opened the pandora’s box
I’ve always felt the first mistake of misjudging the change is the main culprit
BTW, this goes for automation framework changes as well
The rule of thumb I use: “If any other module of even class can get affected by the change, don’t do it at the end”
A few more tips in this article
#QsDaily #QsArticles #NotSoSmallChanges
Jim Hazen and I talk about how we keep repeating the same mistakes..
Some sample cases would be:
– Record and playback
– Automation would render testing team’s unecessary
– Treating automation as a part time activity
– Having the wrong goals to measure automation
To be honest, this can be a very long list, but you get the point
Here’s the discussion about why we probably keep running into the same trap every time a next wave of evolution in automation comes around
#QsEpisodes #AutomationMistakes #HistoryOfAutomation #LearningFromMistakes
Unfortunately, mostly are love and hate relations, especially between Superiors and Juniors
And here’s what I’ve experienced over the years:
– Every Superior has resentment to every team member, it might seldom and negligible, might be frequently and severe
– Similarly, every team member has resentment towards every superior, it might be seldom and negligible, or frequent and severe
Striking a balance between the love and hate is extremely hard, and equally important
The question is, how to deal with it
Well, everyone has their own way, I try to spread goodness and have no ill-will
It’s tough, seems impractical sometimes,
But my faith dictates, in the end those who spread goodness are the ones with the greatest impact and success
Well, it does and does not.. Let me explain
By testing ‘just’ the API, yes some checks programmed for the client side will get skipped
However, API automation is not to ‘replcae’ UI automation
It is supposed to work in ‘conjuction’ with UI automation
And not only UI, but in coordination with unit tests as well
Therefore, if we look at just API / service layer tests only, then we do loose areas to test
But when working in conjunction with UI and Unit tests, (the way it should), then we don’t forego any code from being tested
#QsDaily #UiAutomation #ApiAutomation
There are some ‘dumb’ routine tasks we have to do as a tester
Not all of them require a signifanct amount of intelligence and can be done automatically
And I’m not talking about checking as in testing something, rather activities other than actual testing
Some AUT upkeep tasks like configuration updates, checking installations, maintaining environments, compying stuff from one place to another
Having a single tool to do all of these things might be an overkill in the start
But certainly can prepare a bunch of smaller programs for routine tasks and controlled from one place (Jenkins..?)
Call it a stripped down dumb version of Jarvis if you will..
I know @Alan Richardson has talked about something similar
Care to share something you are using along those lines?
For sure it does NOT save cost upfront at least..
Implementing a DevOps culture and integrating Continuous testing in that will take some effort
A cultural shift will be needed and breaking down of dev / QA silos
Now to the benefits, most importantly provides quick feedback
Coding a feature incorrectly and then fixing it afterwards takes considerable effort
A lot of communication back and forth not to mention time spent regressing around changes
And some times issues are not found until the 11th hour, which is another Pandora’s box
For me the greatest benefit of continuous testing is preventing as many issues from being pushed in the first place
I created an illustration showing this phenomenon of continuous testing avoiding the pile up of issues over time
The article and video can be accessed here:
#ContinuousTesting #Automation #QsDaily #QsArticles
Is it just a number game in terms of man hours?
Mostly that’s how these ‘savings’ are calculated
The reality is quite different IMHO
Equating man hours to execution time by a machine is not accurate
The way a person would test is a lot different, even if they follow a written test case
Equating apples with oranges does not give a true picture,
Well then how does automation help?
Here’s what I think..
#QsArticles #Automation #AutomationRoi #GeneratingBusinessValue
A segment from the talk with Jim Hazen:
Many times we stumble upon a problem and think, hmm seems no one ever had this before
Especially with automation, we probably have reinvented the wheel quite a few times
Going through the entire history of automation evolution would be very lenghty,
This episode talks about some main events and how automation tools, practices chnaged over the years
Very fascinating and interesting subject, here’s the video link:
#QsEpisodes #TestAutomation #AutomationHistory
Here’s what I have learned over the years:
Firstly the test log is not for automation folks only,
Primarily for the whole product team to consume
Plus automation folks would like to help it in their debugging also
Keeping thwese audiences in mind, following should be a part of your log:
– A good heirarchical stucture resembling the test scenarios used for regression
– Lots of debug information indended within the report
– Some info on failures we know are reported issues
– And finally best to have it in a centralized location, Jenkins would be best
More on that here:
#QsDaily #TheAutomationBlogBook #AutomationInTest
From waterfall style having extensive documentation for each SDLC phase till the Extreme programming / SCRUM with very basics like user stories
I must add here, when teams say we’re going ‘Agile’, so don’t need anything except for user stories, that does not make me happy
Too much we run into problems of maintaining them,
Too little, and as soon as one or two key people leave their position, everyone is in trouble
It should be enough to sustain development, and keep the process lean enough
I guess everyone would agree on all the aforementioned, the problem is “Striking the right balance”..
I’d like to propose the following as a yard stick:
– Anyone wishing to understand any part of the code should not have the need to hunt down people to understand. Sufficient commenting and coding practices should be used
– Anyone wishing to learn about how the system should behave in a certain scenario, can find that info without bugging every single developer.
For application domain, one can utilize user stories, gherkin feature files (BDD), mind maps, test cases, client specifications
For code understanding, code documentation, commenting, coding practices, architecture documents
Make sure to follow the “Pillars of Automation Framework Design”
Unfortunately most teams start with UI automation
Which is the toughest and has the lowest return
Nonetheless it is important, so best to do it right
The 4 Pillars are:
Description of those here:
It’s one which is most scarce – TIME – and the successful know that
It cannot be created or borrowed
Every person has 24 hours, and once they pass, they will NEVER RETURN
The trick is utilize your time in the best possible place you can
Ideally we should do stuff only which we MUST do ourselves
DELEGATE anything else that can be done by someone else
This will take disciple, planning and a burning desire to head towards your goals
This is important for testers and software development teams too
“The art of self-discipline” by Brian Tracy
“The Millionaire booklet” by Grant Cardone
A #QsEpisode in which @Jim Hazen shares names of few automation experts over the years
I was familiar with most recent ones, but folks from back in the day I did not know of
Quite a few people have been contributing over the past many yers
Teaching tricks of the trade and their experiences
You either learn by experiencing it yourself or by learning from someone elses experience
It’s best not to reinvent the wheel and learn from those who have seen it before
Here’s the segment of the talk around this topic:
This is not a conclusive list. Just few names recalled at that time.
Jim Hazen shares the story behind the term he is famous for “AutoMagic”
He was kind enough to spend some time talking about automation and it’s history
Instead of uploading the whole 2 hour long talk as one video, I’ll be creating topic wise smaller ones
If you are not introduced to Jim or the term “AutoMagic”, you would want to listen to this:
Stuck with processing red tape to get things done?
Here’s one way how I went around it
I was seeing a major lacking in our code coverage for a safety critical device
The only way to test it was through white box testing, having entrenched silos it was not an easy sell
Instead of selling upfront, we went covert to build a case first
Implement and generated results from the intrusive testing technique,
Once we had a business case opened it up to management and
The miracle happened, despite it seeming impossible to institutionalize, the evidence of its necessity was evident
And since then have tried this few more times with excellent results..
Give it a try next time, I’d be eager to know how it went..
Remember Web front end development 15 years ago, and what has WordPress and other platforms done to it today?
The same might happen with CI processes through tools like Jenkins
The number of tools providing integrations to Jenkins is getting insane
Which is a really good thing, CI and Continuous testing is becoming way important
And getting hooks in there is not as smooth yet, but we can surely get there, probably in the near future
It might feel complex if setting up Jenkins is not approached the right way
Here’s a short guide to the very basic setup and brief description on it’s moving parts:
IMHO – Lessons learned and the relationships you built.
We might have supported financial needs while being on that job
But the lessons we learned through our experiences are far more important
And even more are the relationships we built
On that note, I’d like to thank a team member of mine who just moved on..
Thank you @Arslan for your efforts and your contribution towards the team
I truly wish you all the success and hopefully
You had great experiences with us because I’m sure we build great relationships..
One area I see neglected a lot is code complexity
I was fortunate to have worked for safety critical devices early on and learn some valuable lessons
One of which is code complexity (nested conditions)
When your code goes beyond a certain complexity level it becomes hard to have a deterministic result
More than often, I’ve seen a single area of code made too ccrammed and complex leading to problems all over
Spread out your conditions, instead of going deep vertically, try to go horizontal in your control flow
More on common UI problems causing flakiness: