There are some ‘dumb’ routine tasks we have to do as a tester
Not all of them require a signifanct amount of intelligence and can be done automatically
And I’m not talking about checking as in testing something, rather activities other than actual testing
Some AUT upkeep tasks like configuration updates, checking installations, maintaining environments, compying stuff from one place to another
Having a single tool to do all of these things might be an overkill in the start
But certainly can prepare a bunch of smaller programs for routine tasks and controlled from one place (Jenkins..?)
Call it a stripped down dumb version of Jarvis if you will..
I know @Alan Richardson has talked about something similar
Care to share something you are using along those lines?
For sure it does NOT save cost upfront at least..
Implementing a DevOps culture and integrating Continuous testing in that will take some effort
A cultural shift will be needed and breaking down of dev / QA silos
Now to the benefits, most importantly provides quick feedback
Coding a feature incorrectly and then fixing it afterwards takes considerable effort
A lot of communication back and forth not to mention time spent regressing around changes
And some times issues are not found until the 11th hour, which is another Pandora’s box
For me the greatest benefit of continuous testing is preventing as many issues from being pushed in the first place
I created an illustration showing this phenomenon of continuous testing avoiding the pile up of issues over time
The article and video can be accessed here:
#ContinuousTesting #Automation #QsDaily #QsArticles
Is it just a number game in terms of man hours?
Mostly that’s how these ‘savings’ are calculated
The reality is quite different IMHO
Equating man hours to execution time by a machine is not accurate
The way a person would test is a lot different, even if they follow a written test case
Equating apples with oranges does not give a true picture,
Well then how does automation help?
Here’s what I think..
#QsArticles #Automation #AutomationRoi #GeneratingBusinessValue
A segment from the talk with Jim Hazen:
Many times we stumble upon a problem and think, hmm seems no one ever had this before
Especially with automation, we probably have reinvented the wheel quite a few times
Going through the entire history of automation evolution would be very lenghty,
This episode talks about some main events and how automation tools, practices chnaged over the years
Very fascinating and interesting subject, here’s the video link:
#QsEpisodes #TestAutomation #AutomationHistory
Here’s what I have learned over the years:
Firstly the test log is not for automation folks only,
Primarily for the whole product team to consume
Plus automation folks would like to help it in their debugging also
Keeping thwese audiences in mind, following should be a part of your log:
– A good heirarchical stucture resembling the test scenarios used for regression
– Lots of debug information indended within the report
– Some info on failures we know are reported issues
– And finally best to have it in a centralized location, Jenkins would be best
More on that here:
#QsDaily #TheAutomationBlogBook #AutomationInTest
From waterfall style having extensive documentation for each SDLC phase till the Extreme programming / SCRUM with very basics like user stories
I must add here, when teams say we’re going ‘Agile’, so don’t need anything except for user stories, that does not make me happy
Too much we run into problems of maintaining them,
Too little, and as soon as one or two key people leave their position, everyone is in trouble
It should be enough to sustain development, and keep the process lean enough
I guess everyone would agree on all the aforementioned, the problem is “Striking the right balance”..
I’d like to propose the following as a yard stick:
– Anyone wishing to understand any part of the code should not have the need to hunt down people to understand. Sufficient commenting and coding practices should be used
– Anyone wishing to learn about how the system should behave in a certain scenario, can find that info without bugging every single developer.
For application domain, one can utilize user stories, gherkin feature files (BDD), mind maps, test cases, client specifications
For code understanding, code documentation, commenting, coding practices, architecture documents
Make sure to follow the “Pillars of Automation Framework Design”
Unfortunately most teams start with UI automation
Which is the toughest and has the lowest return
Nonetheless it is important, so best to do it right
The 4 Pillars are:
Description of those here:
It’s one which is most scarce – TIME – and the successful know that
It cannot be created or borrowed
Every person has 24 hours, and once they pass, they will NEVER RETURN
The trick is utilize your time in the best possible place you can
Ideally we should do stuff only which we MUST do ourselves
DELEGATE anything else that can be done by someone else
This will take disciple, planning and a burning desire to head towards your goals
This is important for testers and software development teams too
“The art of self-discipline” by Brian Tracy
“The Millionaire booklet” by Grant Cardone
A #QsEpisode in which @Jim Hazen shares names of few automation experts over the years
I was familiar with most recent ones, but folks from back in the day I did not know of
Quite a few people have been contributing over the past many yers
Teaching tricks of the trade and their experiences
You either learn by experiencing it yourself or by learning from someone elses experience
It’s best not to reinvent the wheel and learn from those who have seen it before
Here’s the segment of the talk around this topic:
This is not a conclusive list. Just few names recalled at that time.
Jim Hazen shares the story behind the term he is famous for “AutoMagic”
He was kind enough to spend some time talking about automation and it’s history
Instead of uploading the whole 2 hour long talk as one video, I’ll be creating topic wise smaller ones
If you are not introduced to Jim or the term “AutoMagic”, you would want to listen to this:
Stuck with processing red tape to get things done?
Here’s one way how I went around it
I was seeing a major lacking in our code coverage for a safety critical device
The only way to test it was through white box testing, having entrenched silos it was not an easy sell
Instead of selling upfront, we went covert to build a case first
Implement and generated results from the intrusive testing technique,
Once we had a business case opened it up to management and
The miracle happened, despite it seeming impossible to institutionalize, the evidence of its necessity was evident
And since then have tried this few more times with excellent results..
Give it a try next time, I’d be eager to know how it went..
Remember Web front end development 15 years ago, and what has WordPress and other platforms done to it today?
The same might happen with CI processes through tools like Jenkins
The number of tools providing integrations to Jenkins is getting insane
Which is a really good thing, CI and Continuous testing is becoming way important
And getting hooks in there is not as smooth yet, but we can surely get there, probably in the near future
It might feel complex if setting up Jenkins is not approached the right way
Here’s a short guide to the very basic setup and brief description on it’s moving parts:
IMHO – Lessons learned and the relationships you built.
We might have supported financial needs while being on that job
But the lessons we learned through our experiences are far more important
And even more are the relationships we built
On that note, I’d like to thank a team member of mine who just moved on..
Thank you @Arslan for your efforts and your contribution towards the team
I truly wish you all the success and hopefully
You had great experiences with us because I’m sure we build great relationships..
One area I see neglected a lot is code complexity
I was fortunate to have worked for safety critical devices early on and learn some valuable lessons
One of which is code complexity (nested conditions)
When your code goes beyond a certain complexity level it becomes hard to have a deterministic result
More than often, I’ve seen a single area of code made too ccrammed and complex leading to problems all over
Spread out your conditions, instead of going deep vertically, try to go horizontal in your control flow
More on common UI problems causing flakiness:
Try changing things around you or yourself?
The kind of life you are living is not because of the people around you
It’s because of how you feel about and react to life
The people, events, opportunities we see around us are a manifestation
Manifestion of our thoughts and expectations
So to change your surrounding, change your thoughts and manifest the change you want to see
Think and become the change you want to be
How to be a badass by Jensen Shero
Here’s why this question comes up IMHO
For decades testers have known to be ‘Non technical’
They mostly never wanted to do much with the underlying technology
When automation came along (or when it became more popular) folks thought, well these guys know nothing of programming
So this automation thingy might not be real programming, just doing some record playback
In reality, automation is not just programming,
It IS hard core SOFTWARE DEVELOPMENT (when done right)
A common question I get when reaching out to people
It is human nature to expect something in return for spending time and effort
Over the years I’ve learned the trick in life is not to get, but to give
And the universe will respond back in many times more
But it takes faith and patience to keep going without return
In essence that’s what a business should do, solve people’s problems without trying to squeeze a dollar for every inch they contribute
I’ve always felt, living for something greater than yourself is a life worth living
Seth Godin, Susan Jeffers (and many more I cannot recall now)
Few things you should know before going in:
Legacy products need a different approach than most modern tech stack applications
The automation framework has to deal with the various legacy platforms used
This means compatibility with those languages, platforms is essential
In some cases it can be advantageous too, e.g. front ends built with some JQuery or custom components might make things easier
The backend might not be as ‘automation friendly’ since it was never designed for anyone other than the developer to understand
Processes, tools and the team, all would come into play
More insights in this article:
Angular 4 + Spring Boots + SQL..
Configuring an ERP can be VERY problematic
Getting settings the exact way a client has can be tough
There are other ways to do that (e.g. use dockerized environments) but there is the old fashion way too
Set your test environment like your client’s do the tests, revert it back
For applications having lots of configs this can be hard
That’s how we started with creating tester’s “own” test product
It had to be scalable, since there are a lot of things we might add to it
So needed a proper structure, which lead to a full-fledged product, a testing tool with:
Front end: Angular 4
Server side: Spring boots
More on this in coming weeks..
#TechnologicalExcellence #TechnicalTester #RedefiningSoftwareQuality
Can your brain focus on multiple things at the same time?
In the information economy the people to thrive are who invent
The terms knowledge worker(Peter drucker) and smart creatives (google founders) are common
Yet how much of our time do we spend on thinking deeply?
Shallow activities, just passing the buck around does not create the needed value
At the minimum work with focus and precisely on one thing only
More on the subject: “Deep Work” by Cal Newport
I think you should have the capability
An argument for not using a multiple tools is “Too many cooks spoil the broth”
While this was more valid years ago, now most vendors know there is an inevitable need for integrations
Automation used to be just UI automation, now we have API, DB level checks as well
Distributed teams have caused different team dynamics needing a whole set of reporting and management tooling
Then we have CI / CD in the mix which also would need to be integrated
Case in point, the automation framework must have the capability to support and use multiple tools
More on that here:
Did you know you can parameterize your jobs?
Especially helpful for automation
You can parameterize things like the app version / browser / DB / automation framework parameters from within Jenkins
Scheduled job runs will have default values selected every time they run
For manual job runs you can choose to select options from drop downs
All execution resources / environments management and results are centralized
Setting up new run routines or special case quick runs makes a big difference
And the kicker is the visibility you get from all the results in one place and accessible to all
In the start I thought just to reduce cost, here’s what I’ve learned till now:
– Yes some folks still think to reduce cost; <False> Automation only increases the testing cost
– Substitute testers; <False> Automation cannot substitute tests done by a person
– Increase test coverage; <True> Can rapidly increase features checked
– Provides confidence; <True> Increases the team and the customer’s confidence in the product
– Reduces time to market; <True> Can reduce testing time and supports continuous integration
– Shows commitment to quality; <True> While employing automation does show that, however not a good enough reason in itself
– Can find more bugs; <False> Automation checks ‘stable’ features, ideally it should not find ANY bug..
Care to share other reasons you observed?
I strongly feel they should
Firstly, let’s define technical: “Who knows how the product is designed and works”
Doesn’t mean you have to be an architect, but reasonable understanding how it works
80% code of an application is written at the back end, isn’t it logical to spend time testing on that layer?
And without knowing how that works, it cannot be tested..
Toyota had to recall 2.1 million cars on account of a bug
More on that story here: