The summary of Automation learning paths I designed over the past few months.
API Learning path – Beginner level:
> Testing fundamentals
– Risk based testing
– Test strategy fundamentals
– Basic Java
> Technical knowledge
– How do APIs work
– JSON & XML structures
– Swagger fundamentals
> How and why of automation
– How to do automation the right way
> Automation tools
– Rest Assured
– Basic API framework
Each course has online content or a workshop designed along with assessment material.
#RSQ #Automation #Training
If your in IT, you have to be open to learning ALL THE TIME.
Was reminded of this while developing an app in PowerApps today,
For a massive training program, was trying to figure out a more intuitive way to do registrations,
After few days of searching couldn’t find anything in-house I could use, had three options:
1) Use whatever mechanism we have even if it’s shitty, 2) wait on some one to get free and do it or 3) do it myself.
Ended up learning PowerApps and building a crude version of the app in a day
Could have easily said, we don’t have a solution, let’s just call it quits
Instead I learned something new, enjoyed it, and made others life easier to get the training
Bottom line, if your in IT, always be ready to learn and work hands on.
#QsDaily #Learning #Coding
A major lacking in large software enterprises is:
Lack of a holistic test strategy across the customer journey
Scrum teams are mostly thinking within their small box,
Worried about getting just the next story out the door, and I don’t blame them. That’s what they are judged on.
But to stitch all that together and build synergies between teams should be a prime objective of testers
The customer does not care about what happened with one story
The customer deals with the complete end product, which is a conglomerate of multiple products with many stories
Let’s not be penny wise pound foolish and ignore investing in the big picture.
#QsDaily #Testing #TestStrategy #TestingAcumen #RSQ #RedefiningSoftwareQuality
Busy designing an automation training program, here are the six main areas:
1. Testing Acumen
2. Programming fundamentals
3. Learning the tech stack
4. Building automation enablers
5. CI / DevOps basics
6. Automation tools and framework design
Drop any of these, and you wouldn’t have people who can build useful automation practices.
I know that’s a lot to do and to prepare, but that’s just me. Mr. Perfectionist.
I want anyone that goes through the training should be self sufficient.
Got a great question in my talk’s Q&A at the #AutomationGuild 2020 :
How to create test data for #BigData projects?
Generally there are three types of ‘test data management’ you want to focus,
1. Mocks / stubs
2. Generate synthetic test data
3. Masked production data
For big data, the most important one is masked production data.
You would also need to create synthetic data too, but will not be enough to see if the model is working properly or not
So make an effort to get masked production data to have greater confidence in your data pipeline and data models.
#QsDaily #BigData #TestData #Testing
Over the years I’ve started to dislike writing formal and long test cases
We need test cases, but written in a better way..
Traditionally we write them detailing each step along the way,
The idea was even if someone is not a domain expert can use them, or to make sure we don’t miss a step and know what exactly was tested
These monster documents become shackles: time consuming to write, a nightmare to update
Instead I prefer writing the ‘test scenario’ in brief with the most important check / validation over a full blown detailed list of steps
Also, importantly, using mind maps instead of test case management tools / documents
One does sacrifice the details this way, but makes things much quicker
Also provides a great holistic view of the many test types / scenarios
#QsDaily #Testing #TestCaseWriting #LeanThinking
I’ve always believed living a life bigger than yourself.
Thank you Joe Colantonio ⚙️for adding me to the top 28 Test automation engineers to follow in 2020.
My vision since 2013 has been “Redefining Software Quality”
I believe testers and testing practices need to evolve considerably to deliver value to businesses
Instead of trying to shoe horn ‘our way’ of how we do things into what people do,
We should develop skills and practices that support the industry’s needs of today.
IMHO there are three values which will help us get there:
– Technological Excellence
– Testing Acumen
– Business Value
#RedefiningSoftwareQuality #RSQ #Automation #Testing #TestersGoingTechnical
Test data management for automation is not just adding some fields in cucumber…
Test data management can include:
– Developing & using mocks for running low level tests in the pipeline
– Data creation & cleaning on the fly (more here: http://bit.ly/QS_tdc)
– Using masked production data & refresh it on a regular cadence (for exploratory / behavioral tests)
For all three, the data generation, storage and usage techniques are going to be very different.
And if you are working in a large enterprise, you most probably will need ALL THREE
No one said automation is easy..
#QsDaily #Automation #TestData
Before any analytics can run on data, sometimes a number of ETL (Extract, Transform, Load) processes happen.
Data might pass through a series of data engineering / ETL processes to make it fit for purpose and categorized as needed
Quality across this process is all about ‘Data Quality.
I’ll explain the concept further and talk at length about how to check data quality in the upcoming #AutomationGuild
#QsDaily #Automation #BigData #DataPipelines
I know implementing the automation pyramid is hard,
And to a large extend, is not a problem with just testing practices either..
I’ve seen teams where products don’t have contracts written up properly (API contracts / JSON schemas)
Back end services are not designed for anyone other than the developer to consume..
Such places do make it hard to implement the pyramid, i.e. 70 – 80% tests at the back end,
The solution: The whole team work on developing those contracts, and then write tests for those contracts..
Easier said than done, but until you don’t have that, testing isn’t going anywhere.
#QsDaily #Automation #Testing #ApiAutomation #AutomationPyramid
The only constant is change, especially in tech, regardless what role you play.
Similarly, dear testers, we need to change and adapt.
Change is uncomfortable and scary at best, but not a good enough reason not to change.
There is no choice but to work on ourselves “throughout our lives”! Doesn’t matter what occupation you are in.
The next time you are exposed to a new technology, language, testing technique, lean ways of working, don’t hold back.
Pushing yourself never comes naturally to ANYONE
I still ‘train’ my mind to push through my fears. And when I stop training it, I start to give into fear.
To train your mind and subconscious, one ritual can be to listen to motivational content, these days it’s Les Brown for me (linking sample video below).
#QsDaily #TechnologicalExcellence #Motivation #TestersGoingTechnical
Automation regression percentage – The metric I hate the most..
Many times the only use of this metric is to provide false assurance that we are efficient in testing.
And the ultimate goal soon becomes to automate regression 100%, which is a bad idea,
And automating just UI tests makes it even worse.
More on why not to use it and what should be done in the linked video
#RedefiningSoftwareQuality #Automation #RegressionTesting #KPIs
Data quality is one of the biggest problems with data science projects,
I’ll be talking about these at the #AutomationGuild, here’s a quick list:
– Accuracy. Is the data accurate in the context to be used
– Validity. Is the data fresh enough, still valid?
– Consistency. Data from different sources / time frames matches
– Completeness. No parts of data are truncated / missing
– Uniqueness. Enough data to uniquely identify records
– Timeliness. Data being collected at the right time & processed in a timely fashion (efficient enough)
More on the conference here:
#QsDaily #BigData #DataScience #Testing
For large enterprises with many interconnected applications, generating test data can be a challenge.
Even more so for big data projects, which is where masking comes in..
Take a copy of production DB from different products around the same timeline and mask the data for GDPR compliance.
Surely easier said than done, but can be very effective.
IMHO three main requirements to build this:
1. A quick guideline to identify what data will be classified as ‘sensitive’ and ensure it’s GDPR compliant.
2. A masking platform which masks value the same way across, helping with consistent data
3. Creation and usage of test environments is efficient and fit for purpose
#RedefiningSoftwareQuality #TestData #Automation
One of the main reasons for flakiness in automation is test data.
Therefore use mocks to run scripts executing in the CI pipeline.
For most folks, automation scripts are executed only in nightly runs with data created on the fly.
And unfortunately these tests are mostly UI tests, another blunder.
I gave a talk yesterday in which I said – 50% of your automation scripts should run in the CI pipeline triggered on each pull request.
And all these tests should use mocks instead of real data from downstream systems / components.
(I hope it’s needless these tests are on the services / API layer..)
So don’t forget your mocks when automating.
#RedefiningSoftwareQuality #QualityTransformation #Automation
“If I know my stuff, my employer will pay me well” that’s a lie we’re taught from childhood.
And here’s my take on this:
The industry is not there to fulfill your needs, it’s there to protect the entity’s interests.
If you want to get paid, knowing your stuff is a given,
Then there’s a heck lot more to do before you get paid well.
As engineers mostly we’re oblivious of social skills, and that’s the next big piece.
Learning how to deliver value and sell it appropriately
A few people I follow who have great content around this:
Thank you Umair Sheikh for inspiring this post.
How to become a ‘well paid’ tester?
I call it “The next generation” tester and here are the main skills…
– Agile practices and DevOps culture
– Ability to match testing goals with product’s goals
– Figuring out KPIs / OKRs which help ‘deliver value’
– Enterprise & Vendor management
– Risk based testing
– Defining a practical, efficient and effective test strategy
– BDD practices
– Context driven testing
Automation in test
– Algorithm design aptitude and programming proficiency
– Automation framework design
– API / UI / Mobile Automation
– OAT (Performance and security basics)
– Branching & merging
– CI tools (Jenkins / M. Azure)
– Provisioning environments Containers / VM / Cloud)
Technical Leadership- Technical acumen
– Team leadership
– Collaborating with architects- Thought leadership
Details of each in the linked article
#RedefiningSoftwareQuality #NextGenerationTester #TestersGoingTechnical
DevOps sounds sexy, but it’s not a piece of cake.
And certainly is not just about Jenkins..
It’s foremost a mind set to deliver at speed and ONLY WHAT’S VALUABLE.
This prioritization of what’s valuable is so under-rated, and one of the biggest cause of failure.
Working on anything that’s not priority is ‘waste’, and waste is what kills it all.
I talk about this briefly in the linked article
Picture – Slide from @Jez Humble workshop showing the ‘cost of delay’ of different features in a specific case study. (It’s it ridiculous how much we would waste trying to implement all features there!)
#RedefiningSoftwareQuality #EliminateWaste #DevOps #Lean
I’ve been reading about psychological safety for while, particularly, how you implement it in a team.
Imagine my luck, had the opportunity to ask that question from @Jez Humber in a workshop..
What I understood mainly was: The leader’s behavior builds or destroys it.
If the leader punishes people for failing or calling out what they feel is wrong, then no matter what, the culture goes down the drain
As an example, he talked about Etsy where a person on the team broke production deployment while following the process, the person was given an ‘award’ for highlighting the problem instead of being penalized.
Link to post here:
If you want your team to perform, build psychological safety.
To build that, the leaders must be seen as supportive to failures, whistleblowers, people who highlight problems instead of shooting the messenger.
#RSQ #PsychologicalSafety #leadership #HighPerformingTeams
I very much dislike automation trainings starting with teaching Selenium,
For starters that’s the wrong place to do automation. One should focus more on API tests and do a handful UI checks.
Secondly, automation is a means to an end, not an exercise for the heck of it. Therefore talk about why to automate.
Thirdly, in some cases folks learning automation don’t really learn how to test well either. If your testing is bad, your automation is just going to amplfy that bad.
Lastly, without knowing how the underlying architecture of the product works, IMHO one cannot truly learn how to test well.
So next time you plan an automation training, make sure you get the fundamentals right first.
#RedefiningSoftwareQuality #TestAutomation #Training
Anyone who has written automation scripts has faced flakiness, and we all hate it to our bone.
But everyone defines a flaky test a bit differently..
For me, a flaky test definition could be “a test running multiple times under the same conditions gives different results”
How would you define flakiness?
#QsDaily #automation #flakytests
Mostly when companies talk about transforming, automation come up as the most hot topic.
While that certainly is important, without your quality strategy & foundations it’s not going to work.
Immediately people start running towards buzz words and new shiny tools.
Knowing what quality means for your product, what to test and how to test should precede any automation effort.
If you used so ship garbage at a slow pace, with new shiny tools and terms it’s going to garbage coming out double time..
#QsDaily #Automation #Testing
Ever been in a working environment with a lot of processes?
While I agree ‘some’ of them are needed, but mostly they become shackles and hamper progress.
It’s a fine line sometimes to make a process which is effective and efficient at the same time,
In case of doubt, my personal preference is always to make it more leaner.
After all, that’s what the manifesto says, people over process.
Many organizations begin with the assumption employees are unable to make right decisions
Therefore, there they need ‘processes’ to ensure they operate as expected.
The problem: it’s impossible to come up with a ‘perfect’ process,
So instead of a rigid process, have general guidelines and have faith the people will make the right decision.
#QsDaily #processes #WaysOfWorking
I hate when testers doing exploratory testing don’t look at automation run results…
This mentality of us vs them among Automation engineers vs Exploratory testers has to go
And I wouldn’t blame testers only for this, when management feels automation will help them get rid of manual testers, It’s only natural for testers to feel insecure
For the 100th time AUTOMATION IS MEANT TO HELP TESTING!!!!
A common question I ask teams – If I stop automation runs today, will that change the time you’ll take for exploratory testing?
If the answer is no, i.e. with or without automation our testing time remains the same, then WHY IN THE WORLD ARE WE DOING AUTOMATION!!!
#QsDaily #CommonSense #Automation #Testing #regression
Testing in big data area has typical challenges,
A big factor is quality of data ingested.
The analysis results has a heavy dependency on the ‘quality’ of data ingested (obviously),
What often happens is inconsistency across the data from down-stream sources, missed records, missing data within records etc and other data quality issues.
Unless these issues are flagged at the lower levels, problems creep up and start reflecting in the analytics results.
While having automated data checks on massive data stores might not be an easy job, it certainly is worth implementing.