Automation training doesn’t start with learning automation tools,
It start with learning testing fundamentals!
Learn what to test, where in the tech stack to test and how to test.
Next we can talk about what to automate, where and how.
The unfortunate part of many folks lerning automation – their training starts and ends with Selenium
And I say this all the time, you cannot pick a worse place than Selenium with Java to begin with.
Nothing against Selenium or Java, brilliant library & language but that’s not the starting point.
Most people feel what they lack is automation skills, unfortunately often engineers lack “Testing skills”.
If that’s not fixed, you’ll get garbage in and garbage out.
#RedefiningSoftwareQuality #Testing #Automation #TestAutomation #ContinuousTesting
Some background to ‘flattening the structure’
In the industrial revolution and mass manufacturing era, companies operated with lots of workers doing remedial tasks.
We had hundreds of people doing atomic jobs and funneling up progress & information up the ladder.
After information consolidating across many hierarchical structures, ‘management’ took decisions and sent instructions down the corporate ladder
In the internet age, we don’t have much remedial work left in many places & the speed of operation has phenomenally increased
Workers have been replaced with ‘knowledge workers’
OR ‘Smart creatives’ who are tech savvy, business savvy and are closest to the ground reality
Therefore decisions should not be made at the top, rather by the engineers closest to the situation
Which renders no need to ‘manage’ people and take decisions for engineers, rather just ’empower’ the engineers to take decisions.
Hence the necessity of flatter org hierarchies and Servant leadership
There is NO silver bullet answer and it might not even be these two (Python or Ruby)
Here’s what I have used historically:
- Browser automation : Javascipt
- Interacting with the DOM is easier,
- Using AUT exposed methods is easy
- Scripting languages quicker to write and lesser code
- Popular tools: WebDriver, Cypress, NightWatch
Services automation: Java (Where prod code is in Java)
- Same language as production code – critical for component tests
- More options for mocks
- Can reuse prod code (e.g. POJOs)
- Extensive community support
- Popular tools: RestAssured, WebTestClient
Again, no silver bullet answer – need to consider various factors to decide for your team.
#TestAutomation #Testing #Programming
Planning you PIs (Program Increments) is crucial (One PI typically spans for 6 sprints).
While planning architectural road maps for quality initiatives across different teams, I’ve noticed a couple of blind spots:
1. Planning for feature deliveries and enablers / architecture road map are very different, planning them both the same way does not work.
2. Test and learn – I am the planning type, plan so extensively and as soon as the rubber hits the road, all plans go up in the air. Have a long term vision, but how to get there is fluid – plan that for just ONE PI
3. Prioritizing enabler features. Depending on what your structure is, getting tech debt prioritized can be the toughest job. Try to get an agreement around that.
DevOps is a mindset change.. BUT
Also requires a hell lot of enablers:
- CI tool in place (Jenkins, TeamCity..) with hooks to SCM, to creating environments, build & deploy the product, run the tests, and quality gates
- You may need multiple pipelines
- Capability to spin up environments (Docker / OpenShift / Cloud..) on demand with desired configurations & tear them down
- E.g. SonarQube or any other static analysis
- Mocks / stubs for tests running on lower levels (unit / component)
- Synthetic data generation on demand
- Data creation and clean up routines
- Unit tests, component tests
- API tests, contract tests, UI tests
…to mention just a few high level enablers.
The best part – engineers and product owners need to have confidence in all this.
So if you want to transform the way you release your product, have a plan for both:
- mindset change and how to develop all the enablers you need
- INCREMENTALLY (waterfall like DevOps is of no use..)
#RedefiningSoftwareQuality #Transformation #DevOps
Documenting your APIs behavior / contract on a confluence page enough for building common understanding?
In my experience, these pages often remain as static pages. The code keeps changing and the documentation does not.
I prefer using swagger to document workings of the API and use that as a reference.
This way you have a living document and no need to keep updating a static page.
Oh, and just to add, functional & contract tests are going to ensure if your building your services properly.
#RedefiningSoftwareQuality #TestAutomation #APIs #Swagger
A key area to consider while working on automation frameworks:
Complexity vs configurability.. and their inverse relation
To support more and more features, and make a solution scalable you’d want it to be more configurable
However as you make it configurable, it is bound to become more complex.
Sure using design patterns will ‘reduce’ the complexity, but inevitably will become complex as you make it configurable
There’s a balance you want to maintain between the two.
E.g. using libraries like cypress & robot you’d be able to create tests very quickly, but you loose control on certain configurability aspects, if you can live with them these are great
So it’s going to depend on your situation, But the principle remains..
Hiring for automation engineers / looking for a position?
Here is my list most important skills you consider / learn:
1. Programming aptitude. Can think and design algorithms, language does not matter.
2. Testing Acumen. Practical experience of developing and using heuristics & identifying risks.
3. Attitude. Enjoys solving problems.
4. Automation exposure. Have an understanding of typical pitfalls in automation frameworks and how to fix them.
Importan point: Don’t necessarily look for someone / stick to any one langauge / tool. Technology is bound to change.
#RedefiningSoftwareQuality #TestAutomation #Hiring
Who feels most emails are a waste of time?
FYI emails, cc’ed emails, bcc’ed emails, the long legal contract like emails, the confusing email with half the department copied in…
There are many reasons of this, most have to do with ways of working
Sometime no one wants to make a decision and let the focal point to keep spinning like a merry go round
Build a culture of test and learn, so people safe making decisions – Psychological safety
Also try to connect to people / have a quick group discussion as much as possible instead of long emails with a dozen people copied in.
One of the biggest challenge in #BigData projects is ensuring the ‘quality’ of data
Debugging ‘anomalies’ across #DataPipelines can be a nightmare. Certain teams end up spending more time debugging than actually coding new features.
That’s because usually there are no automated data quality checks in place to catch issues, and they have to trace back anomalies across huge amounts of data sifting through complex ETL processes.
With other development projects, the behavior is more predictive because inputs into the system are homogeneous.
For #BigData projects, there is no guarantee on the data we ingest & the how it get’s processed will always be accurate – the input is NOT homogeneous.
The solution to this: have #automated data quality checks running in #production across the data pipeline
#RedefiningSoftwareQuality #BigData #Testing #Automation
Project management – waterfall vs agile:
– Project phase 1: 50% over budget & 6 months late
– In X budget & Y time we’ve managed to prove benefit hypothesis works / doesn’t work
Prime difference is:
In waterfall we benchmark spend against ‘planned spend’ (which is ALWAYS WRONG)
In Agile we benchmark spend against ‘value created’
Also – Can you sniff the BIG WALL / Them vs Us in waterfall
And agile is more about we as a group managed to achieve this
#RedefiningSoftwareQuality #AgileTransformation #Agile
Automation training projects should have a course on Git.
Mostly engineers don’t prioritize to learn how to effectively collaborate on source control management tools.
This causes lack of support & maintenance on the code base.
Get in the habit of collaborating more on code (code reviews, adding comments, contributing).
Otherwise we don’t document design decisions, or its in a document buried somewhere and never used because hard to find when needed.
“We don’t want to invest in tech debt right now, need to churn out more features”
That’s like saying, we’re not going to clean the house for a month, it’s just a waste of time..
(FYI, not investing in automation is accumulating tech debt)
Three common stages of a data pipeline:
– Ingestion into data lake. Curating data.
– Pushing into data warehouse, model data.
– Pull data from warehouse and build analytical models.
Can differ according to use case.
Data quality checks should happen before entering and within each stage.
Should a BDD test scenario only be automated on the UI?
Commonly the answer is yes, I have a different opinion.
Let’s say feature is to show total cost of items in a shopping cart along with the shopping cart icon.
Implementation changes will be at the front end & service layer.
I’d personally want to have unit tests for new methods, component tests on the end point and:
Service integration tests for the BDD scenario – Adding / removing products to the cart and seeing if the cart total gets changed – or whatever the scenario may be
No harm in adding an extra check to an existing UI test, but I’d refrain from having a whole new UI script just for this, rather have an exclusive one on the API.
A recent question I got asked:
How to combine Web & mobile UI automation frameworks together?
Now there will be no one size fits all answer here, but here’s what my experience has been:
1. Most teams try to run every test through selenium, and then the same through mobile (Appium, XCUITest etc..)
And that is the biggest problem – don’t have a lot of tests running there, most of you testing should be completed on your services level
2. Depending on your product, there might not be a lot of synergies between the tests you have on web & mobile, keep an eye out
3. Keep your frameworks very modularized, every component can be reusable
4. The utils layer can be shared across both frameworks, can have it as a separate project, added to the web & UI ones
5. Similarly test data / configs should be in sharable format. My favorite is JSON structures & config files
6. Having both web & mobile tests within the same project might not be a bad idea either.
FYI – Google maintains one code repo for all products. But be careful, if you don;t structure your product well you’ll run into trouble – Make sure everything is loosely coupled
These are few on top of my head, anything else you can think of?
#RedefiningSoftwareQuality #TestAutomation #MobileAutomation #WebAutomation
What does shift left really mean?
Here’s my understanding and what it means to implement it:
Any activity to help us prevent or identify issues BEFORE someone spends dedicated time testing the change – Shift identification of issues & risks as early as possible.
I’d implement this by doing the following:
1. Have three amigo sessions before coding
2. Practice TDD & have decent unit test coverage
3. The developer and any other person on the team together test the change on the developers machine
4. Have quality gates on the pull request pipeline including static analysis, unit tests & component test
By the time engineers testing the feature start exploring in detail, most of the issues should have been fixed already.
#RedefiningSoftwareQuality #ShiftLeft #ContinuousTesting #DevOps
An important part of transformations is building technical enablers.
Initiatives around Digital Workspace were been worked on with a lot of vigor at Emirates Group IT for some time
As a result of all that effort we were able to transition smoothly to working from home across the landscape
I had always taken this for granted, being IT professionals lot of these things are common place us
However the magnitude and scale at which large scale enterprises operates, ability to pull this off on such a short notice is an achievement.
Realized this today in a CTO listening session where Microsoft’s head UAE region was sharing his experiences post COVID19.
Bottom line, if you want to truly transform your team, you’ll have to build the technical enablers they need.
Building self autonomous teams is a key change in transformations,
And a key element in there is how to delegate decision making?
Typically enterprises have large groups of people with layers upon layers of decision makers
Decisions and actual hands on work are done in two different groups
As mush possible, decision making has to be delegated, and here are the two key components:
1. Competency. Develop the teams technical skills to have the background to make those decisions.
2. Clarity. Access to information and ability to process, to allow teams to have all the background information they need to make decisions.
Both are easier said than done, but without them the transformation will fail.
#RedefiningSoftwareQuality #QualityTransformation #Transformation #HighPerformingTeams
For engineers technical skills are vital,
Unfortunately so are people skills, because ultimately we have to deal with humans.
And you’ll find every kind of them in a work environment.
Problems occur when an individual or organization neglects any one of these skills
Less technical skills will lead to weaker products with lots of issues,
Inadequate people skills will also lead to weaker products with lots of issues..
Because products are not built by one person, a group of ‘teams’ build them.
And successful teams have people who are good technical and inter-personal skills
And this will drive product quality.
The word automation “framework” is sometimes used incorrectly.
Would you classify Selenium, RestAssured etc. as framework?
IMHO these are libraries, using which we develop frameworks.
Frameworks are built using a number of libraries like the ones mentioned.
There might be some common utilities of your own, some additional reporting you’d add along with a reporting library etc.
In general, a ‘framework’ IMHO will be something you build for your product using a number of libraries.
Linked article for further details:
Have you faced problems like not having enough time to test, or automation not being prioritized?
That means you don’t have your Definition of Done set right..
A story should be considered ‘Done’ only if the ‘Definition of Done’ has been completed.
Enough time to do exploratory tests & automating the important scenarios (Unit, API & UI) should be part of your definition of done.
If these activities are not taken up as part of delivery, then your accumulating technical debt
As your tech debt grows, so will your problems with delivery & quality.
And after some time, your product will be in a heap of mess with everyone scratching their heads about how to fix it.
#RedefiningSoftwareQuality #TestAutomation #Testing #Agile #DefinitionOfDone
Hiring automation engineers has always been a trick thing.
And the worst thing you can do is to add all the buzz words you know of around automation.
Finally when the candidate comes on board they end up doing exploratory testing only!
Here are few pointers for hiring:
- Look for programming aptitude.
- If your framework is in java, not necessary to hire someone with 10 years of Java experience
- Testing acumen.
- Are good in identifying risks
- While a can-do attitude is a must, for automation positions someone who ‘enjoys’ problem solving is required
More pointers in the linked article below
#RedefiningSoftwareQuality #TestAutomation #Automation #Hiring
Have you felt writing and maintaining test case documents a drag?
Because they are…
The old school thinking was, dev write code, testers write test cases
I talk about:
- Why the traditional way of doing test case management is inefficient
- What is a better approach?
- How will this save more time
#RedefiningSoftwareQuality #QualityTransformation #Testing #TestCaseWriting
I’ve noticed few common mistakes teams make in their transformation journey:
- Very detailed long-term planning.
- They have a target state in mind, and then start planning in detail every step of the way.
- Processes, procedures, day to day activities planned in detail
- That is the trap agile was supposed to free us from, we’ve learned we “can’t” plan long term accurately,
- So, don’t design intricate plans
- No long-term plan.
- Then you have teams who believe they are ‘truly’ agile and believe in not thinking more than 2 weeks ahead
- “If you don’t know where your going, chances are you may never get there”
- That too is very problematic, you’ll stay in fire fight mode if that’s the approach
So, what’s the middle ground?
- A) Have a long-term vision.
- Crystalize the vision so everyone knows where we want to go.
- That does not mean over documentation, it means start ‘living it’, should reflect from your actions
- B) Plan few weeks
- Do have an approach to how to get there, but make it very short-term and to the point
- That’s because it will change, you’ll try learn and then try something new
- Your plan to get there must ‘evolve’ along the way
- PDCA – Plan Do Check Act
#RedefiningSoftwareQuality #Transformation #Agile