A recent question I got asked:
How to combine Web & mobile UI automation frameworks together?
Now there will be no one size fits all answer here, but here’s what my experience has been:
1. Most teams try to run every test through selenium, and then the same through mobile (Appium, XCUITest etc..)
And that is the biggest problem – don’t have a lot of tests running there, most of you testing should be completed on your services level
2. Depending on your product, there might not be a lot of synergies between the tests you have on web & mobile, keep an eye out
3. Keep your frameworks very modularized, every component can be reusable
4. The utils layer can be shared across both frameworks, can have it as a separate project, added to the web & UI ones
5. Similarly test data / configs should be in sharable format. My favorite is JSON structures & config files
6. Having both web & mobile tests within the same project might not be a bad idea either.
FYI – Google maintains one code repo for all products. But be careful, if you don;t structure your product well you’ll run into trouble – Make sure everything is loosely coupled
These are few on top of my head, anything else you can think of?
#RedefiningSoftwareQuality #TestAutomation #MobileAutomation #WebAutomation
What does shift left really mean?
Here’s my understanding and what it means to implement it:
Any activity to help us prevent or identify issues BEFORE someone spends dedicated time testing the change – Shift identification of issues & risks as early as possible.
I’d implement this by doing the following:
1. Have three amigo sessions before coding
2. Practice TDD & have decent unit test coverage
3. The developer and any other person on the team together test the change on the developers machine
4. Have quality gates on the pull request pipeline including static analysis, unit tests & component test
By the time engineers testing the feature start exploring in detail, most of the issues should have been fixed already.
#RedefiningSoftwareQuality #ShiftLeft #ContinuousTesting #DevOps
An important part of transformations is building technical enablers.
Initiatives around Digital Workspace were been worked on with a lot of vigor at Emirates Group IT for some time
As a result of all that effort we were able to transition smoothly to working from home across the landscape
I had always taken this for granted, being IT professionals lot of these things are common place us
However the magnitude and scale at which large scale enterprises operates, ability to pull this off on such a short notice is an achievement.
Realized this today in a CTO listening session where Microsoft’s head UAE region was sharing his experiences post COVID19.
Bottom line, if you want to truly transform your team, you’ll have to build the technical enablers they need.
Building self autonomous teams is a key change in transformations,
And a key element in there is how to delegate decision making?
Typically enterprises have large groups of people with layers upon layers of decision makers
Decisions and actual hands on work are done in two different groups
As mush possible, decision making has to be delegated, and here are the two key components:
1. Competency. Develop the teams technical skills to have the background to make those decisions.
2. Clarity. Access to information and ability to process, to allow teams to have all the background information they need to make decisions.
Both are easier said than done, but without them the transformation will fail.
#RedefiningSoftwareQuality #QualityTransformation #Transformation #HighPerformingTeams
For engineers technical skills are vital,
Unfortunately so are people skills, because ultimately we have to deal with humans.
And you’ll find every kind of them in a work environment.
Problems occur when an individual or organization neglects any one of these skills
Less technical skills will lead to weaker products with lots of issues,
Inadequate people skills will also lead to weaker products with lots of issues..
Because products are not built by one person, a group of ‘teams’ build them.
And successful teams have people who are good technical and inter-personal skills
And this will drive product quality.
The word automation “framework” is sometimes used incorrectly.
Would you classify Selenium, RestAssured etc. as framework?
IMHO these are libraries, using which we develop frameworks.
Frameworks are built using a number of libraries like the ones mentioned.
There might be some common utilities of your own, some additional reporting you’d add along with a reporting library etc.
In general, a ‘framework’ IMHO will be something you build for your product using a number of libraries.
Linked article for further details:
Have you faced problems like not having enough time to test, or automation not being prioritized?
That means you don’t have your Definition of Done set right..
A story should be considered ‘Done’ only if the ‘Definition of Done’ has been completed.
Enough time to do exploratory tests & automating the important scenarios (Unit, API & UI) should be part of your definition of done.
If these activities are not taken up as part of delivery, then your accumulating technical debt
As your tech debt grows, so will your problems with delivery & quality.
And after some time, your product will be in a heap of mess with everyone scratching their heads about how to fix it.
#RedefiningSoftwareQuality #TestAutomation #Testing #Agile #DefinitionOfDone
Hiring automation engineers has always been a trick thing.
And the worst thing you can do is to add all the buzz words you know of around automation.
Finally when the candidate comes on board they end up doing exploratory testing only!
Here are few pointers for hiring:
- Look for programming aptitude.
- If your framework is in java, not necessary to hire someone with 10 years of Java experience
- Testing acumen.
- Are good in identifying risks
- While a can-do attitude is a must, for automation positions someone who ‘enjoys’ problem solving is required
More pointers in the linked article below
#RedefiningSoftwareQuality #TestAutomation #Automation #Hiring
Have you felt writing and maintaining test case documents a drag?
Because they are…
The old school thinking was, dev write code, testers write test cases
I talk about:
- Why the traditional way of doing test case management is inefficient
- What is a better approach?
- How will this save more time
#RedefiningSoftwareQuality #QualityTransformation #Testing #TestCaseWriting
I’ve noticed few common mistakes teams make in their transformation journey:
- Very detailed long-term planning.
- They have a target state in mind, and then start planning in detail every step of the way.
- Processes, procedures, day to day activities planned in detail
- That is the trap agile was supposed to free us from, we’ve learned we “can’t” plan long term accurately,
- So, don’t design intricate plans
- No long-term plan.
- Then you have teams who believe they are ‘truly’ agile and believe in not thinking more than 2 weeks ahead
- “If you don’t know where your going, chances are you may never get there”
- That too is very problematic, you’ll stay in fire fight mode if that’s the approach
So, what’s the middle ground?
- A) Have a long-term vision.
- Crystalize the vision so everyone knows where we want to go.
- That does not mean over documentation, it means start ‘living it’, should reflect from your actions
- B) Plan few weeks
- Do have an approach to how to get there, but make it very short-term and to the point
- That’s because it will change, you’ll try learn and then try something new
- Your plan to get there must ‘evolve’ along the way
- PDCA – Plan Do Check Act
#RedefiningSoftwareQuality #Transformation #Agile
Releasing quickly and with less toil / effort allows you to:
1. Get feedback from customers quickly
2. Improves your product quality
3. Helps you earn more money, fast..
To start on this ‘cultural’ change, include all the right things in your DoD – Definition of Done.
And this should be the effect:
The ultimate KPIs for software development?
Here are the top 3 I feel are important:
1. Defects from production – Severity and quantity.
– This should be the end goal that should matter.
– Most of the other quality KPIs like automation coverage % etc. are not the end goal,
– They are how good you are following ‘a process’, which may or may not achieve in the ultimate goal.
2. Mean bug identification time
– Average time taken to report bugs after code commit.
– This does not mean just JIRA tickets raised as bugs, automation results showing failures giving feedback too are included.
– This will capture how quickly we are giving feedback on code changes, which is a significant part of driving costs down.
3. Release cadence
– How quickly do we release a feature, starting from conceptualization till deployment into production.
– This should include all the lead time across all processes.
– The company that masters to speed this process will win, since they can adapt and change quickly looking at how customers respond.
There can be many other KPIs, I personally want to care about only these 3.
#RedefiningSoftwreQuality #DevOps #KPI #Automation
Some wisdom I learned about being an effective communicator:
Speak in such a way that others love to listen to you.
Listen in such a way that others love to speak to you.
Learning to speak is relatively easy. Learning to listen well is harder, and is more important.
In case your wondering, vital skill for improving your product quality..
#RedefiningSoftwareQuality #Communication #Collaboration
Assessing skills of individuals is required. Be it academics, interviews, courses / certifications and so on.
IMHO most assessments are not optimally designed which creates a lot of problems…
My biggest issue: assessments are designed not based on what is practically done on a day to day basis, instead on theoretical, trick questions which are far from practical application.
Most of the time, that’s because perhaps these theory-based ones are easy to design and ensure you’ve ‘read’ the text.
I don’t endorse this even for academics, but in interviews especially I think these are just nonsense.
Judge the person on what practical value they will bring, or value they have delivered before.
Oh, BTW, I add test like ‘codility’ in this category too (when was the last time you needed to solve O(N Log N) for work???)
In an automation training program I put together, introduced folks to learn about risk based testing.
While there is a lot that can be said about it, here are two main types of risk assessments:
Inside out risks:
– Look at how different components across your tech stack interact with each other
– Walk through the control structure (#STPA could be good tool) and highlight risks
Outside in risks:
– How your customer sees your application, from the outside in, the behavior of the application
– This is what usually testers are more focused on
It’s important for automation engineers to be good testers first, and therefore practice ‘testing well’
I am also perturbed when testers are unwilling to do ‘inside out’ risk assessment, IMHO mostly that’s due to lack of willingness to get into technical details.
Reference links in comments
#RSQ #Testing #RiskBasedTesting #RBT
Risk based testing: https://www.satisfice.com/download/heuristic-risk-based-software-testing
Learn to generate, find and use data. For instance, this report from google:
This data can be useful for a number of industries / organizations.
For governments, they can use it to find correlations between new COVID19 cases and mobility patterns across regions and to build policies to phase out lock downs slowly
Similarly for testers, having patterns of failures / issues of bugs from production & automation runs can be extremely useful to concentrate efforts on specific areas.
Case in point, data is the new currency – Generate it, find it and USE IT
#RedefiningSoftwareQuality #BigData #COVID19
IT transformation projects might follow different models, and might have slightly different goals,
But at heart of each of them is : Eliminating WASTE.
I feel WASTE is not just a software development problem, but a challenge in everyone’s daily lives
It’s a bigger problem in software development for sure,
And hence number of practices evolved over decades to eliminate this waste.
– TQM / Lean manufacturing: Invest only on what’s required, do it right the first time (And a bunch of other stuff)
– Agile & scrum: To focus / develop what’s important
– DevOps: Develop enablers to push deployment faster
– Agile frameworks: To make organizations leaner
So if your transformation project isn’t ‘ACTUALLY’ eliminating waste, you are missing something.
#RedefiningSoftwareQuality #Agile #Lean #DevOps #EliminateWaste
In BDD workshops I briefly talk about feature slicing. In a nutshell:
Slice features so every feature should be deployable independently.
For this to happen, slice you features ‘across’ the tech stack, NOT ‘along’ the tech stack. (See attached images)
This is a common mistake, and seems easier to work that way.
For instance, develop the UI first, then the back end / business logic.
The problem here, your not deploying quickly / in small chunks.
Might as well call this waterfall development, because then that’s what this becomes..
#RedefiningSoftwareQuality #Agile #FeatureSlicing #AgileTransformation
Developing change agents in an org transformation is crucial.
Few important things to consider when doing this:
Change agents should be competent / an expert in the relevant field. This might seem obvious, you’d be surprised how many times I’ve seen this being violated
– It’s not uncommon to have resentment and a very pessimistic view about the change.
– Your change agents need to be positive and energetic personalities. (BTW Most people with the right motivation can cultivate this)
– The goals and vision should be clearly ‘demonstrated’ to them, not just explained in slides.
– Living, breathing, practicing management demonstrating the change gives clarity
Any kind of transformation is not easy, but if any entity does not adapt / change, will die out eventually.
#RedefiningSoftwareQuality #QualityTransformation #Transformation
Situation for many businesses is becoming desperate
In such times, it’s very important to stay positive:
1. Try not to feed on doom and gloom news and look on the bright side
2. All of us might have things we have been putting off to do, which can be done now
3. Be compassionate to others. It’s a stressful time for all
This will end eventually and things will get back to normal.
Tough times don’t last, but tough teams do…
Avoid taking decisions which will impact you far after things go back to normal !
While teaching a Java class yesterday I realized something:
Years ago, we didn’t have a lot of platforms to help develop code quickly.
People learning to code back then had to understand fundamental concepts in detail to develop an application
As things have advanced, platforms have emerged which a lot of heavy lifting for you.
That makes writing code quick and easy, however I feel students then lack fundamental concepts
This creates challenges when they have to debug programs
Some programming concepts might not seem like important to write code, but help a lot with developing and debugging programs.
#RedefiningSoftwareQuality #Automation #Training #LearningToCode
BDD workshop is the first course in the automation training I put together,
Here are few of the important points I try to drum in:
1. BDD is NOT just cucumber. I always ask how many people know cucumber, depending I spend time on them ‘unlearning’ that first.
2. How come people end up having different understanding of what is to be implemented
3. How does the three-amigo session help bridge this gap
4. How to do example mapping and difference between rules and examples
5. A quick brief on feature slicing
6. The best way to specify something is to describe how to test it
We do a few group exercises to instill the message across the 2.5 – 3 hour workshop, these are just some of the points.
#RedefiningSoftwareQuality #BehaviorDrivenDevelopment #BDD #Testing
‘Sometimes we are the problem, and need to change ourselves’ – This should be the corner stone of every transformation
Transformation projects usually are more focused around technology and ways of working
While that’s important, the willingness to change, especially at the management layer is vital
Black death of childbed case in point, for decades doctors refused they were the cause of deaths and kept on looking for other reasons (more in linked video)
Transformation starts at the top, and results will trickle down.
Will be speaking about IT transformation at #AgileTestingDays-US in June
#RedefiningSoftwareQuality #Transformation #Leadership
Few important things I stressed on while delivering a BDD / Three Amigo session workshop today:
Cycle of BDD
1. Collaboration to clarify system behavior
2. Formulation of behavior in business terminology
3. AT THE END, Automate documented behavior – WHERE POSSIBLE!
BDD != Cucumber !!
Write stories in the format: As a
Slice your stories vertically, each story should have an action and corresponding behavior
Write stories, rules & examples clearly, so someone can understand it even 6 months later
#RSQ #BDD #ThreeAmigoSession #Agile #TestAutomation
It might seem common wisdom for every software product should be able to deliver at a daily / weekly cadence,
However the Gartern’s Pace layered application strategy would beg to differ.
Some products require a lot of experimentation and rapid adaptation according to the changing surroundings.
However for other products being precise and efficient are far more important.
And that’s in a nutshell what the PACE layered strategy talks about, and divides software products in three types:
– Systems of innovation (revolutionary product)
– System of differentiation (improved product)
– Systems of records (efficient & legacy product)
#RSQ #ProductStrategy #DevOps