Daily Posts

Autonomy is a cornerstone of agile transformation. However I’ve seen it taken the wrong way too.

IMHO, autonomy means:

– Self organizing teams who don’t have to wait for someone to take decisions on their behalf

– Who have the autonomy to estimate work (within reasonable guidelines) and drive decisions related to completing the work

– As they can take technical decisions, so are they responsible for those solutions – you code it, you own it

What it does NOT mean:

– Create a bubble and make sure no one from the outside has any visibility into the working of the team (no one can come to our stand ups / retros or other ceremonies). Agile is about transparency remember!

– Take autonomy as a ‘do whatever and get away with badge’ – for example, give unrealistic estimations and having no technical reasoning for why it’s going to take that long.

A big factor in all this I guess is having a purpose and being driven also.

And make no mistake – this will have a direct impact on quality of your product – and hence should be factored into the KPIs to measure.

What is your experience on autonomy in teams?

Introduction to data pipelines in 2 minutes:

For analytics, AI products lots of data is needed – commonly dubbed as Big data

To get the data needed, and in a usable format – we need to pass it through a lot of different data processing stages, a collection of which can be called data pipeline.

Data pipelines have 3 main stages:
– Ingestion: Gather data from various sources in different formats
– Data hub & warehouse – Cleanse, model data at different stages
– Analytics – Run analytics or use specific data sets for machine learning algorithms

Join me at TestBash New Zealand Online 2020 where I’ll talk about a lot more around testing in big data projects!

How would you go about testing an AI product? Join me at QA&TEST Conferences where we will debate the topic on Oct 30.

I’ve been fortunate to work on that problem a few times in past years, in my experience this is not a straight forward answer..

Biggest reason for me – it’s not easy to establish clear oracles (I didn’t use the word requirements..) which creates a complete new dynamics to test these products.

The test objectives of these systems are therefore going to be a bit different, and wouldn’t be just about testing the actual model,

I’d stretch it from the beginning – ‘capturing of data’ and then take to the other extreme – ‘is the model predictions good enough’ – not an easy answer to give, but worth investigating.

Anti-agile test case writing:
– Extremely lengthy test cases for every scenario anyone has ever thought of, and make sure not to rank them
– Then do a ‘regression testing’ cycle and try execute all of them
– Then try to code ALL the written tests as automated scripts

Test case writing the Agile way:
– Collaborate to identify acceptance tests in Three amigo sessions – these are highest priority scenarios
– Automate tests across the tech stack – and DON’T document them – they exist in your code already
– Write checklists to serve as heuristics / references to do exploratory testing

Some reminders for me after being part of Program Increments (PI) since past week..

1. Have a rough estimate of your capacity plans prepared before hand. That includes public holidays, annual leaves

2. Have your features and related stories defined and get the roughly prioritized before planning meetings

3. PI plans (typically across 6 sprints / 3 months) are ‘plans’ – not to be set in stone and can fluctuate over course of the PI

4. Mark your dependencies and get them prioritized – especially critical dependencies

5. Above all – PI planning is about collaboration and figuring out where you time is best spent across next 3 months – to deliver value to end customers

About:

PI planning is a ceremony in scaled agile framework for release trains (group of scrum teams) to plan for the next program increment

While defining KPIs, have both – Leading and lagging indicators.

Often I’ve seen people not making that distinction which creates misconceptions about what the KPIs are saying

Leading indicators:

Predictive measurements used to influence change. For example – how good are we at creating & working with user stories.

These are not the actual change we want to see, rather the change which will ‘lead’ to the outcomes we are hoping for

The problem I see – folks take these as the ‘ultimate’ outcome to measure, which is the problem

Same goes for automation – an common one is ‘% tests automated’. This ‘might’ be a leading indicator in some situations – but is definitely not the ultimate goal.

Lagging indicators:

Measuring the outcome we are actually looking for. For transformation projects could be e.g. 50% reduction in bugs from the field, 40% reduction in lead time

So next time defining KPIs – do classify them as leading and lagging indicators.

Both are helpful – but be careful not to measure a leading indicator assuming these are the ultimate objective.

Algorithm design aptitude – IMHO second most important ingredient for an SDET / Engineer

My definition algorithm design aptitude: Ability to design a complex solution using small building blocks.

And to do this, you don’t have to start with Java. The first language I learned in high school was HTML & CSS.

I learned how to find small building blocks, connect them and create a solution.

Those fundamental lessons I used everywhere – at Uni in my engineering, on job in test automation, testing, web development, big data, you name it.

And this is what I teach and stress upcoming engineers / SDETs to learn – languages will come and go, tools pop like mushrooms all over

What will stay with you, and help you through all of that – ability to visualize & design solutions using existing small building blocks..

Oh, and by the way – We all do have an aptitude to design things – we just need to learn to harness it.

As part of the automation training program I designed for Emirates IT, conducted another online session of the BDD workshop today..

While designing the course, I deliberately kept a very small portion on Gherkin and using cucumber,

and more focus on why and how to collaborate as part of BDD,

Unfortunately most people as soon as they talk about BDD – the first thing they mention is cucumber – and forget the whole conversation that’s supposed to happen before that.

If your team is just starting your DevOps journey, where to start improving testing from?

Begin with investing in automating your services layer first.

Folks assume that’s just about writing API tests, to me this is more of a mind set change as well

The rule of thumb is, test nearest to where the production code is written,

For instance, any functionality written within a micro-service, should be tests at the micoservice level, functionality on the mesh gateway should be tested there and so on.

DevOps tools are not magically going to test and debug problems for you, unless the tests are planned at the right place, they will be flaky and have overhead while debugging.

#RedefiningSoftwareQuality #DevOps #QualityTransformation #TestAutomation

If automation is not happening within the same sprint, who is at fault?

We testers immediately feel that’s somehow our fault – whereas in 90% of the cases I’ve seen it’s not.

I spoke about this yesterday, and some folks were not confident about this being practical (https://lnkd.in/en8GqEp)

Automating within the same sprint is the ideal world every team should target towards

However if that’s not happening (and mostly it doesn’t), that speaks more towards the ways of working of the organization and team instead of tester’s skills

The first step towards this is ‘commitment from management’ to treat automation equally important to developing the feature.

As long as we treat automation as an after thought, there is no way this (aka in-sprint automation) can become a reality – And this is most definitely not a testers issue!

#RedefiningSoftwareQuality ✔ #DevOps #TestAutomation #AgileTesting

Should automation be one sprint behind?

The argument in favor of this is mostly – we want to deliver quickly and automation slows us down.

IMHO there are a couple of reasons for why this is the argument folks make:

– Definition of Done not defined
– Testability not built into the product architecture
– Automation is only the automation engineers job
– Automation is focused only on the UI
– Automation is considered only tester’s job..

If we agree with that, we are consciously making a decision to incur technical debt and have problems down the line.

To build your product well, you have to spend do things properly and upfront thinking.

#RedefiningSoftwareQuality ✔ #QualityTransformation #TestAutomation #Testing #Agile #AgileTesting

Learning to test well is equally important as to code well..

For decades we’ve managed to keep the wall between dev & test, now it’s just common knowledge that you can be either or.

I can’t see a justification of why an individual can’t have both skills..

Let test plans just be ‘plans’ – not fine print of a contract to limit the liability of testers!

I know it’s not fair sometimes testers have a lot of ground to cover and are unfairly blamed if bugs come from production

I talk about the problem the way I see it and how teams can go about making test plans something to bridge the gap – not fortify dev & test positions in their own bubbles..

Is there a proper way of fixing bugs?

Yes – Fix them AT THE SOURCE

I’ve noticed many times engineers lacking simple understanding of ‘due to which design flaw’ a bug is coming up

And as we humans commonly do, engineers try to fix the ‘symptom’ not the problem.

Let me explain:

A new field introduced in a form on a web page. This data is used by this application, also this data is fed into other applications.

Now our new field here is one big text area, where users are expected to enter let’s say name and phone number.

Naturally every person will write the values in a different order & different formats, which will lead to complications for applications using this.

Now to handle the varied type of inputs we can get, engineers spend time trying to build Regex to ‘extract’ the name & phone numbers. – This to me is suppressing the symptom

The thing to fix is – break that giant field down into first & last names and phone number fields separately with field validations..

This might seem very obvious, you’d be surprised how many times we get this wrong..

#RedefiningSoftwareQuality ✔ #Bugs #Development

✔️ Follow me to transform your quality practices

Is there a proper way of fixing bugs?

Yes – Fix them AT THE SOURCE

I’ve noticed many times engineers lacking simple understanding of ‘due to which design flaw’ a bug is coming up

And as we humans commonly do, engineers try to fix the ‘symptom’ not the problem.

Let me explain:
A new field introduced in a form on a web page. This data is used by this application, also this data is fed into other applications.

Now our new field here is one big text area, where users are expected to enter let’s say name and phone number.

Naturally every person will write the values in a different order & different formats, which will lead to complications for applications using this.

Now to handle the varied type of inputs we can get, engineers spend time trying to build Regex to ‘extract’ the name & phone numbers. – This to me is suppressing the symptom

The thing to fix is – break that giant field down into first & last names and phone number fields separately with field validations..

This might seem very obvious, you’d be surprised how many times we get this wrong..

#RedefiningSoftwareQuality ✔ #Bugs #Development

Is testing only tester’s job?

All engineers on the team should be involved with testing in some shape or form..

Doing exploratory testing, providing enablers for exploratory tests

Writing automated tests, fixing scripts, creating enablers for automated tests

Some argue – Testers have a unique skills, due to which only they can do testing

IMHO we haven’t been very good at explaining ‘how to test well’ and therefore find it hard to train developers to test well

If you want get on the DevOps journey, This should be part of your DevOps transformation..

#RedefiningSoftwareQuality ✔ #Testing #TestAcumen #QualityTransformation #DevOpsTransformation

Due to many reasons, a wide spread perception about testing has been ‘the blockers to release’,

Not just because of time, IMHO also because the internal goal of testers is somewhat ‘stop the release from going’!

Here how I’ve seen it happen:

– Testers have lots to cover, and less time
– Anything goes wrong in production, tester’s get blamed
– To avoid that situation, testers try to mount substantial evidence that there are risks in the release, so ‘Release at your own risk!’

The main problem here is NOT testers or testing – it’s this blame game mentality

To improve quality in such a setup, start working on build psychological safety first, rest will follow..

A 20 second intro to risk based testing..

What – Identify and rank features according to risks, and focus on high risk areas

Why – Instead of buck-shot approach, try a targeted approach to testing

Two approaches – Inside out risk analysis & outside in risk analysis

Inside-out risk analysis: Brainstorm internal working of a component, walk through the design and understand the vulnerabilities, threats & victims

Outside in risk analysis: Explore a functionality from the outside, use different type of ‘risk checklists’ like risks from past experiences, ‘ilties’ list, domain / industry specific check list

Best to create risk matrices giving an overview of different features, their risk level and associated heuristics to use

Test plans are not ONLY to improve quality,

They are also a defense mechanism against – why was this bug not caught

More like a legal disclaimer saying ‘Release at your own risk!’, we’ve done what the test plan says

This behavior leads to adding in fields like the ones shown in the image..

So instead of trying to build a defense like an attorney, let’s try and build a test strategy to which EVERYONE agrees and contributes towards..

Something one of the points I discussed in yesterday’s webinar. If you missed it, you can still register, will put up the recording later.

#RedefiningSoftwareQuality ✔ #QualityTransformation #Testing #TestStrategy

Don’t just ‘understand’ requirements – IRON OUT requirements !

Let me explain..

No one has a clear picture of exactly what we want to build, that is the reason we build iteratively.

Therefore while discussing requirements, engineers are mostly interested in – just tell us what to develop.

The approach instead has to be – let’s ‘brainstorm’ what to develop.

The product owner should have a decent picture of what they expect, but they’d lack technical insights

Plus they CANNOT know everything and most proably have not thought of all the risks / assumptions

It takes different engineers wearing different hats (Dev, test etc.) to point out assumptions & risks BEFORE we lock in what to develop..

PDCA – Plan Do Check Act..

The old and gold secret to superior quality

Although PDCA’s roots were from TQM, I feel its the same concept of ‘lean’ which has been defined and refined through Agile & DevOps

It’s all about planning ‘enough’, doing what’s important, delivering it and analyzing the results.

#RedefiningSoftwareQuality ✔ #Agile #Lean

Testing does not have to be ‘they’ vs ‘us’.

Traditionally teams talk about what should be done ‘before’ a user story is handed over to testing.

Shouldn’t be that way, if everyone is testing in some capacity, the need to ‘hand over’ wouldn’t be there anymore.

#redefiningsoftwarequality

What is a PR pipeline?

PR (Pull request) pipeline is a CI pipeline to run tests on EVERY pull request.

Let me explain the terms in there:

– CI : Continuous Integration – term used to run jobs to merge code, deploy code, run automated tests and give alerts.

– CI pipeline: A group of CI jobs stringed together to complete a set of activities on any event trigger

– Pull Request : A term used by Git users when an engineer has created some new code on his feature branch and raises a request to merge his / her code with the master branch

– Tests on PR : Tests running on every PR have to be very quick and localized since the application is not deployed at this stage.

Let’s repeat, what’s a PR pipeline?

Tasks & activities that run once an engineer raises a Pull request.

For more details on which tests can run on a PR pipeline refer blog post on #DTMS (link in comments).

Also doing we webinar on #DTSM, again link in comments.

#RedefiningSoftwareQuality ✔ #TestStrategy #DevOps #DevOpsTestStrategyMindmap #TestAutomation

Trying to change behavior – Carrot & stick might help..

Unfortunately most of the time we don’t change until it becomes a necessity (the stick).

That’s why ‘necessity is the mother of all invention’.

Recently I’ve been banging about a specific topic with teams for a while, and somehow still struggle to get buy in.

Once, out of necessity they are being forced to do it and they have no choice, suddenly all the problems are starting to go away.

Take remote work for example. Pre-coivd some folks were hell bent that remote working cannot work.

Now that we all are forced to work from home, suddenly those problems don’t feel challenging anymore.

The ‘stick’ can be many things – a declaration to achieve something, create circumstances to force certain behavior.

Summary:

Preaching, explaining, collaborating is all needed to change behavior – but sometimes so is the stick.

#RedefiningSoftwareQuality ✔ #Testing #Automation #Transformation

Are testing skills and technical skills mutually exclusive?

IMHO:

1. let me start from: being technical does NOT mean being an automation engineer

2. Technical skills would be: Understanding of how your product’s tech stack works, ability to understand the product’s control flow and code base.

3. Having testing skills does not mean you cannot have technical skills or vice versa

I think the problem stems from when we see technical skills and exploratory testing on two opposite sides

To me they are both very related and compliment each other, one SHOULD fuel the other.. therefore:

1. One cannot test a product very well UNLESS they understand the tech stack

2. One cannot develop a great product UNLESS they understand looking at the product from different perspectives..

This argument of testing vs development skills is not helping anyone – ALL ENGINEERS ON THE TEAM SHOULD KNOW BOTH

Yes some would be better at on thing, and might play a bigger role there, but should have the capability of doing both.

That’s why the very first thing I talk about in #RedefiningSoftwareQuality is #TechnologicalExcellence

#RedefiningSoftwareQuality ✔ #Testing #Automation #Transformation