Daily Posts

2201, 2020

Develop your API Contracts

By | January 22nd, 2020|

I know implementing the automation pyramid is hard,

And to a large extend, is not a problem with just testing practices either..

I’ve seen teams where products don’t have contracts written up properly (API contracts / JSON schemas)

Back end services are not designed for anyone other than the developer to consume..

Such places do make it hard to implement the pyramid, i.e. 70 – 80% tests at the back end,

The solution: The whole team work on developing those contracts, and then write tests for those contracts..

Easier said than done, but until you don’t have that, testing isn’t going anywhere.

#QsDaily #Automation #Testing #ApiAutomation #AutomationPyramid

2001, 2020

Do NOT Automate regression 100%

By | January 20th, 2020|

Automation regression percentage – The metric I hate the most..

Many times the only use of this metric is to provide false assurance that we are efficient in testing.

And the ultimate goal soon becomes to automate regression 100%, which is a bad idea,

And automating just UI tests makes it even worse.

More on why not to use it and what should be done in the linked video

#RedefiningSoftwareQuality #Automation #RegressionTesting #KPIs

1901, 2020

Data quality quick list

By | January 19th, 2020|

Data quality is one of the biggest problems with data science projects,

I’ll be talking about these at the #AutomationGuild, here’s a quick list:

– Accuracy. Is the data accurate in the context to be used

– Validity. Is the data fresh enough, still valid?

– Consistency. Data from different sources / time frames matches

– Completeness. No parts of data are truncated / missing

– Uniqueness. Enough data to uniquely identify records

– Timeliness. Data being collected at the right time & processed in a timely fashion (efficient enough)

More on the conference here:

http://amp.gs/Dydu

#QsDaily #BigData #DataScience #Testing

1801, 2020

Test data and masking

By | January 18th, 2020|

For large enterprises with many interconnected applications, generating test data can be a challenge.

Even more so for big data projects, which is where masking comes in..

Take a copy of production DB from different products around the same timeline and mask the data for GDPR compliance.

Surely easier said than done, but can be very effective.

IMHO three main requirements to build this:

1. A quick guideline to identify what data will be classified as ‘sensitive’ and ensure it’s GDPR compliant.

2. A masking platform which masks value the same way across, helping with consistent data

3. Creation and usage of test environments is efficient and fit for purpose

#RedefiningSoftwareQuality #TestData #Automation

1601, 2020

Using mocks

By | January 16th, 2020|

One of the main reasons for flakiness in automation is test data.

Therefore use mocks to run scripts executing in the CI pipeline.

For most folks, automation scripts are executed only in nightly runs with data created on the fly.

And unfortunately these tests are mostly UI tests, another blunder.

I gave a talk yesterday in which I said – 50% of your automation scripts should run in the CI pipeline triggered on each pull request.

And all these tests should use mocks instead of real data from downstream systems / components.

(I hope it’s needless these tests are on the services / API layer..)

So don’t forget your mocks when automating.

#RedefiningSoftwareQuality #QualityTransformation #Automation

1201, 2020

How to become a well paid tester?

By | January 12th, 2020|

How to become a ‘well paid’ tester?

I call it “The next generation” tester and here are the main skills…

Quality Engineering
– Agile practices and DevOps culture
– Ability to match testing goals with product’s goals
– Figuring out KPIs / OKRs which help ‘deliver value’
– Enterprise & Vendor management

Testing
– Risk based testing
– Defining a practical, efficient and effective test strategy
– BDD practices
– Context driven testing

Automation in test
– Algorithm design aptitude and programming proficiency
– Automation framework design
– API / UI / Mobile Automation
– OAT (Performance and security basics)

Continuous Integration
– Branching & merging
– CI tools (Jenkins / M. Azure)
– Provisioning environments Containers / VM / Cloud)

Technical Leadership- Technical acumen
– Team leadership
– Collaborating with architects- Thought leadership

Details of each in the linked article

#RedefiningSoftwareQuality #NextGenerationTester #TestersGoingTechnical

 

The Next Generation Tester Skill Set

 

 

1101, 2020

DevOps isn’t just sexy

By | January 11th, 2020|

DevOps sounds sexy, but it’s not a piece of cake.

And certainly is not just about Jenkins..

It’s foremost a mind set to deliver at speed and ONLY WHAT’S VALUABLE.

This prioritization of what’s valuable is so under-rated, and one of the biggest cause of failure.

Working on anything that’s not priority is ‘waste’, and waste is what kills it all.

I talk about this briefly in the linked article

Picture – Slide from @Jez Humble workshop showing the ‘cost of delay’ of different features in a specific case study.  (It’s it ridiculous how much we would waste trying to implement all features there!)

#RedefiningSoftwareQuality #EliminateWaste #DevOps #Lean

My key learnings in 2019

901, 2020

Building Psychological Safety

By | January 9th, 2020|

I’ve been reading about psychological safety for while, particularly, how you implement it in a team.

Imagine my luck, had the opportunity to ask that question from @Jez Humber in a workshop..

What I understood mainly was: The leader’s behavior builds or destroys it.

If the leader punishes people for failing or calling out what they feel is wrong, then no matter what, the culture goes down the drain

As an example, he talked about Etsy where a person on the team broke production deployment while following the process, the person was given an ‘award’ for highlighting the problem instead of being penalized.

Link to post here:
https://www.ryn.works/blog/2017/06/17/on-failure-and-resilience

If you want your team to perform, build psychological safety.

To build that, the leaders must be seen as supportive to failures, whistleblowers, people who highlight problems instead of shooting the messenger.

#RSQ #PsychologicalSafety #leadership #HighPerformingTeams

 

 

 

701, 2020

Don’t start with learning Selenium

By | January 7th, 2020|

I very much dislike automation trainings starting with teaching Selenium,

For starters that’s the wrong place to do automation. One should focus more on API tests and do a handful UI checks.

Secondly, automation is a means to an end, not an exercise for the heck of it. Therefore talk about why to automate.

Thirdly, in some cases folks learning automation don’t really learn how to test well either. If your testing is bad, your automation is just going to amplfy that bad.

Lastly, without knowing how the underlying architecture of the product works, IMHO one cannot truly learn how to test well.

So next time you plan an automation training, make sure you get the fundamentals right first.

#RedefiningSoftwareQuality #TestAutomation #Training

2310, 2019

Definition of flaky tests

By | October 23rd, 2019|

Anyone who has written automation scripts has faced flakiness, and we all hate it to our bone.

But everyone defines a flaky test a bit differently..

For me, a flaky test definition could be “a test running multiple times under the same conditions gives different results”

How would you define flakiness?

#QsDaily #automation #flakytests

1910, 2019

Automation without test strategy

By | October 19th, 2019|

Mostly when companies talk about transforming, automation come up as the most hot topic.

While that certainly is important, without your quality strategy & foundations it’s not going to work.

Immediately people start running towards buzz words and new shiny tools.

Knowing what quality means for your product, what to test and how to test should precede any automation effort.

If you used so ship garbage at a slow pace, with new shiny tools and terms it’s going to garbage coming out double time..

#QsDaily #Automation #Testing

1610, 2019

People over process

By | October 16th, 2019|

Ever been in a working environment with a lot of processes?

While I agree ‘some’ of them are needed, but mostly they become shackles and hamper progress.

It’s a fine line sometimes to make a process which is effective and efficient at the same time,

In case of doubt, my personal preference is always to make it more leaner.

After all, that’s what the manifesto says, people over process.

Many organizations begin with the assumption employees are unable to make right decisions

Therefore, there they need ‘processes’ to ensure they operate as expected.

The problem: it’s impossible to come up with a ‘perfect’ process,

So instead of a rigid process, have general guidelines and have faith the people will make the right decision.

#QsDaily #processes #WaysOfWorking

1510, 2019

Automation is meant to help testers

By | October 15th, 2019|

I hate when testers doing exploratory testing don’t look at automation run results…

This mentality of us vs them among Automation engineers vs Exploratory testers has to go

And I wouldn’t blame testers only for this, when management feels automation will help them get rid of manual testers, It’s only natural for testers to feel insecure

For the 100th time AUTOMATION IS MEANT TO HELP TESTING!!!!

A common question I ask teams – If I stop automation runs today, will that change the time you’ll take for exploratory testing?

If the answer is no, i.e. with or without automation our testing time remains the same, then WHY IN THE WORLD ARE WE DOING AUTOMATION!!!

#QsDaily #CommonSense #Automation #Testing #regression

1410, 2019

Big data and data quality

By | October 14th, 2019|

Testing in big data area has typical challenges,

A big factor is quality of data ingested.

The analysis results has a heavy dependency on the ‘quality’ of data ingested (obviously),

What often happens is inconsistency across the data from down-stream sources, missed records, missing data within records etc and other data quality issues.

Unless these issues are flagged at the lower levels, problems creep up and start reflecting in the analytics results.

While having automated data checks on massive data stores might not be an easy job, it certainly is worth implementing.

1210, 2019

Automation Regression Percentage

By | October 12th, 2019|

Have you ever calculated Automation Regression Percentage?

I think it’s one of the most misused metric. Mostly used to show ‘cost savings’ on pretty graphs.

Here’s how to use the metric get some actual value:

#QsDaily #Automation #Regression #Testing #AutomationRegression

910, 2019

Give me a fixed date to automate

By | October 9th, 2019|

A client asked me to ‘Quote a budget and timeline for completely ‘automating testing’ while giving no context of the product.

I refused to work with them (obviously), here’s why:

Even if disregard the ‘automating testing’ ask, which is a BIG ask..

Getting a budget and time estimation of a complete project might be a common ask before any project sign off.

But I consider this the biggest anti pattern to agile.

Decades of practical experience has showed us we are horrible at estimating.

On the other hand I understand we need to budget and calculate what return we might get.

What might be more practical is to have a ball park range for budget and work we can deliver in first three months.

You can extrapolate and calculate the total estimated cost and use for long term budget planning, but remember it’s always going to be wrong.

We get so hung up with the detailed plans & good old

gantt charts (which were invented in WWI) and forget the ground reality.

#QsDaily #projectestimation #agile

710, 2019

Fault injection post Agile

By | October 7th, 2019|

Fault injection / mutation testing is among the things I miss from the old waterfall days.

More on fault injection in my talk at QA&TEST conference this month end.

In waterfall days it was okay to spend time on improving testing and coming up with better testing techniques.

After the agile apocalypse, many teams found excuses to get away with perfecting their craft under the inaccurate pretense of urgency.

In my understanding, agile never meant not to get half cooked stuff out of the door, in fact quite the opposite, important activities should be part of the Definition of Done.

The only difference was to break a big project into very small deliverable.

So essentially all the good practices from before should continue, just implement in bite sized pieces.

Fault injection is one of those practices.

#QsDaily #Agile #Testing #MutationTesting

210, 2019

Fault Injection at QA&TEST Conference

By | October 2nd, 2019|

I am furious on hearing a tester saying ‘this is out of scope’.

I’ll be talking about one way to solve this at QA&TEST this month (https://www.qatest.org/ali-khalid/?lang=en).

Most of the time I’ve seen testers say this because they don’t know how to test that feature.

Instead of trying to go under the hood and understand the architecture and code base, they just raise their hands.

What really should be done is understand how that piece of code can be tested.

Yes developers should be doing unit tests, but even then tests on module / component level should be figured out

In the world of embedded systems and especially safety critical devices, letting a functionality go untested is not in the cards.

Using fault injection / mutation testing is one technique which I’ll be discussing at the conference.

2809, 2019

Big data and Data Quality

By | September 28th, 2019|

Like industry types, testing applications dealing with big data is very different.

One of the biggest quality challenge in the big data space is of ‘Data Quality’.

Big data mostly deals with analyzing large data sets and extracting useful information / insights from it.

The real currency here is data, therefore quality of the data you get will heavily affect the results, hence quality.

What ‘data quality’ would mean for each product & insight will be different.

So the first step is, figuring out the ‘definition of data quality’ for a particular product and the insights we are trying to get.

#QsDaily #BigData #DataQuality

2309, 2019

User Journey testing instead of End to End testing

By | September 23rd, 2019|

I very much dislike the word end to end testing,

It means different things to different people, instead I use testing the ‘user journey’…

End to end can mean:

1. From the UI to the DB level and back (ends of the tech stack)
2. Can mean a user story executed from the UI
3. A feature (group of stories) demo over the UI
4. The end user’s complete journey which cuts across multiple applications

For me, only no. 4 is end to end, but it’s so hard to explain to certain people.

I find testing the ‘user journey’ much easier to get across to the audience.

And this part of testing is the most neglected, where a user flow is going through multiple systems,

Each team’s feels responsible from the start and end of their product only..

#QsDaily, #testing #softwarequality

1909, 2019

Reorgs and centralizing decision making

By | September 19th, 2019|

Many re-orgs are done to change the culture and how decisions are made,

Often I’ve noticed in the process they end up centralizing decision making even more,

This notion of a select group of people having enough background to make right decisions has to go away,

The typical old management style of information flowing up and decisions coming down, like in the military just does not work.

The complexity of today’s work place is far too much for any select group to be able to make all decisions.

Although the re-orgs mean to decentralize but letting go is just so hard,

Ultimately going even further way from the goal of gaining efficiencies by reducing waste in decision making

If we stick to the fundamentals of reducing waste, making these structural changes might become more easier.

For more, refer “scrum” By Jeff Sutherland

#QsDaily #agiletransformation #scrum

1109, 2019

Waterfall from NASA

By | September 11th, 2019|

Did you know ‘phased program’ – (waterfall SDLC) was developed by NASA

And inspecting the failed programs, this process was found to be extremely inefficient

The phased program / gated development process / waterfall model was said to be very good at trying to show an over-glorified picture of the software

The focus was on documenting rigorously and proof.

I guess that’s where the Agile manifesto said – we value working product over comprehensive documentation

Many of our teams might have moved into an ‘agile’ model, but this idea of valuing working software over documentation still has not gone away

We insist on over documenting nonsense stuff, and under-documenting where needed.

Classic example – still maintaining those long test cases, and having very less context around stories or importantly writing feature files to agree on the product’s ‘behavior’ with the team

#QsDaily #agile #testing

1009, 2019

You are paid what you are worth

By | September 10th, 2019|

A guru of mine once said – You are paid exactly what you are worth – I thought he was joking

Looking back, I can say he was right, at least in the long run, here’s one reason why..

I was having a chat with fellow testers the other day talking about tester’s career progression.

The biggest mistake IMHO people make is to not learn how to solve problems..

In today’s world, all businesses are focused on solving their customer’s problems, especially in tech.

Folks hired in these tech firms primary role is to develop solutions to solve these problems.

A few people find this very taxing and unexciting, because they never trained themselves in problem solving.

If you are working in tech, regardless of your position in the org, your primary role is to develop solutions for the target market’s problems.

If you can’t handle that job, you have no place in tech, and let’s then stop talking about career progression (or griping about it).

If you are solving problems, and your organization is not treating you well, you can get a better opportunity and leave.

So, in the end, you are paid what you are worth..

 

409, 2019

Most Important KPI for Software Quality

By | September 4th, 2019|

I figured it out.. the first and most important KPI for software quality..

Can you guess?.
.
Time from ‘prioritizing a feature’ to the time it is ‘ready for deployment’ in production.

Now some might say testers don’t have control over the whole process, that is true but we should be working to develop that maturity

Or we might say time to market is not as important as quality of the product,

My answer would be – Quality is subjective, best to get feedback from people spending money determine quality

That does not mean we ship crappy stuff, some things are no brainier, seeing exceptions, basic functionality not working are obvious

The part which is subjective is, certain features we might feel are very helpful, or as the user expected and therefore give the product a higher quality,

In the end customer’s experience, they might see that as a hindrance in using the product and would have a different perspective of the level of quality here

So, anyone working towards developing software (including testers), the foremost measure should be: How quickly we can go from ideation to deployment.

Thoughts?

#QsDaily #testingmetrics #transformation

109, 2019

Quality Metrics Headings

By | September 1st, 2019|

After years of going around in circles, I’m going to get it done.

A set of quality metrics that help improve productivity, and I need our community to pitch in..

I’ve gone through relevant content I could find, and to be honest there is just so much on this topic, both good and bad content.

The mistake I have done and see other do is, we measure things we ‘think’ will generate value, like automation coverage, but never the actual outcome we want – time to market and product quality.

Also there are some process factors which usually get skipped, which are also responsible for hitting these goals

In lieu of these, want to classify the metrics in three categories with some “SAMPLE” metrics, not an exhaustive list, just to explain the heading better.:

  • Practices maturity
    • g. Scrum ceremonies and best practices in them
    • Three amigo sessions
    • Defining the complete user journey for each epic & feature
  • Continuous testing
    • Automation pyramid
  • Business value
    • Time taken from pull request to ready for demo
    • Trend of issues coming from support which have to be fixed

Under these three headings will be a list for each.

Thoughts?

Load More Posts