Daily Posts

606, 2018

The Marshmallow Test

By | June 6th, 2018|

And it’s implications for automation learners

It goes like this, place a marshmallow on the table, tell the child if they don’t eat this marshmallow for another x minutes, they’ll get a second one, then leave the room.

The jist of it is, kids who can delay gratification will probably be more successful.

And I feel this holds very true for automation learners.

Starting and staying with very basic automation concepts and trying to squeeze every opportunity sitting at that level is not helpful.

Learn to do the grind today and wait for the rewards later on.

Invest in learning how to do automation well instead of doing just the bare minimum and hoping to succeed.

Yes the industry does need automation engineers, but far greater is the need for excellent ones!

#QsDaily #LearningAutomation

406, 2018

Uses of Docker in automation

By | June 4th, 2018|

Here are the few places I’ve used it in:

Create isolated AUT instances
– We have separate DBs for different AUT versions
– Using docker can instantly create multiple AUT instances
– Different approaches to do it, one can be to just hook up multiple DB versions to the same app server

Multiple execution environments

– Selenium grid(s)?
– If your automation tool allows, can create multiple execution environments
– Can hook them up with Jenkins and everything will be done on the fly

I’m sure there are other uses where the AUT is NOT containerized and we can still leverage docker.

Have you tried something different? would love to hear..

3105, 2018

Deciding what to automate – Part 1

By | May 31st, 2018|

Deciding what to Automate?

First let’s talk about what to test

Often when teams talk about what to automate they jump straight to regression tests

I like to start with ‘Do we have a good set of scenarios to test’ to choose from?

What to test would depend on changes, features inmportant for product positioning, most sought out features by clients, features with most bugs from the field and so on..

Once we have that list, we can talk about what to automate.

Here’s the discussion on the subject:

#QsDaily #QsEpisodes

3005, 2018

Ethics, privacy and testing

By | May 30th, 2018|

Does verifying ethical boundaries of a product come under testing, E.g. Data privacy?

If it’s expected by management then off course, if not then?

In this age I feel we’ll have to face more ethical problems than ever before

As technology’s reach increases, more possibilities open, creating more social challenges as a by-product

As a tester, Identifying and factoring end user’s needs and wants is paramount

Keeping a moral high ground has always been tough

The decision would boil down to every person’s core values I guess

Felt this is a question many of us might face in the coming years (if not already)

Thoughts?

#QsDaily

2905, 2018

Whatever can go wrong, will go wrong

By | May 29th, 2018|

We all learn in different ways, lessons that stick usually have more effort and emotion involved

This is one such story of debugging an automation framework problem

There were quite a few lessons we learned and have been vital in our success

If I’d have to mention just one, “Whatever can go wrong, will go wrong” (Murhpy has a lot to answer for..)

So try to figure out what can go wrong and prepare for it

For a brief summary of the story:
https://pos.li/29s544

The complete TechBeacon article here:
https://pos.li/29s545

2805, 2018

Object oriented design principles – SOLID

By | May 28th, 2018|

Perhaps one of the most common and used among the Java community

For automation engineers, specially working with object oriented languages like Java and Python this is important

These will create clearner, less complex and reduced coupling in you framework

The principles might be a bit complex to understand at first,

But are very useful and worth the time spent

#QsDaily #TestersGoingTechnical

2705, 2018

Seen major issues at the last minute or regression?

By | May 27th, 2018|

Many time it’s just inadequate planning

Sometime the change is misunderstood

Assumed as a small / localized / easy fix

One thing leads to another and we’ve opened the pandora’s box

I’ve always felt the first mistake of misjudging the change is the main culprit

BTW, this goes for automation framework changes as well

The rule of thumb I use: “If any other module of even class can get affected by the change, don’t do it at the end”

A few more tips in this article

The ‘not’ so small code changes

#QsDaily #QsArticles #NotSoSmallChanges

2405, 2018

Have we learned from past automation experiences

By | May 24th, 2018|

Jim Hazen and I talk about how we keep repeating the same mistakes..

Some sample cases would be:

– Record and playback
– Automation would render testing team’s unecessary
– Treating automation as a part time activity
– Having the wrong goals to measure automation
….

To be honest, this can be a very long list, but you get the point

Here’s the discussion about why we probably keep running into the same trap every time a next wave of evolution in automation comes around

#QsEpisodes #AutomationMistakes #HistoryOfAutomation #LearningFromMistakes

2305, 2018

My theory on office relationships

By | May 23rd, 2018|

Unfortunately, mostly are love and hate relations, especially between Superiors and Juniors

And here’s what I’ve experienced over the years:

– Every Superior has resentment to every team member, it might seldom and negligible, might be frequently and severe

– Similarly, every team member has resentment towards every superior, it might be seldom and negligible, or frequent and severe

Striking a balance between the love and hate is extremely hard, and equally important

The question is, how to deal with it

Well, everyone has their own way, I try to spread goodness and have no ill-will

It’s tough, seems impractical sometimes,

But my faith dictates, in the end those who spread goodness are the ones with the greatest impact and success

#QsDaily

2205, 2018

Does API Automation skip testing areas of AUT?

By | May 22nd, 2018|

Well, it does and does not.. Let me explain

By testing ‘just’ the API, yes some checks programmed for the client side will get skipped

However, API automation is not to ‘replcae’ UI automation

It is supposed to work in ‘conjuction’ with UI automation

And not only UI, but in coordination with unit tests as well

Therefore, if we look at just API / service layer tests only, then we do loose areas to test

But when working in conjunction with UI and Unit tests, (the way it should), then we don’t forego any code from being tested

#QsDaily #UiAutomation #ApiAutomation

References:

UI Automation Vs API Automation

https://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid

 

2105, 2018

Developing a tester’s Jarvis

By | May 21st, 2018|

There are some ‘dumb’ routine tasks we have to do as a tester

Not all of them require a signifanct amount of intelligence and can be done automatically

And I’m not talking about checking as in testing something, rather activities other than actual testing

Some AUT upkeep tasks like configuration updates, checking installations, maintaining environments, compying stuff from one place to another

Having a single tool to do all of these things might be an overkill in the start

But certainly can prepare a bunch of smaller programs for routine tasks and controlled from one place (Jenkins..?)

Call it a stripped down dumb version of Jarvis if you will..

I know @Alan Richardson has talked about something similar

Care to share something you are using along those lines?

#QsDaily #Jarvis

2005, 2018

How does continuous testing help?

By | May 20th, 2018|

For sure it does NOT save cost upfront at least..

Implementing a DevOps culture and integrating Continuous testing in that will take some effort

A cultural shift will be needed and breaking down of dev / QA silos

Now to the benefits, most importantly provides quick feedback

Coding a feature incorrectly and then fixing it afterwards takes considerable effort

A lot of communication back and forth not to mention time spent regressing around changes

And some times issues are not found until the 11th hour, which is another Pandora’s box

For me the greatest benefit of continuous testing is preventing as many issues from being pushed in the first place

I created an illustration showing this phenomenon of continuous testing avoiding the pile up of issues over time

The article and video can be accessed here:

#ContinuousTesting #Automation #QsDaily #QsArticles

The Magic of Continuous Testing

 

 

1705, 2018

Calculating Automation’s ROI

By | May 17th, 2018|

Is it just a number game in terms of man hours?

Mostly that’s how these ‘savings’ are calculated

The reality is quite different IMHO

Equating man hours to execution time by a machine is not accurate

The way a person would test is a lot different, even if they follow a written test case

Equating apples with oranges does not give a true picture,

Well then how does automation help?

Here’s what I think..

Does Automation Save Money?

#QsArticles #Automation #AutomationRoi #GeneratingBusinessValue

1605, 2018

Evolution in software and changing automation landscape

By | May 16th, 2018|

A segment from the talk with Jim Hazen:

Many times we stumble upon a problem and think, hmm seems no one ever had this before

Especially with automation, we probably have reinvented the wheel quite a few times

Going through the entire history of automation evolution would be very lenghty,

This episode talks about some main events and how automation tools, practices chnaged over the years

Very fascinating and interesting subject, here’s the video link:

#QsEpisodes #TestAutomation #AutomationHistory

 

1505, 2018

Best practices for logging automation results

By | May 15th, 2018|

Here’s what I have learned over the years:

Firstly the test log is not for automation folks only,

Primarily for the whole product team to consume

Plus automation folks would like to help it in their debugging also

Keeping thwese audiences in mind, following should be a part of your log:

– A good heirarchical stucture resembling the test scenarios used for regression
– Lots of debug information indended within the report
– Some info on failures we know are reported issues
– And finally best to have it in a centralized location, Jenkins would be best

More on that here:

Logging automation results – Best practices learned over time

#QsDaily #TheAutomationBlogBook #AutomationInTest

1405, 2018

How much documentation is enough?

By | May 14th, 2018|

From waterfall style having extensive documentation for each SDLC phase till the Extreme programming / SCRUM with very basics like user stories

I must add here, when teams say we’re going ‘Agile’, so don’t need anything except for user stories, that does not make me happy

Too much we run into problems of maintaining them,

Too little, and as soon as one or two key people leave their position, everyone is in trouble

It should be enough to sustain development, and keep the process lean enough

I guess everyone would agree on all the aforementioned, the problem is “Striking the right balance”..

I’d like to propose the following as a yard stick:

– Anyone wishing to understand any part of the code should not have the need to hunt down people to understand. Sufficient commenting and coding practices should be used

– Anyone wishing to learn about how the system should behave in a certain scenario, can find that info without bugging every single developer.

For application domain, one can utilize user stories, gherkin feature files (BDD), mind maps, test cases, client specifications

For code understanding, code documentation, commenting, coding practices, architecture documents

Thoughts?
#QsDaily #RedefiningSoftwareQuality

1305, 2018

Starting an automation project

By | May 13th, 2018|

Make sure to follow the “Pillars of Automation Framework Design”

Unfortunately most teams start with UI automation

Which is the toughest and has the lowest return

Nonetheless it is important, so best to do it right

The 4 Pillars are:

  • Maintainability
  • Reusability
  • Scalability
  • Robustness

Description of those here:

Pillars of Automation Framework Design

#TheAutomationBlogBook #TestAutomation

1205, 2018

Our most important asset?

By | May 12th, 2018|

It’s one which is most scarce – TIME – and the successful know that

It cannot be created or borrowed

Every person has 24 hours, and once they pass, they will NEVER RETURN

The trick is utilize your time in the best possible place you can

Ideally we should do stuff only which we MUST do ourselves

DELEGATE anything else that can be done by someone else

This will take disciple, planning and a burning desire to head towards your goals

PS.
This is important for testers and software development teams too

References:
“The art of self-discipline” by Brian Tracy
“The Millionaire booklet” by Grant Cardone

1105, 2018

Automation experts over the years

By | May 11th, 2018|

A #QsEpisode in which @Jim Hazen shares names of few automation experts over the years

I was familiar with most recent ones, but folks from back in the day I did not know of

Quite a few people have been contributing over the past many yers

Teaching tricks of the trade and their experiences

You either learn by experiencing it yourself or by learning from someone elses experience

It’s best not to reinvent the wheel and learn from those who have seen it before

Here’s the segment of the talk around this topic:

P.S.
This is not a conclusive list. Just few names recalled at that time.

1005, 2018

It’s automation not AutoMagic

By | May 10th, 2018|

Jim Hazen shares the story behind the term he is famous for “AutoMagic”

He was kind enough to spend some time talking about automation and it’s history

Instead of uploading the whole 2 hour long talk as one video, I’ll be creating topic wise smaller ones

If you are not introduced to Jim or the term “AutoMagic”, you would want to listen to this:

905, 2018

How to get around red tape processing

By | May 9th, 2018|

Stuck with processing red tape to get things done?

Here’s one way how I went around it

I was seeing a major lacking in our code coverage for a safety critical device

The only way to test it was through white box testing, having entrenched silos it was not an easy sell

Instead of selling upfront, we went covert to build a case first

Implement and generated results from the intrusive testing technique,

Once we had a business case opened it up to management and

The miracle happened, despite it seeming impossible to institutionalize, the evidence of its necessity was evident

And since then have tried this few more times with excellent results..

Give it a try next time, I’d be eager to know how it went..

805, 2018

Jenkins for CI like WordPress for Web development?

By | May 8th, 2018|

Remember Web front end development 15 years ago, and what has WordPress and other platforms done to it today?

The same might happen with CI processes through tools like Jenkins

The number of tools providing integrations to Jenkins is getting insane

Which is a really good thing, CI and Continuous testing is becoming way important

And getting hooks in there is not as smooth yet, but we can surely get there, probably in the near future

It might feel complex if setting up Jenkins is not approached the right way

Here’s a short guide to the very basic setup and brief description on it’s moving parts:

Jenkins – Installation Background

705, 2018

Most important assets when moving on from your current job?

By | May 7th, 2018|

IMHO – Lessons learned and the relationships you built.

We might have supported financial needs while being on that job

But the lessons we learned through our experiences are far more important

And even more are the relationships we built

On that note, I’d like to thank a team member of mine who just moved on..

Thank you @Arslan for your efforts and your contribution towards the team

I truly wish you all the success and hopefully

You had great experiences with us because I’m sure we build great relationships..

605, 2018

Seeing flakiness in your UI automation scripts?

By | May 6th, 2018|

One area I see neglected a lot is code complexity

I was fortunate to have worked for safety critical devices early on and learn some valuable lessons

One of which is code complexity (nested conditions)

When your code goes beyond a certain complexity level it becomes hard to have a deterministic result

More than often, I’ve seen a single area of code made too ccrammed and complex leading to problems all over

Spread out your conditions, instead of going deep vertically, try to go horizontal in your control flow

More on common UI problems causing flakiness:

https://goo.gl/tSGm8A

Load More Posts