Original subtitles

So this course is a case study. I tested

an application. It was tracks and I made

a bunch of notes and I tracked what I

did and I did debriefs as I was testing.

So I've recorded all the videos for

those and they will be injected in the

course. You will also see sections like

this which are essentially theory which

are slide based and I'm going to talk

about the aspects that I was working on

because what I've done is I've uh taken

an application

tested it to try and demonstrate how to

test it um in practice. So I'm not

covering all the theory of testing right

because with each application we will

only ever use subsets of our knowledge

but in practical terms if I look at a

single application and a single feature

I can go into real detail on what I

actually did and why I did it and

hopefully that will come through. Now

what I've done as a result of this is

I've modeled the uh basic process that I

used and the basic uh I guess sessions

that I've used when I was doing testing

and I've put them here. So you'll see

this diagram as we go through. This

diagram is not meant to be um a full

scope of what we do when we're testing,

right? It's what I did when I tested

this particular application. In any part

of your applications, when you're

testing it, you may have other sessions

that you use. Um, I have generic

sessions in here. Like the exploratory

session is very generic. I can do

anything in there. The admin section is

basically if there's something I did

that I haven't covered, I'll throw it in

the admin section. So, that could easily

be spun out. But I wanted to try and

simplify the important aspects of what I

was doing. Planning, setting things up,

checking things are working, doing some

recon and modeling and doing and the

difference between a coverage session

and an exploratory session. An

exploratory session, a technical

session, debrief. These are really

important aspects. An admin is really

important, but it covers a whole bunch

of processes. So when you see this, this

isn't saying this is what you do. This

is saying this is a high-level

diagrammatic representation of what I

think I did when I was testing and I

didn't necessarily do it in this order

although it came close. Um so you would

mix and match the different aspects even

though it's showing it as a flow model.

It's looping around. We do different

things at different times. I just

haven't put all the connections and

relationships in there. So you will see

that as we go through. Now this is a

micro case study. So essentially I

performed about a day of testing. So 4

hours of hands-on testing that was

pretty intense. And then I debriefed

that. Then I tidied up my notes. Then I

thought about it. Then I figured out

what I was going to do next. So all

those extra bits are kind of equivalent

to the admin processes we would do that

would take up probably like a full 8

hour day, but we never have full 8our

days at work. So this is about a day day

and a half of actual testing wrapped

into this. Um then what I'm doing is I'm

looking at what I did at a micro level

and extrapolating it up to a what can I

learn in general about testing at a

macro level. That's what this means. And

so it means not everything that I've did

will be discussed. Not all testing will

be discussed. We won't look at all

testing techniques. But everything that

I'm talking about has some context. So

there's quite a few lectures in here. Uh

there should be uh slides available and

the PDF for my notes and we'll see what

else we we put in here. But hopefully

there's enough in there to give you an

understanding of how I approached

testing. And do ask questions as you go

through. Leave um comments because then

I may be able to answer by text or um

possibly create another lectures. But

it's useful to know how much of my test

process and thought process actually

gets communicated through this.

So this micro course is a spin-off of a

talk that I'm going to do. And I'm going

to do this talk twice. It's an unusual

thing. I'm going to do this talk at an

online conference in December and a

conference in America in April 2020.

Don't normally repeat talks, but uh I'm

making I'm trying to make my life a

little bit easier. And as part of that,

I'm also going to record it as a micro

course for Patreon. Now, the the intent

for this is

I get asked, how do I test? How do I

know what to do? What tools do I use?

what was the thought process etc etc and

that's this course so but in order but I

don't want to work in theory so in order

to work as a practical exercise I'm

actually going to think through from

start to finish and these are going to

be short sessions so finish is a fairly

arbitrary term um how I go about testing

a web application and this is a kind of

micro example so what I'm going to do is

once I've done a test session I'm going

to go macro to that and see what lessons

learned learned that apply across the

board to other sessions. So, I'm going

to use the tracks API.

Okay, so this might seem a little bit of

an odd section, choosing the right

person for the job, right? Cuz I am

testing this on my own. But what I'm

really talking about is the notion that

I had to choose which application I was

going to test. And I did that based on

uh my technical knowledge, my

understanding of the web, my

understanding of uh open source

applications, my knowledge. So I'm

choosing the right application for me.

But in the real world, this translates

into choosing the right person for the

job. So I've chosen tracks, not because

I've used it before, although I have. So

my familiarity with it before means that

I know that it's a web app. It's got a

lot of JavaScript. It has an API, it has

a database, there's a lot of scope for

testing, there's a lot of richness in

there. So tracks is a an open-source

to-do management system where you create

actions, next actions. You put them into

context, i.e. where are you going to do

them, and you can create projects for

them, and you can set up reminders. and

it's a a to-do app based on um Dave

Allen's getting things done methodology,

but ignoring the domain, it's a very

useful open-source app that I think is

very easy to install because I can

download a a virtual machine or docker

and use it that way. So, I'm using

tracks

and I have used it before, but it's been

so long since I used it and previously

I've looked at different parts in the

application. And so every time we

approach a different an application,

we'll learn something different. Going

macro to this, like most of the time we

don't get to choose what we test cuz

we'll be working in um environments, we

working in companies, there'll be a

manager, the manager will choose what we

work on. But the thing is we um

influence that because we have skills

and experience and some of our skills

and experience and domain knowledge will

not translate into some applications. So

we will not be suitable people to test

that without additional training or

without um offering a risk essentially

to the project cuz we may not know what

we're doing. Um if I have to test a

Hadoop app um I really need to work on a

team that really understands Hadoop

because I will be learning that um from

scratch. So I've done consultancy on

Hadoop teams and but I was relying on

their um massive amounts of technical

knowledge, me reading around the subject

and conducting experiments from me

asking them questions when they didn't

know the answer so we could deep dive

into it, me applying my technical domain

testing domain knowledge onto their

project and mixing it that way. So we

have to combine the right people at the

right time to have the right team mix.

This can be hard when you're in

management. Any of you that are in

management will know that this can be

hard because um someone that you need on

a project may not be available because

they're on another project and you're

trying to juggle things around and

projects get delayed and put people off.

So that means we have to have some slack

in our teams or we as managers have to

have the ability to support people or

know that they'll be supported on the

team and sometimes we allocate people to

teams because of their um soft skills,

their personality and how they will fit

into that team. But it is a key lesson

that we have to choose the right people

for the right technology for the right

job. Otherwise there's a risk otherwise

there's a process that means we have to

manage it to make sure that they

understand what they were doing. So that

is a tiny tiny lesson but it's massive.

I mean it's it's essentially what is

test management choosing the right

person for the job. What is our career

management? making sure we have the

skills we need for the job. This is

absolutely key to what we do. But a

micro level, it's choosing a particular

application to test.

So environments is a big concept and in

terms of this model it's planning and

the environment install and setup and

getting everything working. Now for

tracks I could use Docker. There's a lot

of Docker images out there for tracks.

It's very easy to install a Docker image

once you know what you're doing and

experiment with them to see which one

works. Um I could do a local install.

Tracks is a Ruby on Rails application. I

think it's a bit hard to install

locally, so I try to avoid that. But if

someone was familiar with Ruby on Rails,

had the environment set up, a local

install might be best for them. Um, I

tend to use a virtual machine, which is

pre-built, may not be the most

up-to-date version, which could be a

risk. And the same virtual machine can

also be cloud hosted. So, I have a lot

of options in the environment that I

choose. Now, from a testing perspective,

I have requirements from that

environment, right? to support my

testing. I'm pretty sure I'm going to

want to proxy traffic. So, it needs to

be somewhere that I can proxy traffic.

So, it doesn't really matter what

environment that is. I should be able to

do that. But some corporate environments

make that really hard that make that um

difficult. I may want to um view the

database which would limit me so that I

don't use Docker, right? because I don't

know Docker well enough to know how to

configure a Docker image or running

Docker container to allow me to access

the database. I don't know how to do

that. I'd have to do research. That

might take more time and I'm trying to

not necessarily learn how Docker and

other environments work as this

exercise. I'm trying to learn how my

testing works. So, I didn't want to

impose those extra constraints. I wanted

a low impact on my machine, which ruled

out a a local install. And the knowledge

that we have will limit the

effectiveness that we have when we're

working with an environment. That's why

we have to work in teams. That's why

it's really important that we don't just

get an environment given to us that we

don't have the permissions to it. Right?

The environment really, really impacts

testing. The amount of times I've gone

on site and I've worked on projects

where we do not have permissions to

access the environments. We don't have

permissions to SSH into the box to see

the logs. That means we can't track the

memory usage. There's a whole bunch of

things we can't do if we don't have

permissions. Sometimes we have too many

not necessarily permissions but

responsibility. So we're responsible for

um releasing the application in the

live. It's the same as if I had to

install tracks on my local machine. That

would be an issue. I may not know how to

do that and I may not have support if I

do it myself. Hopefully on teams we have

support but very often we don't. We're

left to get on with it. It can take

months sometimes to set up environments

for certain test teams. I've seen

projects where they have literally spent

months setting it up to get it in the

right state and they can't change it

once it's there because they don't know

how to do backups. It's a completely

different install process than live,

which itself is essentially a risk for

the project because if you're deploying

something differently in the different

environments, how do you know that your

deployment process hasn't changed

things? We really want the same

deployment process as live. We really

want the same version management. We

really want the same level of

observation, tracking, monitoring on

these testing environments. We want the

same process for doing upgrades,

downgrades, making sure the database is

okay, making sure the same version of

the database software is being used on

the different environments. Environments

are a massive thing. I mean, one of the

interesting things with Docker is it

makes environment provision a lot easier

and once we understand it, then we can

do a lot with it. But it's only really a

particularly good thing if we're also

using Docker in live. If we suddenly use

a different process in live, we have to

do some sort of process to mitigate the

risk that the live environment might

operate differently from the docker

environments that we've been using and

we need to try and as a project mitigate

that. One of the ways we can do that is

by repeating the automated execution on

our live environments, but then we have

to tear down data. We might have a

subset but there's a that notion that

environments impact testing and when we

are not looking at those properly

there's a risk I mean this is a constant

source of risk when I'm looking at

projects when I'm doing consultancy the

environments that people use is a really

really important and people

underestimate it so that was an

important lesson to come across early in

my particular process

and what's interesting for me is the

very first session that I have to do is

an install session right if if I was on

a project there'd be lots of discussion

etc etc but for me I'm not working on a

project but I can't do anything until I

have an environment until I have an an

application installed that I can work.

So for me, the first session is an

install session. And for tracks, there's

multiple ways that I can install track.

So I could go off to the tracks website,

download it, and install it and run it

locally. To do that, I need to have so I

think it's Ruby on Rails. I need to set

up an environment to do that. And given

the time scales that I've got, I don't

really want to do that. Given the risk

that that might pose to my main machine,

I don't really want to do that. So I

want to try and isolate it somewhat.

There are Docker uh instances that I can

use to install this. There's at least

three that I know of. If I go off to

Docker. So there's a bunch of tracks

installed on um Docker. If I do tracks,

um there's a few in here. The one that I

mainly use is the Stano one, but I'm not

going to use the Docker install for this

because my Docker knowledge isn't

particularly good. So the technology,

the environment would impact my testing

and I want to give myself as many

opportunities as possible for um

exploring the thought processes behind

testing. So I want to make my

environment as flexible as humanly

possible which is why I'm going to use a

virtual machine. I'm going to download

the tracks virtual machine from Turkey

Linux. Turnkey Linux, not Turkey. It's

not Well, it is Thanksgiving at the

moment, but it's not Turkey Linux, it's

turnkey Linux. So, I'm just going to

install a VM and run that using uh

Oracle VM Virtual Box that's open

source, that's free, allows other people

to repeat this if they want. And then

TrackX runs quite happily in a VM and I

can use it. The benefit of that is um

it's just another machine. I can SSH

into it. I can access the database

there. I can proxy requests to it quite

easily without any issues. There's a

specific IP address. Everything becomes

a little easier. It's basically I've got

tracks installed on another machine in

my environment. I'm connecting it to it

there and I can give myself forever

permissions I want.

This comes with some constraints because

this version of tracks is not the most

up to date. For the purposes of my

exercise, I don't mind that. Right. So

the first session I have is an install

session to download it, set it up, go

through the process. I've gone through

the process. I've set up an admin user

on here. Just going through all the

prompts and now tracks is running. If I

go back to the main I do not want to

quit. If I go back to the main virtual

machine configuration, this is what it

looks like when in the running box. And

if I look at the networking tab, then I

can see the IP address it's running on

because every time I start up the

virtual box, it could be on a different

IP address. Currently, it's on

192.168136.

It's captured. There we go. So, if I go

off to 1 192.168.1.36,

I'll see tracks and I'm logged in and

this is the application I'm going to

test. So, the very first session is

simply an install session. Took me half

an hour to get it installed. I'm using

the virtual machine. I'm running in

virtual box. And the basic lesson

learned from this is the environments

that we choose impact our testing,

right? Cuz I need to understand the

technology that this is running with. If

I don't, I'm I'm going to be limited to

the extent that I understand the

environment, to the extent that I can

access the stack. If I don't understand

the environment, if this is running on

some really odd new technology, I have

no idea what it is, my testing is going

to be limited to the front end. I'm

going to be limiting my testing because

of the environment, because of my access

to the environment. Sometimes we have

issues that we don't have access to

environments. We don't have permissions

and I essentially need permissions to

observe uh so I can observe traffic

without anyone's permission. But I

probably want to observe logs. I may

want to observe memory overhead on the

server. I can only do that if I have the

permissions. I may want to interrogate

what is happening in those logs longer

term. I can only do that if I have

permissions. I may want to go into the

database. I can only do that if I have

permissions. So the environment is

really important. It's really key to

testing our understanding of the

environment and the permissions that we

have on that environment.

So we kind of mentioned this in the

environment section releases and

deployments. We really want the same

process as live and we want to share the

knowledge. Like too many times I see um

projects where the programmers are doing

all the environment management which

means the test team never learn which

means that they never have the ability

to go deeper into the uh logs or the

environment to do testing. So they're

limited by the knowledge and it's a very

simple thing to pair on projects to

share the knowledge but a lot of people

don't do that for some reason. But for

me I had to um install the application.

So I this is where you'll start to see

me making notes. This is the first of my

notes in the testing and these are my

actual notes. I'll try and show you as

much as possible the actual notes that I

made but you'll also see those in the

debrief sections. So you can see exactly

when I started. I tend to do dates uh

completely backwards because then I can

if I need to I can sort files based on

the date. So I get used to writing the

dates backwards. I think it's readable

like that. just I don't even have to

think when I'm doing it. So I I've made

a note of where I downloaded the tracks

VM from. I've just installed it, which

is simple. I've made a note of the IP

address that it started on and the admin

password that I've got. And you can see

I've got timestamps for when I started

and when I stopped. Didn't make a lot of

notes in this because I know how to do

this, right? So the notes are there to

support me doing debriefs. They're there

to help other people. I think this is

fairly obvious what I did when I was

working through it. Had anything gone

wrong, then I would probably have made

more notes. Had I had to do anything

different that was not the way that I

normally configure virtual machines, I

would have made a note because then that

would remind me later on. But this is

basically telling me nothing untoward

happened and it took me half an hour to

set up tracks. So then the question is

okay so I've got my environment and I've

installed it as a VM I can log in as

admin but is the actual tracks ready

right now I don't know that right I can

get tracks up I can go to it I can kind

of click around I can maybe do I don't

really know whether I've installed it

properly this is a big issue on a lot of

projects so I know that I'm going to

need to do a kind of health check

session to do this and I've I've tried

to be really careful careful when I do

the sessions here so that I don't bleed

sessions into each other to try and

chunk as much as possible what is the

overriding concept I'm doing at this

point in time. So I'm trying to install

the application. I could bleed it into

going okay has it installed correctly

and is it ready to test. But really what

I did is I got it up and running. I

logged in as an admin and then stopped

because I had installed. I wanted to be

very clear my aims because it's too easy

to let test sessions bleed into each

other. Now with this particular system

there is no dashboard in tracks that

tells me it's working that all the

services is working that the database is

up and running. I've got no way of

seeing the logs from within tracks. I

can't click on a link and it opens the

logs up. I can't go to a single

dashboard where there's a whole bunch of

green lights. So it's not easy for me to

ping I mean I could ping this this and

see that I get response. I could curl,

make sure the admin page comes up and do

and put that into a GP and do some

checks. I could do some very basic

DevOps type scripting checks in there

and sometimes those might be all you

need for a particular setup. And I've

seen companies where they deploy the

application and all they do is ping it

to make sure it's working. I don't think

that's enough, but um that's what

sometimes people do. I think it's really

important to build dashboards or health

checks into our application to make sure

it's working and comes back with status

codes. Even in some of the REST APIs I

build for support and testing, there's

usually a um am I running which responds

with a yes with a message. So, it's not

just a ping. I'm actually calling the

API to make sure it can trigger

something on the back end to make sure

the application is started up. because

sometimes you can ping something, it

seems like it's there uh because the

port is taken up, but it doesn't

actually respond with the correct

information because it may not be

running. And if you've got a micros

service type architecture with multiple

services, you really want to make sure

they're all working. So this is where

automated execution really can help,

right? because we can do automated

execution to go through the GUI make

sure things are as expected in the right

places to make API calls to make sure

we're getting the information back and

we might go through a simple CRUD check

where we create an entity read it do an

update then delete it to make sure the

application state is not changed with

our health checking creating dashboards

that self-report can be useful because

what we want to avoid is the issue that

we start testing and we do days, weeks,

potentially months of testing, which

used to happen, and we discover that

there's something wrong in the

environment, that the wrong version has

been deployed, that one micros service

that is relying on isn't quite the same

as everything else. So, and if that

invalidates testing, we waste a lot of

time. So, we're trying to de-risk the

notion that we might do stuff and find

out too late, and we really want to

avoid wasting time. So I need to do a

health check session in order to check

whether that's working.

So I split the environment um setup and

install from the health check to make it

really discreet. So I I created a health

check session. Now, this can be useful

um in multiple different places, like

just a quick session to double check

that things are as we want before we

dive in. We can do that with are we

working with the correct version? Are we

doing this? Sometimes it's only 5

minutes. I do like sessions don't have

to be an hour long, half an hour long.

Create a name. Do your testing in

chunks. We call that a session. um and

stick to as much as possible what you're

trying to do in there. And that's what

the health check session that I did was.

So, my health check is basically just

going to be okay, I'm going to do a

create, read, update, delete in the main

areas, keep it simple. I'll try and

create a to-do uh read it, check that

it's there, update it, delete it, make

sure I can go through the same the the

basic process. And I I created notes as

I did that. So, you can look in the PDF

and it's not a lot of notes. It's not a

big session. What was that? 2:00 we

started and then went through. Yeah. So,

I started at 2:00, went through until 20

passed. And you can see I stuck fairly

well to my plan. So, I did a create,

read, update, delete. But also, I made

notes as I was going through. It's it's

almost impossible not to observe as

you're doing any kind of activity. So,

I'm observing as I go through. What what

happens when I'm doing uh a very simple

high level session is I don't tend to do

a lot of interrogation. I don't tend to

do a lot of manipulation. So what I'll

do is I'll do observation and note down

anything that I may want to investigate

in the future. Try to avoid going deeper

into that and doing an interrogation or

manipulation on it to really investigate

it because I'll create another session

for that. Um and I didn't really do that

I think in this particular session. So

there's in this particular set of tests.

So there's no there's no session type in

there for defect investigation or an

investigative exploratory session,

right? Because that didn't really come

up in the time scales, which is why I'm

saying that the the model you see at the

start of each session is not um

complete, but it's a very very simple

way. And these are the actual notes I

made, right? So these these are

relatively detailed. They say what I'm

going to do. they say um roughly what um

happened and I'll make more notes if I

really need to be more clear. Sometimes

I'm very formal and I'll have um like

question in uppercase. Sometimes it will

be defect. Sometimes I write note.

Sometimes I don't really care about the

language. So I don't really care about

consistency. But over time I tend to

just write note, question, issue,

defect, investigate, to do, right? So,

and they stand out. So, I'm putting them

in uppercase. Now, I made my notes in

Evernote. Um, which sometimes is a good

thing, sometimes it isn't. The reason I

did that is because it allows me to

embed screenshots easily when I'm doing

um screen capture. But the notes that

you'll see are markdown. Right? So, the

PDF that I've released, I converted this

into markdown because this is mostly

markdown. Right? What you can see there

is that some of the outlines are not

marked out, which meant when I copy and

pasted them into the text editor, I had

to reindent them to get that working.

But I tend to write in text. I tend to

write very textual notes that are

linear. They go from start to end. And

I'll rely on debriefs and other admin

sessions to tidy them up and format

them. So this is the kind of thing that

I observed as I was working, right? I

found the delete quite hard to click on

depending on how you move to it. Um, so

that would be something that I would

probably raise in an admin session

afterwards, a communication session. I

also found that when I was dragging

stuff, there were a lot of rendering

type issues. The um Z ordering wasn't

quite correct at different points

because I'm moving the mouse, the the

drop down appears. So again, these are

things that I would probably raise in

discussion. It doesn't matter that much.

But the point for this was as I was

doing this, I realized very quickly that

I needed to have a screenshot tool uh to

hand when I'm doing my testing because

there's a lot of visual things that I

would see in passing and I needed to try

and capture them as I went through going

macro. Everything that we do, every

session that we do, every questioning,

every grooming of stories, these are all

exploratory sessions. They're all

observation. We observe the people that

we're working with. Do they really

understand what they're talking about or

are they trying to bluff us in a a

story? This really important when

working with product owners. Do they

really understand what the customer

means or are they bluffing because they

just want to get ahead and then they'll

rely on testers or developers later on

asking more questions that will flush

things out. The whole notion of story

grooming is an exploratory process. We

have to ask questions as we go through.

Now the exploration that we do when

we're working boosts our technical

knowledge of the app because we see

things happening and we go how is it

doing that? Is that CSS? Is that

JavaScript? I don't know. But it's also

constrained by our technical knowledge.

This is why we want the right people

testing the right app. This is why we

want to work on our skills as we're

testing to make sure we understand the

technologies that we're working with.

For instance, with tracks, I have no

idea whether this is using a JavaScript

framework or not. I don't think it is. I

think it's all custom handwritten

JavaScript, but if I was working on the

team, I would know that. If I did more

investigation in the tracks, I would

know that. And if I knew it was a

JavaScript framework, then there'd be

risks associated with that. And there'd

also be things I wouldn't test as much

because they're part of the framework.

If I know it's completely custom, then

I'm going to test it in more detail

because there's more risk. Now, in terms

of time box testing, this is an

interesting thing because some people

like force sessions to be an hour or

half an hour. With some sessions, it's

really important to time box them. If

you have an exploratory session and you

have a a really relatively large charter

that you're going to work through, a

relatively large set of aims, then you

want to time box it so you don't spend

too long. Some of my sessions here are

very small. I'm quite happy to create

small sessions, small tasks and

activities which may not be time boxed.

They may just be tracked. So here you

can see that I'm tracking the time. I'm

not necessarily time boxing them. So

don't feel that you have to pursue a a

sessionbased

time management approach all the time.

What you have to do is take ownership of

your testing and do what is right for

your testing. Splitting it up into small

chunks can be really useful. And I would

previously on some projects create

tasks, put them in Jira to make things

visible, do my testing in very small

chunks, take my testing notes, copy them

into that task again to make it visible,

do a debrief session, then summarize it,

then put those notes on the higher level

story rather than the tasks, my summary

notes. But it varies depending on

project to project, tool to tool, and

what communication we want to do. But it

was obvious to me that I needed a

screenshot tool. I tend to use Snagit.

Um, and I have a tool called Cloud App.

All these animated GIFs you're seeing

here, I created using Cloud App. Um, but

most operating systems have built-in

tools. I'm just so used to using Snagit

and other tools that um I I just

installed those. You don't have to pay

for them. Snaget isn't expensive, but

use whatever is appropriate. But you do

need to have the tools. But I didn't

start with a screenshot tool ready. I

mean, it's always running, but I didn't

start thinking, right, I need to make

sure my screenshot tool is there. But it

became apparent as I was exploring very

early on that I'm going to need certain

types of tools to help me. Now, my note

takingaking is I need to write down what

my aim is. So, in the notes that you

saw, you didn't see an aim section. You

didn't see me write down I'm going to do

a CRUD section, but that's because I was

kind of making notes in the slides and I

put my CRUD health check in there as a

slide before I went and did the testing.

So, I did have those notes in there.

It's just they weren't in my testing

notes. They were in my kind of highlevel

plan, which was the the slides. So, we

do make notes prior to doing things and

we're writing down our aims or our

charters or whatever else we want to

call it. I tend to I tend to think in

terms of aims or objectives. I tend to

communicate in terms of charters and

sessions because that's how people talk.

But in my head, it's much more basic.

It's much more time managementoriented.

And it's what are my aims? What are my

objectives? What do I want to achieve?

What do I want to investigate? What am I

what am I trying to do? And that's my

plan. And what's more important is

making notes during the testing. Right?

This is what differentiates us doing

exploratory testing from scripted

testing. Scripted testing is making a

really detailed plan, right? And the

plan goes down to the stepbystep level.

What you will do, what you will change,

which means that when we actually do our

testing, we end up having to maintain

our plan because our plan is wrong. So

we need to get the script right as we're

testing. With exploratory processes, we

come up with high level aims and

objectives. we go down to the level of

detail that we need to in order to be

clear about what we're doing and to

communicate it and to help people

understand what's going on to

prioritize. But it's really important

that we make more notes during the

actual testing. Now, I think it's really

important to make that those notes when

we're doing scripted testing as well,

but very often the tooling that is in

use doesn't support that. It has like an

actual column where you just type in

what happened, what actually happened.

And very often people just put in a tick

or they go as expected and they don't

write down the actual notes. They don't

write down the data they did. They don't

write down the adjustments. So but with

exploratory testing it is key that we

gather evidence as we are testing and we

make a lot of notes during the testing

that I mean I can't stress that enough.

This is why I spend a lot of time

improving my note takingaking and doing

exercises like this where I try and

improve what I'm doing. Make notes that

are actually reflective of what you're

doing and can stand up as evidence and

can support you reviewing it later on.

Okay, so the next thing I have to do,

I've done my install session. The

application is running. Does that mean

that I'm actually ready to start testing

now? And the answer is, well, no, not

really. Because I don't really know

whether the application has properly

installed. I don't know if it's set up.

One of the issues we have when we're

testing applications is did the install

work? One of the risks we have when we

do testing is if I start testing now and

days from now discover that there was an

issue with the install of this

application in the environment. Does

does that invalidate my testing? I might

be investigating problems that are not

there. So we have to have some mechanism

for telling us whether even though we've

installed it, the application is good

enough. Now hopefully we're using the

same release processes um into the test

environment that we're using in live. Um

if we in our software processes have

different release processes throughout

the uh life cycle, we're we're creating

a problem for ourselves. We're creating

a risk that may bite us later on. In

this instance, I could simply just trust

that the virtual machine has been set up

correctly and is working correctly, but

I don't really want to do that.

So what I have to do is I have to do a

kind of health check to make sure that

works. A health check session. And if

this was a real world system, I would be

expecting some form of automated

execution to help with that because that

can really derisk deployment and and

start up the environment. We automate

the deployment. We automate the creation

of the environment. We automate the

startup. We run some uh checks against

that environment to make sure it's

running. We ping the machine. We have

something that comes in and goes to the

homepage. We have something that logs

in. Make sure we can log in. Maybe

double checks the database. Make sure

all the tables are there. Does some

check sums on whatever we want. Make

sure it's in the right folders. Make

sure that there's log files being

written. It has access to the write the

log files. All those kind of health

checks. It's great when an application

itself has its own dashboard and

everything goes green internally. the

old arcade machines when they were

switched on they would go through a a

selfch check validation process that

really really helps when uh you're

working with systems because you can

automate the does the uh dashboard the

internal dashboard show all green is

that status working so the more that we

can have the application selfch check

the better otherwise we have to come in

and check it ourselves now I don't have

any automated checks on this so I'm

going to have to do a health check

session

Now, I've done my health check session.

So, I'm just going to quickly show you

the notes and then I'm going to

partially repeat this so you see what's

going on. What I didn't want to do in

this course is show the exact testing

process cuz that can be a little bit

boring. Now, these are the actual notes

I made when I did my health check

session. So, you can see that I ran

through a basic CRUD process, right?

That's why I really wanted to make sure

that the application uh could create

entities. I could read them. I could

update them. I could delete them. If I

could do those basic things, I was

fairly happy that it was going to work.

So, I used the front page, which you can

see here, to create some actions.

And I almost couldn't help myself

because I don't know this application.

I'm learning at the same time. So, it

wasn't completely a health check. I had

to make up what my health check was. I

have no idea what a health check for

this application is. So I actually made

it up as I was going along. So I tried

I'll create an action. So I'll just add

an action. Then I can see that the

description can't be blank. But I saw

that the form didn't uh refresh the

page. So there must be some XHR

messages, some XHTTP messages going on

that I can't see, which hints to me that

I'm going to need late more tooling

later on to observe the messages. Um, if

I I don't know what the the rules are

for this. So, I'm going to add something

with a description. I'll add a. And it's

it's added it. So, I now have something

with a an A. Uh, and that worked this

time because there was a context. By

default, it's putting in one of the

contexts,

which is there. So, if I do it with just

the description, then I'm getting told

the context can't be blank. So, I'm

learning the validation rules during

this health check cuz I don't know how

this application works. Um, I can read

the data by jumping between the um tabs

and seeing that data appears in those

different tabs. So, I'm doing that.

That's fine. Um, I can amend these

things. So, I created a how can I amend

this? I can amend this by clicking here

and amending it in situ. I've got the

ability to create notes. Let's change

that. And it amends there. And again,

actions saved here. So, there's XHR

messages being sent that I'm not

covering in my testing at this point,

but I need to at some point in the

future.

How do I delete these?

So, there's a drop down here, and I can

delete or defer them. Now, I'm looking

at this, and I'm seeing risk, right?

This is clearly

uh there's some JavaScript in here. This

might be CSS styling on hover. I suspect

it's not, but I don't know. So there's

um something I have to investigate

there. There's potential risk in there.

Um but I can delete. Let me just delete

this. Um and that's interesting. So

we've got an alert. So we've got

JavaScript in there. This is still here.

It hasn't gone away, which makes me

think it's JavaScript rather than CSS.

Okay. Then it's gone. XHR message. What

if the messages are not getting through?

There's test ideas coming out of my head

as I'm testing even though I'm doing the

the health check.

Now,

what I see that's interesting here is uh

this moving thing. So, I can move these

around and and stuff happens, which is

interesting. But also, look at that.

When I move this, and I don't know if

that's cuz the mouse is getting close to

the drop down or what is happening, but

I'm pretty sure that shouldn't re

there's no point to that. There's no

benefit to me as a user that that

happens. So, I'm assuming that that's

actually not a a necessarily a good

thing. That would be something that I

would probably want to um report as a

should this really be doing this

possibly an issue. So, even when I'm

just doing my health check, what happens

if I just drop that? It goes back. So,

um even when I'm just doing my health

check, I'm I'm exploring and learning

about the application. And that might be

important because it may have changed

slightly from the last time that I did

this. So the my initial session took 20

minutes. I've summarized it there for

you in whatever that was 10 minutes. But

the first session, the first real

session was a health check session and

it was based around the the the CRUD

concept, the create, read, update,

delete. Can I do the basic um steps that

I need to and the lessons are all

testing essentially is is exploration

and observation. And I was doing that

through the health check. um the

exploration is boosted by the technical

knowledge of the app, right? That I

identified things I need to investigate

there because I'm seeing stuff happen uh

that I believe is related to JavaScript.

I'm seeing stuff that might be CSS

related. So, I need to learn the

technology to help me go through. This

was time boxed. Um I originally gave

myself half an hour to do it. It took 20

minutes. I made notes of the times as I

was going through to track it. I made

exploratory testing notes as I was going

through this process. I have my notes

that I can refer back to and they're

they're pretty detailed. And the reason

that I'm going back and redoing the

recording of the sessions is to check

how good my notes are. This is

essentially a review process for my

notes. I can see that there's I'm

alerting myself to things that I did not

do, right? I didn't check in the

database. I didn't hell check the API.

All the validation that I'm doing, all

the observation of did this work is

being done on the GUI itself. And I've

made notes on defect. Now, one of the

things I I noticed in there is that

sometimes this can be a little bit hard

to um click if you go the wrong way,

right? It hovers up. I come over here,

it goes away. You have to be pretty

quick to come down to it. So again,

that's a kind of usability type issue

that I've mentioned in my notes. Now, my

plan for this was essentially just a

crud. Make sure this thing works in the

health check. But what was important was

that as I did the testing, I made more

notes than I did prior. So exploratory

testing is really trying to emphasize

the gathering of evidence rather than

the upfront planning. And it we track

our notes to help keep us on track. It'd

be very easy for me to go off track

during the session and follow all these

things I don't know and understand. But

the concept I've got in my head is

create, read, amend, delete or create,

read, update, delete. And until I've

done that, I don't really want to go off

track too far. And the time boxing can

also help. The recording that I've made

here is also evidence. I didn't take

screenshots as I went through originally

cuz I knew I was coming back. But some

of the later sessions, I've been taking

screenshots as I go through to gather

evidence as they come through because I

didn't start with all the tooling

available. I didn't know there was going

to be so much on screen that I was going

to try and capture as I did my

exploration. So, having done the health

check session, I know I need to have a

screenshot tool very handy to support me

because there's stuff happening in here

that I may find hard to recreate or I

may forget about or I may want to

capture the nuances at the time that I

see it. So, I know I'm identifying some

of the tooling that I need as I go

through very quickly.

But one of the key essences is to gather

as much evidence as possible. So I think

that's probably all I want to say about

that particular session. So that's a

health check session to make sure that

the system is actually suitable for

testing. And again, automated execution

might help with that, but we don't have

any here. So I'm doing this purely

exploratory.

So this may be one of the most common

questions that I'm asked. How do you

know what to test? How do you choose

what to test? And there's so many

different ways. So this particular

section is split into I think three

lectures. So this is just a short intro

into choosing what to test. Um but how

do I I know what to test? Well, I I do

kind of planning sessions. I build

models. I investigate it. But I rely on

everything that is around me in the

team. So if we have requirements, I will

use those. I will use those as a basis.

Now, one of the things that's

interesting in kind of ISTQB

terminology. I think they got one thing

right where they talk about test basis,

which I don't think we used to talk

about um not in in those terms. I used

to talk about test conditions, not

necessarily test basis, but I think

basis is a a useful word. So if we have

requirements, I'll use those. If we have

stories, I'll use those. Um, if we have

defects, change requests, commits, but

I'll also use the models that I've built

up over time as I am testing. But since

at this point, I haven't done any

testing, right? I'm focused on um the

requirements, stories, and information

that we have to hand. and the processes

we go through in terms of discussing

stories is really important to identify

what it is we're actually going to test.

So for this particular application

I'm going to test holistically. So I'm

going to test the system rather than the

story. um which is basically analogist

to an agile process where we've tested

all the stories, we've tested all the

tasks independently and at some point

all of those stories exist in the

application at the same time and we want

to look for do these impact each other

and this is very often something that is

missed out on a lot of projects because

we work on story story basis but don't

look at okay holistically how do these

fit into the system what is the user now

going to be able to do across these

flows. Um, so very often just by the

nature of the design process used on

agile projects, we miss out a whole type

of approaching testing holistically. And

I'm using the phrase holistically simply

because system testing is vague,

ambiguous. It could be functional

testing, could be flow testing.

Holistically means I'm going to look at

the entire thing. I could use other

words, but that's just how I tend to

communicate that these days. Um, I'm

going to assume that other testing has

been performed, right? This is an open

source application. There is release

notes. There may be automated tests. I

don't know. I'm going to assume that.

So, in order for me to figure out what

I'm going to test now, I've got my

overriding approach, which is testing

holistically, but I need to plan what

I'm going to do. I need to figure that

out. So that's what the next session is.

How do I do the planning?

So how do we know what to test? We plan.

And for some people, planning is a dirty

word because they're on agile. The

notion that you plan things. We do

exploratory testing. We learn as we go.

We don't plan. Everyone plans. Planning

is key to how we work. It's what how

it's how we focus. So planning is an

important stage. And you can see in this

model here, pretty much before I do

anything, I'm planning what I'm going to

do. Even if that's as simple as what is

my next action? What is the next

objective I want to achieve? What is the

next question I want to ask? That's

planning. And that is a key step, right?

So

when I'm planning

um I treat that as a session, right? So

very often what happens is when we do

testing a lot of the work that we do

doesn't make its way into the tracking

system. It's not visible to people. They

don't know what we're doing. So the only

thing they see are um huge gaps of time,

then an hour where we did some testing,

then defects magically appear and

testers running about asking questions.

Right? It's really important that if you

can

chunk your work as much as possible into

sessions. I mean, I do this in my daily

notes regardless of whether um I'm

putting it into a tracking system or

not. So, I will have a in my daily notes

a time stamp what I'm trying to achieve

at this point in time. At the top of my

daily notes, there'll be a list that

says here's what I'm trying to achieve

today. And I'll pull the objectives out

of there, then play them, put a time

stamp in, start making notes as I'm

going through, record the time, record

interruptions. Um, and that it's my

entire life is chunked into sessions,

but that's cuz I've been looking at

different time management approaches for

years. And the the most common time

management approach is um track your

highle objectives. Don't spend too much

time on um doing that because you'll

spend more time planning than actually

doing. So plan the highle objectives.

Then decide to play one of the highle

objectives. Plan in enough detail so you

know what you're doing so that you know

that you're still on track. So you know

that it's still important. Then track

what you actually do. Make progress.

When you finish, debrief, see if there's

any new information that comes in. Will

that impact your plan? have you learned

anything? And that what I just described

there is a time management approach.

It's um the essent essence of agile

story planning, right? All of these

things are the same. It's sessionbased

uh test management. Um all of this I

keep thinking session based time

management because really um James and

John took a time management process and

mapped on the testing because that's

what they were doing. they were managing

the time they were spending on testing.

So that's how it translates. Now the

planning process itself

is an exploratory process, right? We're

investigating what we want to achieve,

what are the aims, what are the

questions, um what are the factors

involved, what are the um risks and it's

a decision making process. We can track

it and plan it like any other activity.

If we know that we're going to have to

do a whole bunch of testing, we can plan

in a planning session before that, which

might only be 20 minutes, and to think

through what we're going to do before we

actually do it. It is planning can be

planned. One of the issues that I think

people have with planning, they get

nervous that they've got to plan it in

real detail, but we don't. We go as

deeply into it as we currently

understand based on the models that we

have in our heads. Sometimes

the planning process will let us know

that we don't really understand what

we're about to do. Therefore, we know

that it's going to be hard to estimate

cuz we're working with unknowns. We know

that we can't plan it in too much detail

because it's liable to change. We know

that we're working variable information.

Therefore, we don't want to write

everything down to the local steps. But

if we don't plan, then we may not have

the resources that we need to hand. We

may not realize that we need access to

the database before we do the testing.

So we may start hit a blocker stop

because we didn't think it through

effectively, which is really what

planning is. But sometimes people don't

plan because they're agile or because

they're exploratory or because whatever

their misunderstanding of a particular

word is. But if we don't plan, then

we're losing the opportunity to

effectively target our testing, to think

about it, to identify our aims and

objectives, to really keep on point and

test as effectively as humanly possible.

So the results of my planning session,

and you can see this in the the PDF, and

it fits onto one slide, which is nice,

and it only took me 10 minutes, right?

So 10 minutes for me in the planning

session was what am I going to use to

test? So I know that there's release

notes and should I look at the commits

or the defects and should I just think

about the general functionality and I've

written down the things that I'm

thinking about. So I made the decision

to go holistic which I already

mentioned. I had a look in the release

notes and saw that there was a statement

you can now change the state of a

context to closed and I thought okay

there's a lot in there. There's a lot of

ambiguity in there. What what is a

context? What's a context state? What

states are there? It doesn't mention

that in there. How many are there? Is it

a natural state machine? Are all the

states valid? Are there transition

rules? There's a lot of ambiguity in

there to investigate. And it may be

clear if someone knows the system well,

but I don't. So, I can create some

sessions for investigating this, for

looking in this. So, I've made a

decision in 10 minutes. I've focused it

in. I've rejected certain approaches for

doing it. So, I know what I'm not

covering. Like the planning session is

also about identifying what you're not

going to do, what the potential risks of

not doing that are. And I track it. I

write it down because it's useful to

revisit and go, "Oh yeah, we kind of

thought about that but discounted it for

these reasons." So that's the result of

my planning session.

So one of the things I have to do is

decide what on earth I'm going to test.

And we have to do this all the time as

we go through a project. So very often

we'll have requirements. They're often

uh driven by stories on an agile project

or what the person wants to achieve. And

we might look at when we've got a story

how this thing has been implemented,

right? in in order to understand if

there's any technical risk or functional

descriptions or take into account the

unit testing that's been there. Um we

have to very often when working on agile

and this is really important is that we

have to look at the decisions and

constraints that are made during

development because sometimes stories

span multiple sprints. So we've we're

constrained by the fact that the full

story is not there. The validation may

not be fully implemented. the

integration with other parts of the

system may not be fully there. So the

testing that we do in sprint has to

adjust to that and may not really be of

as much value as we hope. Um and it can

make testing look like it takes a long

time because we'll be testing the story

at each point through the sprint. But

when we get to the very last sprint when

the full story comes together, people

are expecting it to take less time

because it's a tiny little change. But

now the full story is there. But we look

at it and go well now the full story is

there. So now I have to take in account

all the the things that we have built in

all the validation rules. How does this

integrate with the the the system and go

beyond the story acceptance criteria. So

this can really impact our testing. I

tend to think of it in terms of we've

got the inprint work, the story work and

then system testing. And by system

testing I mean the system as a whole

like stories in combination functional

flows throughout the system that span

multiple stories going beyond the

acceptance criteria that's listed in the

story. And this is the part that's often

neglected on agile projects and it's how

things slip through because very often

we're so focused on the acceptance

criteria and the um automated assertion

of the acceptance criteria. But for me

and because I have to decide what I'm

doing and I'm not working on the project

here on an agile or sprint basis, I'm

almost working on the go beyond the

story basis, right? Because I have to

figure out well how can I do something

as an exercise here that can add value.

So I assume that the team working on

tracks have tested it. They've done unit

testing. they've checked out their

stories. What I do then is go right

where can I add value in that process

and it's by looking at the project

holistically looking at going beyond

just the functionality in isolation. So

where can I find out what is important

to test? So I've got the release notes,

right? This is the most obvious place to

look for um items to test. I could go

through the source code. I could look at

all the um changes that have made, the

check-ins that have been done. I could

look at all the bug fixes and retest

those. But I'm looking at this in terms

of how can I add value. So I have to go

beyond what I believe the team has

already done. So I'm going to pick one

of these uh items which is you can now

change the state of a context to closed.

I'm just going to pick that one. I could

pick any of these. I could pick

something completely random. But this to

me it is interesting, right? Because

we're talking about state. So as soon as

we have state, I'm thinking of a state

machine. I'm wondering if there's

constraints. I don't know um all the

states that it can be in. I don't know

what else impacts the context state. I

don't know what other entities are

related to that because very often with

state machines, state machines are

analogist to entity life history

diagrams from the olden days. An entity

life history diagram is kind of a

hierarchy of states that the entity can

go through. Entities um relate to each

other. So entities in different states

may relate in different ways. So I have

to look at just not just not just the

state of a context but what else the

context relates to to see if the state

um impacts that. So I think that's even

though that's a very subtle and small

thing, I think there's a rich vein in

there to tap and to to explore um in

short short bursts because that's what

this of course is short bursts of

testing. So I'm going to focus in on

that and then I'm starting to think well

what is my uh what are my questions

around that? What I mean I don't know

what context is. It's been so long since

I used tracks I've forgotten. I don't

know what a context is. I don't know

what the context state is. I don't know

how many states there are. I don't know

if it's a state machine. I don't know if

all states are valid as each other. I

don't know if there's transition rules

or constraints between. So, I'm going to

have to find that out. But this video is

a representation of a um planning

session, a decision session, right?

Because we have to spend time planning

our work and making decisions about what

we're going to do next. And we can track

that as sessions. They're perfectly

valid things to track as sessions

because that is an exploratory process.

Thinking through what I'm going to test

is an exploratory process because I

considered multiple factors. Um I

considered whether I was going to read

the manual, look at the the code. Um I

thought this through to see if it's rich

enough. That was a session. That was a

timebound session. took me I think 10 15

minutes to try and figure out what I'm

going to test in a planning session.

So, I'm going to go macro to this to see

what I learned, which is a 10minute

planning session. And I've spent four

videos on this, right? Because this is

so absolutely important to what we do

prior to doing anything. Plan it, think

about it, get your head sorted, get your

aims and objectives, figure this out.

So,

when we're looking at like, what do I

actually use to test?

Um we often have stories in agile

projects and very often people get hung

up on well they must be given when then

as a user I want to blahy blah right we

must state why someone wants very often

people are inflexible in their language

and that I think is an issue but that

just means that there's more ambiguity

which means there's more scope for

questioning now when I'm working on

agile projects and I ask questions I

will write them down I will write down

the answers. Um, I will put those

written notes possibly in the tracking

system. If it's feasible to make them

public and it's politically acceptable

to make them public, I will make my

notes public. Otherwise, they'll stay in

my notes so that I know what we

discussed and what time with whom, so

that if there's any issues, we can come

back and go, well, on this day we said

we weren't going to do this, so we said

that this was out of scope in this

story, so we're not testing that. But

it's important often to know what drives

um the requirement because then we can

test around it because one of the key

things we want to be able to do is test

around these stories to go holistic to

the stories and not just focus on the

actual details. And one of the things

that people very often miss out is they

only look at the acceptance criteria.

They don't look at how has it been

built. What is the how do people

describe that? Where are the risk areas?

Are there new new technologies in here?

Have we just adopted and brought in a

new library into the the system? All of

these things can be risk which we can

look at in terms of testing. We can also

use like decisions and constraints

during the development to force us to

test differently cuz sometimes we'll

have stories and they may not have

validation yet. We've decided not to put

any uh validation in place. So there's

no error handling. So we won't test

that. But very often that then drops out

and it just magically appears at the

end. But we then have a whole bunch more

testing to do. Priorities and agreed

risk areas, they're important for

driving it because we do have to agree

where our focus is going to be when we

test. We don't just make this up. We

discuss it and agree as a project where

our focus and priority is going to be.

And the more we do that, the more we are

sharing responsibility for testing

rather than taking it all on board

ourselves. If we take on board all the

decision-m about testing and we we skip

an area and it's perfectly valid for

people to come going why did you not

test this? If you share that knowledge,

prioritization, agreed risk areas,

thought processes, more people are

involved and you you get less of that

blame game. If you do take

responsibility for it, then you have to

cover yourself by writing down the

decisions that were made, the reasons

you made those choices, what impacted

it, why you decided not to do something

because person X said this in meeting,

why make those things written down so

that you you know why you made

particular decisions. We should also

look at some of the ways that we figure

this stuff out in agile processes

because most people will be doing agile.

So very often in agile we've got the

inprint testing. So what we're often

looking at there is we're constrained to

the work done scope right we won't do

error handling testing yet. There's no

essence no focus on security. We're not

looking at performance yet. So we're

artificially constraining what we might

think about as we test. We do need to

track these emissions as coverage gaps

for later testing. Otherwise, it can

slip away, right? Or we forgot to do

performance testing because no one

mentioned it in the acceptance criteria

when we played the last story in this

functionality in this feature and we

said, "Well, what about performance

testing?" They said, "Well, it's not

going live yet, so we won't do that."

Then when it goes live, we forgot about

it, right? We have to track these things

and think them through. And where

possible, do them in sprints. But

sometimes we're going to be constrained.

And we're very often testing bits of

stories for some reason. Um, a lot of

agile projects don't need to be agile

projects, right? They because we're not

delivering value to the user at the end

of each sprint. So, a more canbanlike

approach where the the stories take as

long as they take until they're ready

can reduce that. But very often we'll

take features and split them into

stories in order to fit them into

sprints where we release something at

the end of the sprint but only into the

test environment and not to the user. So

we kind of have half done work but it

looks like completed work because it was

stories. So we we very often have the

the notion that in sprint testing is

constrained and there will be gaps. So

we have to figure out a way of tracking

that either on wiki pages, but you need

something that's going holistically

outside the the scope of the normal

planning process. Stories hopefully are

complete chunks of work, but sometimes

they're not. Sometimes stories span

sprints because they're not finished. So

we have work ruling over um that can be

an issue if we are going live with a

particular uh set of stories or feature

set because we then have to test it in a

state that it's not designed to go live

with to make sure it works which takes

time away from other activities we could

be doing in sprint. To be honest, a lot

of agile planning and processes are a

complete mess. But we have to figure out

how to handle that in our test approach

to help mitigate the risk of omission or

commission like um missing something out

or doing something that we're not

supposed to. That that making sure that

feature toggles work and there can be

huge combinatorial explosion when we do

that when we choose to go live with um

half-finish work that was never designed

to be there and very often it's right at

the end where we have to work squeezed

in. We need to go beyond acceptance

criteria. So acceptance criteria is a

start point. We need to ask more

questions. We need to quiz in the

ambiguity. What's a good approach for

acceptance criteria is having automated

execution because acceptance criteria is

very often it must do this when it's

when we say this it must respond in this

particular way. They can be very simple

assertions that we write in automated

execution, which is very often why we in

sprints automate as much as possible,

but we're really only automating the

acceptance criteria. We need to go

beyond that with our testing. And if we

have automated the acceptance criteria,

hopefully we've automated in such a way

that we can reuse the abstractions that

we created for that to support us in our

exploratory testing to do massive data

coverage. If we can do that, that's

great. a lot of random data thrown in

temporary tactical automated execution

to help us cover the acceptance criteria

more detail whilst allowing us to do

exploratory testing somewhere else. We

have the issue that stories are often

tested in isolation in small chunks.

Sometimes we're working on a branch

where the that story exists and we test

it there. But then when we merge it in,

we rely on automated execution in the

continuous integration process, checking

the merge. That's probably not enough.

But agile processes can be hard to test

in simply because of that small focused

chunk aspect.

Now we have the issue of system testing

is a very vague term and after a merge

we probably want to do system testing

where we're testing the integration of

all these stories in combination. We're

looking at how people might use this in

live. We're looking for API calls that

then show up in a story but that wasn't

mentioned in the acceptance criteria. Um

we want to look at the using the stories

in different orders because certain

users have different needs.

This is one of the key areas that is

missed out in agile projects going

beyond the acceptance criteria and the

stories. So I mean this is why I focused

on that in tracks which is very often

how I'm able to find defects right

because I'm going beyond what other

people are doing. So with tracks um the

release notes were there I was able to

review them. I read this, I focused and

I zoomed in on something. And I was kind

of lucky that that thing that I zoomed

in actually had some issues and problems

which made the testing even more

valuable than just as a process. When

I'm doing this, and I do this on actual

projects, right? I assume that people

have done story testing, they've done

acceptance criteria, that all the

stories are complete, that people have

done the reviews. So, I'm not going to

focus in on those aspects. I'm not going

to review the automated execution

because there's a whole bunch of risk of

duplicated effort and scope if I if I

don't do that. But that's fine, right?

Because I assume that most of the time

when people do automate execution, they

work in tiny chunks rather than the kind

of testing that I'm going to do. One

question is how does a manager add value

on an agile project? And this isn't a

joke, right? This this is a real

question. How does a manager add value

in agile project? because I've had to be

manager on agile projects and I've had

to try and figure out how to add value

and support the testers, but I don't

have the time very often in huge chunks

to focus in on a story, right? And

someone on the project will be able to

focus in on a story for two or three

days potentially and they'll be able to

test that. I might only have half an

hour, an hour, 15 minutes. How can I add

value on a project? I do that by looking

at the project holistically. I look at

the gaps in people's thinking. I know

that people are focused on

functionality. I know that they very

often don't have time to look beyond it

to look at how stories are interacting.

So very often that's where I look. So I

use the process and look for risks in

that process to drive my testing. I look

where others don't. I go more technical.

I look at the messages because I know

people aren't looking at those things.

Essentially, I look wherever anyone else

has not looked

and by I do that by trying to get a

handle on the process people have used

and to look at it holistically. And this

is a key thing to try and do. If you can

build this skill, modeling not just the

application, but modeling the process

that people are using and understanding

the risks that that imposes on a

project, you will be able to find

defects and you will be able to add

value. You will be able to expose

ambiguity that other people are not

seeing.

And you can do this as a manager, which

is fantastic because you can use

whatever gaps you have to add value in

the processes. And the project will hate

you for this.

They don't love this stuff. But um

sometimes you come up with really

important things that they have to spend

an entire sprint or two fixing because

it was a gap in what they were thinking.

So recon sessions, I mean this is a term

that is used a lot in military. We go

and do recon and you look in commando

handbooks or things like that. It's also

used in security testing a lot where

they do recon to find out the scope that

is available to them. Not necessarily

the scope that people said was there,

but the scope that is available to them.

And recon is really important to help

you uh debrief and feed into your

planning process. Recon is really

important to help you build models.

Recon is really important to help you

understand. Recon is really important to

ask very small questions, get

information back that might change your

view of how your testing should approach

this particular application.

So my chosen scope was in the release

notes where you can now change the state

of a context to closed. Great. But I

don't know what that means yet, right?

So I'm going to have to do something to

find out. I so I need an exploration

process, a recon process to do that. On

some teams that might be going to speak

to the product owner and the programmer

and looking at how it's built and

possibly so for me I the only I only

have one place to ask these questions

which is the application. So I need to

create a session around the context

state to help me answer these questions

to know what that actually means. which

means I then have to do a um recon

session.

So, I have um chosen what I'm going to

test. I've figured out what my scope is

going to be. I've planned what I'm going

to do. Now, what's interesting for me is

I don't actually know what that real

scope means yet. I don't know what the

requirement means. So, my my the thing

that I'm going to target is you can now

change the state of a context to closed,

but I don't know what that means. But

what I do have is some models of that in

terms of both state machines. Um, and I

know how to explore state machines. So I

have a vague concept of a state machine

in here and I have the concept of an

entity called context, but I don't

really know how that fits into the

application. So rather than just

immediately starting to test um this

particular uh concept, what I'm going to

do is I'm going to do a recon session,

right? Because I have to build models. I

have to understand what it is that I'm

working with. And so I'm going to start

with a an aim which is to try and

understand and model this right the the

rough charter that I I created the set

of questions to explore this to to

figure out if what it is I have to

actually do here to come up with a more

concrete um scope and plan.

So I've made my notes as I went through

um when I started, what the time was uh

the uh application is now running on a a

different IP address because I've

restarted the virtual machine. So I'm

trying to figure out what is a context,

right? So a context is I can see it

here. So I can create a context when I

create a to-do item on the home screen.

So then I'm wondering well what are the

views to read this? So I'm thinking in

terms of CRUD like the context is an

entity. So I can create it here. Where

can I read it? I can read it there. I

can see it's there. Where can I update

it? There's an organized there with

context. I've got an edit button there.

I can edit the context there. Where else

can I update it? I can update it on the

context view itself. And then I've to

see that I've got a different edit view

there. So I've got a but I can't edit

the state here. So there's a place where

I can edit the state and a place where I

can edit the entity itself. So I'm

exploring the application just to try

and learn where that entity fits into

the um application. I assume that if I

do something to the context then I'll be

able to export it cuz export says

context. So that can be a good source

for observing the expected results. So

now I've got access to observation at

the guey and observation of an extract.

I haven't yet um extended my observation

abilities into the API. I know there's

an API. I haven't extended it to um how

things are created by looking at the

message that that are sent back and

forward. So I'm making notes on that in

my actual exploratory notes. I did

notice as I was exploring this that

there was a uh defect, a perceived

defect on the uh starred view here with

this dropdown. You can see I've got a

different result now than I had earlier

on. It's lucky I captured the

screenshot. There's differences in how

this is rendered, but I mean clearly

that rendering is not as good as it

should be. And clearly the rendering

that I experienced earlier on was not as

good as it should be. But there's a

variation in there that I I don't

understand why it would be different

from one to another because this is the

same browser that I was using earlier

on. That's one of the reasons why it's

useful to capture um images as we go

through our testing and exploration

because things change and if we don't

have evidence of that, we might not even

I might not even have realized that what

I'm seeing here although this is um not

correct is different from what I thought

was not correct before. So having

figured out um that how to change the

state I don't do it in that one I do it

in

I can change whoops organize I can

change the state in here and then it

will move between these different um

contexts. So then I started to

experiment with this and changing it to

closed. Do an update and then I get this

message. State cannot be changed. The

context cannot be closed if you have

uncompleted actions in this context. So

I can see that there's a guard on one of

the state transitions. Can I hide it

when there's a a an unsaved thing? Yes,

I can. Can I move it back to the state

that it was in before? uh even though it

has these actions

and the answer is yes I can. So I can I

can move it to closed but there seems to

be no constraint moving from active to

hidden. So that's interesting. We have a

constraint.

Let's move that to closed. And it says

no we can't do it. And then I thought

okay so I can't move uh context into

closed when there's uncompleted actions.

But what if I have a closed context? Can

I add an action which is uncomplete to a

context

which is closed? The context B is

closed. So let's add this to context B.

And yeah, I can do that fine. So I

cannot move a context into a closed

state when it has uncompleted actions.

But I can add uncompleted actions into a

context that has a closed state. And I

don't know if that's a a a bug or not.

But it it feels like I shouldn't be able

to do that. It feels like I shouldn't

necessarily let me do that. And I've

explored that in my notes in here. Now,

what I've also done, I mean, that's a

mild distraction, right? because what

I'm supposed to be doing are learning

the rules around this, not necessarily

testing it. But the session was so short

um that I I followed that through. I've

made a notes of a more formal set of uh

transitions that I want to do. I've also

made notes on well

can I move from the current state to the

current state? Uh now I when I edit the

context in here and I change the name

and I go and I leave the state active

did I change it from active to active?

No I don't know cuz I don't have the

ability to observe the system at a low

enough level to know whether it put the

state across at the same time. So I've

got limits in my ability to observe the

system which prevent me from testing

effectively parts of the system. So I

know I need to increase the my

visibility. So I'm going to need tooling

in order to do that. And then I started

to explore what entities relate to other

entities. And then I found that it was

odd being able to um add that to-do into

a context that is already closed. So, I

would probably raise that as a a bug.

And I've gone through here looking at

are there any other state combinations

because there's not a lot of entities in

here. We've got contexts, we've got

to-dos, we've got notes, we've got

repeating to-dos. And again, look, we've

got a rendering issue in here. So,

really, I need to get the um screenshot

tool up to have that to capture that

rendering issue. So, in the repeating

to-dos, um, can I create a repeating

action for a context that is closed?

What would happen under that situation?

Would it be fine? Would it work? Should

that happen? And it's that overlap

between state machines that I'm finding

interesting because it wasn't mentioned

at the front. And that's why I'm

thinking, you know, that holistic view

looking at the system, looking at the

going beyond the beyond the story and

looking at stories in combination. And

it doesn't have to be stories when we

think about that. It can be entities in

the system. It can be functionality in

the system. Stuff that was there already

that we take for granted but we put in

something new like a new state and we've

got some constraints but does it work at

all the constraints? And then I'm

thinking well does the API um enforce

these constraints or can I bypass these

constraints by going through the API?

And again in order to test that I would

need more tooling because I can't do

that through the front end. So, because

this was a

a recon session, I summarized my

findings to help me go forward because

what I'm doing really is building a

model of this uh concept. So, I've got

I've got my model, I've got my

constraints, and I've made some notes

about what to to do and made some notes

about what I might do going forward. And

that whole session had a gap in the

middle of uh 8 minutes cuz someone came

to my front door. And I started at

uh when did I start? There we go. 22

minutes past 12 and finished at uh 13

minutes past uh 13. So that's uh less

than an hour. Now, I didn't um really

put limits on the uh the session, right?

Um I didn't have a timer, but what I've

done is I've just tracked my time. And

as you can see that I stuck in some um

timestamps as I went through because I

was conscious that I was making a lot of

notes, but not necessarily tracking the

time. So, I've got my to-dos in there. I

can revisit this. The notes are quite

detailed. I can use that to debrief

someone else. So I can use that to

revisit to help me plan other sessions.

But I've also got my model which was the

key point for the recon session to

understand how this fits together and

how I want to work with it in the

future. This would lead onto a planning

session so that when I come up with my

other charters, I know what it is I'm

going to do. Um and also there's there's

nothing really different about the the

functional testing in here. Right? I

used all my functional testing thought

processes to come through to work out

the state transitions to explore the the

state transitions to see if I can skip

states. But what is key here is that my

modeling and assessment of my findings

is limited by my ability to observe the

system and I and limited by my ability

to manipulate the system. Um because can

I move a state from the current state

into the current state? I don't know if

that's what that functionality did. And

if it didn't, I need the ability to send

through whatever message it sends when

it's changing state to change it from

one state to the other in order to do

that transition. So I'm currently

limited by the technology that I'm

using. But because I understand the

architecture of the system, I know what

technology and tooling I need to bring

in to help me. So I don't start with

tools. I iterate towards tool usage as I

as required as I hit limits in my

ability to test and observe the system.

Let's go a bit meta on the the recon

session.

I have to plan before doing a recon

session. My recon session is about

helping me understand what's going on.

So I but I'm still making notes, right?

Every time I hit anything in a session,

I'm making notes. Anytime I'm going into

a meeting to discuss um how the story

works, I'm going to make notes. Anytime

I go over to someone's desk, I'm going

to make notes and I'll tidy them up when

I come back because these are

exploratory, investigative sessions to

help me understand. And I'm I'm making

notes. I'm making to-dos. I'm writing

down the findings. I'm asking extra

questions. Um I'm making notes on what

things seem to be, but I'm not doing a

lot of um interrogation to find out.

Sometimes I will, sometimes I won't.

Because it's a recon session. It's I'll

do whatever I need to look at in order

to get things working. I'm tracking any

defects that I see again and I'm asking

questions as I go through to help drive

my um investigation. And again, I'm

making notes on my questions, what I'm

thinking about, what the answers are.

This is a session just like anything

else. We don't necessarily know what we

will find when we do these sessions, but

we know what our aim is. And the aim is

for me to come out the other end with a

better understanding of the contexts to

know how to test it. And at the end of

my session, I made some findings. This

isn't a debrief summary. This is just a

a kind of insitue

set of findings.

So this is my rough model at the time,

right? I've got to-dos or actions. They

can be added into contexts. I seem to be

able to move context into different

states in any order, right? But I know

that I need to cover that in more

detail. I did find one constraint that I

cannot move a context to closed if it

has open actions, which seems like

something that would be really good to

drill into in more detail because that's

a constraint. It has to be enforced. How

many ways are there of potentially

bypassing it? I know also that I can add

open actions to closed context when I

create an action, which to me seems to

violate the constraint. So I think

there's a lot of scope in there for

looking at that state and the overlap

between actions and actual constraint

and and contexts. I found it a little

bit odd that all contexts are shown in

the dropdown. Uh I would have thought

that closed contexts wouldn't be there.

I haven't investigated how the system

actually updates the back end. It looks

like it's XHR requests, but I'm not

completely sure. So, I know that I've

I'm making a note that I need to get

more detailed in my investigation. And

even though XHR is used, there's no

observation pattern so that all screens

are sent messages or screens are pulling

the back end to see if the state of what

they're displaying is complete. So,

there's a potential there for multi-user

access, possibly overriding someone's um

state change without being informed

about it. Who knows, right? There's

extra scope in there that is occurring

to me when I'm doing this. So, I've got

new test ideas on how I can um approach

this. So, going meta to this, the basics

of functional testing are functional

testing, right? all the stuff that we've

learned about boundary values, um, odd

values, error validation, functional

flows, looking at different ways,

ordering, um, checking defaults,

whatever you know about function testing

that is that remains the same in any

testing session. But the architecture

and technology change how we test,

right? They change our ability to

observe the system. My ability to model

this and assess the findings is limited

by my ability to observe. I didn't have

anything that was looking at the network

traffic. So I don't really know how this

system is properly working. I'm limited

in how I'm approaching things.

Therefore, I know that I need additional

tools to go through. I know that I may

need to understand more of the

technology. And this is an evolving

process as we test, as we work on the

project. I don't start with a particular

tool set. I don't start with a

particular list of technologies that I'm

going to study. I iterate over these

things as I work on projects, as I work

on the system to help me ensure that my

ability to interrogate the system,

observe the system, manipulate the

system at all the different touch points

increases as required to be the right

person to test this. That's constantly

what I'm trying to do. I'm trying to

evolve to make sure that I am the best

person to test this so that when we like

do actually get to the end of this

project,

I am like at the peak of being able to

test this particular application, this

particular technology, this particular

functionality, and I'll learn that over

time. I may not be able to de-risk at

the start, but that's fine. I've been

put on the project and I'm evolving as

we go through which helps derisk the

fact that I'm on the project in the

first place. So it's a everything is an

exploratory process. Our learning our

skill sets even as we work on it is an

exploratory process.

So, the recon's given me quite a lot of

information so far. I've got a lot of

notes. Remember, I haven't really

started testing at this point. So, what

I really want to do is uh debrief is get

my head around what it is I've actually

done. So, I'm going to look through uh

the notes that I've got. I'm going to

try and collect them into some sort of

summary so I understand the status that

I'm at. If this was a real project, I'd

be pulling out the um issues, the

to-dos, the questions. Um I'd be

clarifying those with the team. I'd be

writing down the defects and raising

those in a system even though I haven't

started testing yet. I've I've learned a

lot of stuff and I just want to codify

that in my head. And that's going to

feed into a planning or a modeling

session. So the way I do the debrief is

I've got all my notes. I basically just

uh start at the top then start working

through them. And what I'll do is I'll

have the notes I've taken in whatever

format and I'll have a completely clean

editing session up. I'll just write down

debrief session the time date so that

again I'm tracking I'm always tracking

what time I'm doing. Um I write down the

aims for my session which is to review

notes blah blah blah. So, I've always

got an aim, right? Every time I'm doing

something, whatever it is, um you want

to call that a charter, that's fine. You

want to call that objectives, aim, plan,

whatever it is, I don't mind. I just

clear in my head, this is what I'm going

to do from this point until I finish.

And if it seems like it's taking too

long, I'll add more timestamps in here

so that I can track it. Um, and I just

went through making all my notes, um,

expanding it. Um, and I've made a

decision, right? The what I need to do

is I need to explore the state and

entity life history relationships in

more detail and move this into a

planning and modeling session to prep

for a more comprehensive coverage

approach because one of the things that

happens when you're doing exploratory

testing is uh you get wrapped up in it.

It can be hard to step back do the

modeling understand the scope of what

you've done. So we need to put sessions

in place where we can do that. And for

me that's a debrief session. Uh what do

other people call this? Um not not

stepping back but defocusing I think

some people call this. So but for me

this is a debrief. I'm just I'm

debriefing with myself. And this is an

invaluable process when you're working

on your own. Very often we're testing on

our own. We don't we can't bring someone

over and say hey interrupt what you're

doing. Come and look at my stuff. It's

useful to have that if you're doing a

formal sessionbased test management

process. you have that in the absence of

that and most people are working in the

absence of that we can do these debriefs

on our own by looking through our notes

and if our notes are good enough that

we'll be we'll be fine. If the notes are

not good enough then we need to make

them good enough to support this process

because this process is essential for

getting our head round. We're modeling

what we've done. We're building a status

of where we are. We're reflecting on

what has happened. This is absolutely

essential to our process. You can see

that what I've also done is as I went

through here um I collected the times so

that I know how much time I've spent on

this and what I'm working through and uh

I made decisions going forward and

basically I then took these notes. If I

was working in Jira I might post these

into a task on Jira. If I'm working more

informally I'll save this as a file. Um,

at the moment I'm just saving everything

in a a big block in Evernote. That seems

to work fine for me for what I'm doing

here. But I now have a reflection of

what I've got. If it was a real project,

I would do more in terms of writing down

all the questions. They'd be stated here

as questions, to-dos so that I could

plan things down, issues, um, so that I

could raise those. And then I go and um

communicate this to the team, ask

questions because remember I have no

idea when I'm testing this particular

application if the things that I'm

seeing that I think are problems are

problems. I don't know if it's a problem

that I can add a new action to a close

to-do. I think that's a problem because

it it seems like bypassing a constraint,

but I don't know that. So if this is a

real project at this point in time, I

would be communicating and asking that

as it is. I'm going to um build that in

to my next exploratory session because

that gives me something more to focus

on. So, the next session I'm going to do

after this is a a modeling session to

help me get my head around what it is

I've actually got to try and explore,

what coverage I want to build on that,

but what models I want to use for that

coverage.

So debriefs are essential. You can see

in this model here, debrief is happening

after every single type of session. The

only session it didn't happen after is a

planning session. But in theory, that's

part of the planning session. That

debrief to make sure that we're happy

with the the plan that we've got working

forward. And essentially we revisit that

plan when we start doing any other

session. Anyway, so in um concepts like

session based test management or formal

management of exploratory testing, the

debrief session is often done with

someone else. So you'll sit down,

debrief, you'll pair with them, you'll

go through your notes, you might have

the system up and running at the same

time to demonstrate, oh, I did this.

That's how we interpret the notes to

trigger your information to have a look

at the data in more detail. Sometimes

it's as it's is as simple as just

sitting down with someone and working

through what you've done. Debrief

sessions are so key, but people

sometimes leave them out when they're

actually working on their own. And we

can't afford to do that. So by debrief

session, it basically means stepping

back a little bit, looking at what we've

done. It might mean um tidying up my

notes. It might mean collecting my notes

so that I look through and pull out all

the to-dos. I pull out all the defects

which gives me the chance then to plan a

admin session for raising defects an

admin session for updating the test uh

conditions that we might use in the

future right all of these this collation

process and I will track this collation

process because it's a debrief session I

will track the time that I take on it to

work out what we're doing I might refine

the models the notes that I made on the

models but I might also then have to

plan a modeling session later if it

becomes clear that it's going to take

quite a long time to actually plan this

and that my debrief session is really

moving into a modeling and planning

session. So I going to have to be

careful about bleeding the sessions

across. So debrief session is a way of

tidying up making us clear of what we

did and recognizing if we didn't achieve

enough coverage. So potentially creating

some to-dos to revisit some of the

charter that we said we were going to

work on. debrief session is absolutely

key. I can't stress it enough. So, my

notes were relatively small for the

debrief session. Um, my debrief session

started at 10:40

and scroll and scroll, when did we

finish? At 10:53. So, this was a a

13minute process. I wrote down uh my

aims to try and figure out what I'm

going to do. Um, I summarized what I

did. Um, I made notes on what I need to

do going forward. I've also said that I

tidied up my notes to had a timing

session so that I can track this. And

then I've made notes on what I would

actually do on an ongoing basis next for

an admin session if I was doing this in

a real project where I' take all the

to-dos, take all the defects, raise the

defects, ask the questions of the people

on the team. This is what a debrief

session is to figure out what is my next

action. It's kind of like a planning

session, but it can lead into a proper

planning session. But it's taken all the

information from what we learned because

sometimes we have to when we do explore

to test, we have to extract

our coverage. We have to extract the

questions that we had because we cover

so much stuff in that short period of

time that the debrief session is a way

of pulling as much value much extra

value out of that beyond the

observations and interrogations that we

made during the session. So this is a

way of pulling out more value. Debrief

is absolutely key. It's why at the end

of the day um I will look through my

notes and debrief and write down what

went well, what didn't, what do I want

to do carrying forward. Are there any

new things I want to track? Am I still

on track my overall goals? These

debriefs are just a standard um to-do

management approach and just management

approach for getting better and

improving. It is a feedback mechanism

because it is information coming out of

our process that is feeding back into

helping us do the next thing. So lessons

learned. So I rely on linear notes now.

So I I just make my notes in order,

right? And I I use my debrief session to

potentially add in summaries to

potentially add more uh titles in there

to potentially add more structure. I

very often don't edit the notes very

much, but I can do that in a debrief. I

tend not to worry about formatting and

aesthetics very much, but because I'm

very often writing in markdown, I may go

back and add extra blank lines to make

sure things format in markdown.

Um, I do tend to use text files a lot.

Um, I do use Evernote, but sometimes

Evernote is a little bit clumsy to work

with because Evernote adds formatting.

When I when I start writing markdown,

Evernote will add extra formatting

there. It would be really good. The

ideal for me would be Evernote supported

markdown, but I could paste screenshots

in and then when I extract it as text,

it extracts out the whole thing, but I

haven't found a proper editor that does

that. So, I tend to use text files.

Cloud app is really good um screenshot

tool because it will paste um a URL into

my clipboard. When I take a screenshot,

there's a URL there. I can paste that

directly into my notes, which is

brilliant. Um, so different tools in

combination work in different ways, but

my standard approach is I'll have all

screenshots in an images folder because

that's kind of standard for working with

markdown. What I often find is that

having done a debrief, I've got a

summary of what I'm doing or what I've

done, which I can then use as a status

report to copy or paste into Jira or

whatever management system we're using.

So, I may not put my raw notes into

there. they may be saved somewhere else

but the summaries at a minimum will be

put into the overall project tracking

system so that people have insight and

visibility into the testing that I'm

doing. But the debrief is an absolutely

key approach when we do exploratory

testing.

So I moved on to a a modeling session

right when I went through the debrief I

made some notes and I realized I need to

try and understand the the context the

states the action states and the

relationship between those two entities

and their life cycle in more detail so

that I can try and identify and plan for

a better coverage approach to feed into

my exploratory testing. And you can see

here in my um notes, I'm saying see the

notes that I made on the smart pen,

right? Cuz what I'll I don't really I

want to try and do things as effectively

as possible, as fast as possible. So,

I'm not going to get out a diagramming

tool and start diagramming. I am going

to make notes. And the way that I make

notes is I have a smart pen. And you can

see here at the top of my notes, it says

test new pen, right? because my smart

pen broke and I used the guarantee and

got another one and it's working. So now

um when I write on my smart pen pad, it

automatically syncs into Evernote so I

don't have to do anything. Um the most

uh reliable way is to probably have a

pen and take uh photographs and we do

that of whiteboards. We do I do that in

my notebook when I'm doing things when I

don't have access to the smart pen. But

when I have access to the smart pen, I

will use that because even if the smart

pen doesn't sync properly, I can still

take a photograph of it and feed it in.

But I didn't have to do any work. I drew

my diagrams, my notes, and it

automatically syncs into Evernote. So

you can see what I've done here. I've

explored the state of the context. It's

not a big state model, but I can go from

active to hidden to closed. And I can do

that in a variety of ways. And I've

marked down the constraint like that. I

cannot move or I should not be able to

move to closed if the context has open

actions. And I made a note there that I

only did it with one open action.

Really, it's multiple open actions. So,

I really need to expand the scope to be

greater than or equal to one. So, I've

got a bigger set of things in there.

What I'm also conscious of is I really

only did the switch from trying to move

from active to closed. I didn't move

from hidden to closed and see if that

makes a difference. So I have to explore

the um moving into closed states um to

check that constraint. But even that's

not going to be enough

because

this the context here is related to

actions. So I really have to explore

actions in relation to this and actions

can have their own state model. So they

can be open or closed. That seems to be

what's important uh for this particular

constraint. Actions have other states or

attributes. They can be starred, not

starred. They can be dependent on others

or not dependent on others. So what I've

done is I've tried to make notes on that

using a a crude entity life history

model. So these are not like formal

diagrams. You you could probably model

these in formal tools, but I'm not using

them in completely formal. I'm using

them as the concept of there are states

when we switch between them. There may

well be there's in theory there's more

states in context because it's being

edited. it's there's other things that

are going through in the messaging but

I'm not modeling that at this point.

So I thought through well what is the

actual life cycle of the context so I

can create it when I create it it's in

active mode it's alive as a context i.e.

it's it's been created and it's not

deleted. So I've called that live. Then

I can do multiple things. That's an

iteration symbol. I can do multiple

things at different times. I can make it

active. I can make it hidden. I can make

it closed. I can do other things. I can

edit it. um edit the details but I

haven't represented in that in here.

This is a model. It doesn't include

everything. Um then I'm aware that well

there's a dependency between the

actions. So I can add actions to a

context. Um either moving them in or

editing the action to be part of the

context or creating the action to be

part of there. I can delete actions from

the context. I can move them out. I can

have repeated actions that will add on

to the context. So I really have to res

explore that because that is related to

the context and I really have to explore

that with all the different states that

the context can be in and I've got a a

fairly good set of coverage then if I do

that but then if I look at the uh action

itself I can create an action and an

action has to be part of a context when

I do. So I've got a reference there to

remind me there's a relationship between

the state when I create this. So I

should really create an action for an

active context, create an action for

hidden context, create an action for

closed context. I should do those

combinations.

Then I'm thinking, well, I need to amend

the actions. What does an amend mean?

And I haven't represented in here that I

can amend the text or the due date or

anything like that cuz that doesn't seem

relevant to the uh part of the system

that I'm exploring. So I'm deliberately

excluding parts, but it's obvious what

I'm excluding because it's not in my

model here. Um, so I can star something.

I don't know if that's important or not,

but it might be. And so I put it in

there. I can change the context. I can

go into the action and amend it and

change the context, which essentially is

starting to relate what happens to an

action with what happens to a context

over here. When I change the context for

an action, it deletes it from one of the

context over here. So these events are

related. Um, so I can close a an action,

I can open it, and you can see that

that's represented as the action state

because I'm interested in the action

state. But that's an event. I can open

it or close it. It the event then causes

a state change. And I'm really only

interested in these state changes. So

whatever here impacts that state change,

I'm modeling as an event. Then I'm also

thinking, well, does it matter if um I

try and have dependencies on action,

right? Because this just says context um

can't be closed if it has multiple open

actions. What if it's a an anclosed

action that is dependent on something

else? Can I do that? Can I have that? I

don't know. So I'm putting in here I

want to try and explore the dependencies

at this point as well. So that's what

this rough model is and it's a a well

this is a diagram of the model in my

head because I'm trying to clarify the

model in my head. So it's not a set of

formal models and I think of these as a

state model and an entity life history

model. And I'm trying to pull out the

events. And the reason I'm pulling out

the events is that if I ever want to do

any um ad hoc or quick um automating on

this and really have a lot of

combinations. Um I can just have event

triggers that say um just explore these

different events. We've got a context.

when we have a context, have a an event

that switches it into active. Um, have

an event that adds an an action to it,

have an event that explores this, and we

can just iterate over all of these and

at some point and we can make decisions

about, well, should we allow this,

should we not, or I can review the event

sequences that come through and see when

a message came um across and when it

didn't. So I've got a lot of scope for

this, but thinking it through this way

really helped because what I was then

able to do was then start make a

decision about what to do next. So this

is essentially the results of a modeling

session which then feeds into a planning

session and my plan is very simple and

again I did this in the notepad because

I'm I'm still working on that model

because I don't know whether this is

complete or not. So what I've done is

I've identified the things that I think

are most important to explore. So you

can see also that the the names that I'm

giving these sessions are are not like

fixed in stone. They're not so it's not

like you have to do a modeling session

then a planning session. It's the main

aim for that previous session was a

modeling session which fed into a

planning session but the planning

session was so small right 2 minutes I

didn't track it as a separate session.

So I've included that in the modeling

session. So I'm being flexible about how

I model this. It's really important that

when we model our processes, we're

flexible about it. Um because otherwise

we're going to get hung up on did I do

this properly? Did I do this formally?

Am I doing this the same way that so and

so does this? And we we never want to

get hung up on those things. We want to

get hung up on being effective and doing

the right things. So here I've

identified what I believe are the most

important things to explore next. If I

was working on a project, I would

discuss this. I would then agree this,

but I'm not. So, I'm making the decision

myself. So, I'm thinking the most

important things are well, put a context

in a closed state and then try and do

these things to it, right? Um try and

have an action event. So, I can create

an action event on something that's

closed. Create a normal action event and

create a repeat action event on there.

So, I'm trying to go through. I'm also

aware that I need to explore the action

state, but action state isn't relevant

to created because it's always created

in an open state. But then I'm also

thinking, well, I want to change the

context. So if it's a an action that's

on a closed context, I'm going to want

to change the context, possibly move it

to another one that's closed, another

one that's open. And is the action

itself going to be open or closed? Well,

the only ones that's really important is

open. So here I'll probably explore an

open action which is on a closed context

because I know how to do that. Then see

if I can move it into an open context.

Move it into a closed context and try

and move it between these things. Um and

I'm just going to try and explore this

model. This would be my rough plan for

doing that. And I probably won't um go

further than this in a planning session.

If it was an agile project and I didn't

have time to do this straight away, then

I would probably spend time um planning

this out. If the functionality wasn't

ready yet, then I might spend time

planning this out in more detail and

writing down exactly context and close

state and create an action. Context

closed state, create a repeating action

um against that context. context and

closed state. Um, take one of the

actions on it and try and change the

context and move on to another closed

state. So, I would expand that at this

point

and because that's the planning is as

in-depth as you need it to be to either

remind you what you're going to do next

to plan it so that someone else can do

it to plan it so you can communicate it

so that people can say no and review it.

Planning is supporting communication and

it's supporting helping you think about

what it is your next steps are. It's

helping you estimate how much time am I

going to spend on this? Am I going to

have to do going forward? So, I combined

two things into there into a rough plan

cuz my assumption is I'll just go

straight away and do this. If for some

reason I don't get to go straight away

and do this, I can come back later and

expand this, right? Because it will

remind me what is going on.

it's clear enough to help me. So that

was a a modeling slashplanning session

to help me get my head around what I

need to do. And also I'm aware now the

amount of uh complexity

uh and relationships in here and I

haven't included projects in here. I

haven't looked to see whether a project

state is related to this. I'm also

conscious that this is very informal. So

there's a risk that I might not uh have

ex have explored the complexity

possible. There might be a better way of

modeling this such that I could use

techniques. Maybe there's an an actual

event modeling um diagramming approach

that I can use to help me understand

this. And then maybe there's um

different ways of uh covering that

automatically that just aren't springing

into my head at the moment. Maybe

there's a tool that I can use to model

events and have it come through. Maybe I

could feed this into um Graph Walker and

have it pull out the the states

properly. Maybe I can combine multiple

graphs um in Graph Walker and model it

that way because maybe I don't have to

use it to actually automate, but maybe I

can use it to generate hints that I will

follow. Right? So there's possible

options around tooling here because I'm

starting to think this through in a more

formal way. but modeling, planning, um,

and the tools that I've used are text

tools at all the time, but I've added in

a little bit more technology. So, this

is a smart pen so that I don't have to

take um, photographs of it and tidy them

up and put them on in my notes. And uh,

I'll see what I do next.

So after the recon session, it was clear

that while I understood

roughly what was going on, I really

needed to model it to be able to

approach it from a effective coverage

position. Different phases feed into

debriefs where I decide what I'm going

to do. Go back to planning and go right

now I'm going to focus on modeling to

work out how best to cover this and it

all feeds in the different sessions feed

into each other because we're gaining

more information and understanding about

what we're doing. So this is the note

that I made in my earlier uh session the

recon session. I needed to prep a more

comprehensive coverage approach because

I didn't test this, right? Test is in

quotes cuz what does test mean? But I'm

aware that I kind of did a cursory

overview of it. Um, so I did an initial

basic exploration, but I didn't really

track what I did. I didn't really make

decisions about this state followed by

this stage. So it was a bit ad hoc. I

may not have effectively taken notes on

the coverage. I could probably extract

that out the session, but it was easier

just to go right, let's just do this

properly. Let's think this through.

Let's model this and get a good

understanding. Rather than dive straight

into a diagramming tool, and there are

loads of very good diagramming tools, I

instead decided to draw it. And here I'm

using a a smart pen. So, it's a

Livescribe aer pen, which means that

when I draw on a notebook, it

automatically updates in Evernote. So,

I've always got an image of what I'm

doing. The pens can have issues. I had

to resend off for a new one because my

um pen broke, but when they work, they

are fantastic. And so, this was um two

days ago, I got a new pen, which was

just in time for this. So, I used that.

If I didn't have a smart pen, I would

write it down in a bit of paper and take

a photograph of it. And I've written

scripts over the years, I think

somewhere in GitHub, which uh use image

magic to straighten images and make them

black and white and get rid of all

background and other stuff. And I've

used that on whiteboards. And there's

actual tools you can use and buy where

you take a whiteboard photo and it will

um get rid of the background, straighten

up, make it look nicer. But the point is

we don't need expensive tools. We just

need something that supports us thinking

through the problem. And I make the

distinction between modeling and

creating a representation of that model.

Because what I'm doing here is whilst it

looks like I'm drawing diagrams, what

I'm doing is I'm modeling the entities,

the states,

the relationships, I'm trying to get

that in my head. What I've got on paper

is a representation of what is in my

head. What you can see here is something

that I would interpret. It doesn't have

all the bits of information. It doesn't

have everything that's in my model.

There's no screenshots there. There's

nothing about the actual fields that are

on a context. All of those are in my

model which are in my head. But I'm

making a diagrammatic representation

which allows me to zoom in on some of

those um aspects. What I'm interested in

modeling here is the state transitions,

the constraints and the relationships

between the different entities. And I'm

concentrating on two entities here which

are the action and the context. I'm

ignoring projects. So you have an

understanding of state transition

diagrams, entity life history diagrams,

um flow diagrams,

sequence diagrams, all the formal types

of diagrammings you can have and then

once you've got that understanding,

throw it away so that you can model

informally as you require. The formal

models are really good when you're

working with tools that will then do

something with them. Right? A formal

state transition diagram is really great

when you're working with a tool like

graph walker which can then interpret

your state transition model and uh

automate the application for you from

that model. Formal models are great when

they have tool support that will do

something beyond creating a diagram.

Diagrams are there to be interpreted

into supporting someone else's modeling

process. So with this modeling is a

fundamental skill with testing and when

people um people are often not aware

that they're modeling all the time right

everything that we see observe learn

that goes into a model in our heads very

often we talk about what intrinsic

models internal models mental models

doesn't matter all models are mental

models what we do is we create

diagrammatic representations or

communications of those to help us focus

or to help other people understand what

we're doing. I knew I had to do a

modeling process because the model that

I had after doing the recon session was

ambiguous. It wasn't clear. So I needed

to do a modeling process to get clearer

so that I could understand what I was

doing so I could build scope for going

forward. Very very simple concepts can

expand into very complex models.

not um complicated models but complex

models, rich models, models that have a

high degree of interaction,

a lot of combinations. Now the benefit

of formal models is that they have tool

support. So you have state transition

diagrammers that will help you um create

very exact state transition diagrams

that are exact to the spec that can then

be interpreted by other tools.

We don't need to use those. It's very

useful to have a grounding in those. so

that we understand what we're doing and

we understand the different uh

perspectives that we can create when

we're drawing diagrams. We can look at

state, we can look at the life

histories, we look at entities, we look

at attributes, we can look at

relationships, we can look at flows, we

can look at data flows, we can model the

data. There's a whole bunch of different

diagramming approaches that can support

us in that form of modeling. But too

often I see people jumping to a tool.

What we really need is some way of

supporting us with our modeling. So I

use pen and paper, sometimes just a

camera and I'll capture that, copy that

from wherever the photograph is stored

into Evernote. I I've done consultancy

reports which are have appendices which

are just diagrams drawn on paper or on

whiteboards.

I think it's important to just capture

and provide and then interpret around

it. So if I do need to create diagrams,

I'll use something like draw.io. That's

where I created the diagram that you're

seeing in this um set of slides. Um I

draw.io is free. You can download a

desktop version. I find it quite useful.

Um if I'm wanting a codebased solution,

um I'll use graph viz which uses a

domain specific language for creating uh

diagrams. And there's a whole bunch of

text to uh image diagramming tools that

I sometimes use as well, but I don't

necessarily need to use complicated

tools. The days where we would buy the

pro version of Vizio and script it have

gone now. so many options open for

creating diagrams, but I still rely on

pen and paper when I want to try and

think through the problem and change it

before um codifying it for more formal

communication processes in an actual

diagram. But the modeling process, it's

that modeling session to actually spend

time getting your head straight is

really important and that will lead on

to planning sessions.

So, another planning session. Having

modeled the states, I then want to plan

what I'm going to do next. And it's a I

kind of bled those two sessions together

because the modeling session was maybe I

don't know 40 minutes. And then the

planning was just a quick plan

diagrammatic drawn on paper um which

took about 5 minutes. So I didn't really

track that separately but it is the

concept of a planning session. And this

was it. This was what I planned. This

was my plan.

So I've identified the most important

areas to explore which is um some states

and some state transitions. Then I'm

looking at the events that an action can

undergo and then I'll try and map the

states to the actions and I'll I'll

explore this as a model. So that's my

quick plan. Now if this was a a real

project, what I would be doing is I

would be adding these as notes in a Jira

task or making them visible because we

may not do them immediately. Um putting

on a canban board, discussing the ideas

with a team. Um, if I wasn't doing it

immediately, then I may need to track

this better, expand it a little bit

further so I don't forget my thoughts.

A lot of our planning is related to how

quickly we're going to implement that

plan. How detailed do we plan, it's how

quickly we're going to do it. If we're

not going to do it immediately and we

think we're going to forget, but it's

important and we've got lot of ideas, we

may go into it into more depth as a plan

now. But if we're going to do it

immediately, we may plan on the fly. So

it depends how exploratory our project

is going to be. So one of the things to

to point out is that modeling and

planning are so related because when we

have a model and when we have a a

relatively formal model um that's that's

can be used for coverage. So planning is

really prioritizing parts of our model

coverage.

So I have a a model there which is a

diagram with the state transitions and

I'm going to cover those state

transitions when I do my testing. It's

going to be a coverage approach.

Um having a representation of the model

from a specific perspective state life

history um hierarchy flows events just

makes it easier to see. It makes it

easier to see what coverage we're going

to do now and what we might have to

leave until later. Right? Because an

initial plan will help us just focus

which allows us to maximize the

effectiveness of our testing. And we

only this is this is key. We expand our

plan only as far as we need to support

the next session. One of the issues we

had with uh old-fashioned structured

testing is that we planned too far,

right? We planned in so much depth we'd

spend months planning and create more

plan than we had ever got time to

implement and then you'd have to go

through planning sessions to scope down

and depp prioritize and it was a mess.

It was just a huge waste, right? But

that was what we did but we overplanned

and the what we want to do is plan

effectively which means minimal planning

to support us in the next step. And my

next step is going to be a coverage

session based on that simple plan, which

means I'll have to expand that plan in

my coverage session.

So I've got the concept of a coverage

session. And a covered session for me is

when I have a relatively well-

definfined model and a a formal

representation of that model and my aim

is to cover it. That's a coverage

session because I'm focused on working

through the uh steps. And a coverage

session could be as simple as here are a

list of acceptance criteria.

Check whether all these acceptance

criteria have been met. Here is a a mind

map of areas. check that all these areas

exist. Here is a list of status quotes.

Check that the API returns these status

codes. Right? I'm covering a specific

set of things. So a covering session is

is the first real session um in this

entire exercise that is actually

testing, right? All the rest have been

investigating, modeling. I've learned

things. I've I've found defects. I've

got notes. I've do this communication.

But this is the actual formal testing

session. And if I wasn't splitting all

of this into sessions, all the previous

work would be invisible, right? People

wouldn't know it was happening. Um there

would be less notes for it. It would

just be me doing stuff. Now the I

thought that the diagram that I created

in the modeling session was a little bit

weak to support a coverage session. So I

created a mind map to expand it. Did the

normal thing as I'm doing the coverage

session. I keep notes. Um, I actually

found missing items from my plan and

added them into my notes, which is what

you expect, right? Because we're doing

minimal planning.

As we go through the testing, we learn

more. We adjust our plan. Some things we

do immediately because they they feed

directly into the plan. Some things we

put off until later. But one of the key

things for a covered session is that I'm

trying to keep um additional exploration

to a minimum. So I'm trying to really

focus in on the coverage. I will still

be doing observation. I may do a lot of

interrogation, but I'm going to try and

not go too far outside the coverage

scope. So I'm I'm using the coverage as

a constraint to really zoom in on that

area. So this is my expanded plan. So, I

went from a very rough uh text model

into a a a hierarchy. Now, a lot of

people look at this and go, "Well, it's

a mind map." Well, it's a mindm tool,

but it's it's really a hierarchy.

So, I've got the context closed and I've

got the things I'm going to do for that

closed context. Now, my initial thought

was all these different action events I

would use during the state transition

process as well, going from closed to

open, but it didn't fit. So, you can see

that I've expanded my plan out so it's

very clear what I'm going to cover. It's

very clear that I'm not going to look at

dependencies at this point in time. So,

I can come back and look at that later.

Um, and I'm going to repeat some of the

testing for the open and the hidden. So

it's I mean it's I think it's clear

there what I'm going to do and it's a

very simple diagram but it's very very

tight and I can work through that for my

actual um coverage session. So as I make

notes um you can see that the headings

that come through are actually related

to the mind map. I'm just going through

mind map branch by branch and doing

testing and making notes as I go

through. And uh and I mean you'll see

the notes in more detail during the

debrief session. But I mean this having

this degree of coverage was useful

because it allowed me to explore that

relationship between the uh moving the

context state and the actions. So it

became very obvious that there is a

constraint that you can't close a

context when it has open actions. But

it's possible to add an open action to a

closed context. It's possible to have an

action on an open context and amend the

action so that it's moved to a closed

context. The constraint isn't enforced

in all the different combinations it

might expect. But having that model that

shows the combinations is what I was

exploring to allow me to see that in

real depth.

So, I've just finished an actual test

session. This is the first session

really where I've done like what you

might call proper testing. And because

what I did is I took the plan that I had

where I'd prioritized what I was going

to do and what was going to cover the

different states. Um, and because I'm

about to act on it, I expanded it out

into a form that I can follow. And I I'm

I'm calling this a coverage session

because I'm going to stick as much as

possible to the uh the model that I'm

using to drive this particular session

and it's a coverage model. So to

represent this, I put it in a mind map

and just expanded out the um states. So

this mind map here represents this. I've

got the context.closed closed. I want to

create an action from the homepage via

repeated action. And I've just expanded

these things out. And you can also see

that as I did it, um my initial thoughts

were I'll do the contact. Close the

different the action events. But uh when

I actually come plan it, I discover well

the open closed switch is really only

for um open actions or repeated actions.

And I can't really do it for um repeated

actions quite easily, but there are

repeated actions there. So I didn't have

to do it with the um action close

because you can't do you can't have this

multiple states happening at the same

time for a single user. It might be

possible to do that with multiple users,

but the timing could be quite hard. So I

didn't um try and explore that. So my

initial plan expanded

uh into this and it's not quite as large

as I was anticipating.

Remember I haven't covered all the

relationships that were in place on the

uh previous uh diagram covering all my

thoughts but it's a coverage session

because my aim is to cover this. I'm

going to work through each one of these

things and I'm still exploring because

I'm observing and I'm keep paying

attention to what happens. In fact, as I

was uh performing the testing, I noticed

there were opportunities for additional

testing that were not on this coverage

plan. So, I followed those up. I didn't

go too far because it's a coverage

session. And had I seen anything else I

want to do, I would have made notes. So,

here are my notes. You see with my notes

in this session, there's a lot more

evidence, right? I've taken a lot more

screenshots as I've gone through because

I'm actually testing. I actually want to

gather evidence as I go through. So, I

need tools that support me in that

evidence gathering in a way that is

fairly seamless. So, snag it, hit a

shortcut key, take screenshot, and paste

it in. I might use cloud app, take

screenshot. um

then it automatically creates a a URL

and I paste it in and then I can see the

uh screenshots afterwards. But because

I'm making my notes in Evernote, it's as

easy to copy it from Snag It as it is in

here. It also makes the notes pretty

simple and easy to read because you can

see where I've been, what I'm switching,

what I'm doing. Um so one of the

interesting things is I wasn't sure I

was actually going to be able to um test

the repeated actions. the repeated

actions are in the system. I go in and I

create a repeating to-do.

Then I have to add a new recurring

action and I choose the context. I want

to be in the closed one, but because you

have to set the occurrence and it starts

in the future. I wasn't sure this was

going to happen, but I experimented.

What I did is I set one into the past

and then uh when I actually went into

the the context, I saw that the repeated

action had been added. So I was able to

test the repeated action and repeated

actions were added into a closed context

as well. And then I just went through

the different combinations identified

making notes as I went as to which

context I was working with, taking the

screenshots to see the evidence. And all

of this will help me later on when I

review this. And the actual taking

evidence and making notes doesn't add an

awful lot more to the session time, but

it does add an awful lot more to the

authority that your session has when

you're debriefing or explaining or

showing it to people or uploading it

into the um tracking system that you're

using. All that extra um evidence just

adds more weight to the fact that you

are doing thorough testing. Um there

were a couple of new paths that I found

as I was going through because remember

the plan is not necessarily guaranteed

to be complete. It is a more expanded

plan than I had at the start. So I've

gone from something very basic to

something bigger. Um and then as I went

through I saw um paths that were new

because I hadn't considered uh

moving from the hidden to closed. Well,

I mean, I had in the initial um diagram,

but I forgot to add it into this plan.

So, I've added it there. I for I hadn't

realized that deferred might have an an

an option here cuz when you click the

drop down, there's an item to defer a

to-do. So, should I be able to defer a

to-do on a closed context uh and push it

off to tomorrow? So, I get reminder for

some context actually closed. Don't

know, but I tried that and I can. So I

will get a reminder tomorrow in contexts

that are closed. So um we've explored

fairly thoroughly

or have I? I don't know. But we can

certainly review the coverage to

determine whether it's thorough or not

and make a decision as a project. I feel

it's um fairly thorough. It's probably

more thorough than the actual

investigation warrants, but I don't have

anyone to ask a question to. And part of

the reason if this was a real project

and I was following uh an assumption

like this as far as I have done here,

this would probably be a mistake because

we might discover well no no it's it's

perfectly valid to um make all these

state changes on a a context that is

closed. The only thing we're stopping is

uh creating a new

context on a closed state. and you can

amend it and change it and do whatever

else you want to to get it on a closed

state. But all we've really decided is

when you create a new one, it can't be

on a closed state. That's what closed

context state means. And if that was the

answer, that would be uh fine. We'd live

with that as a project. If that was the

decision that the project has made and

that's how the story is being

implemented, we might flag that that

might be confusing to people and it

might not give the constraints they

want. But um the project decision is

kind of what we go along with and

there's no real bug here. It's not

crashing the system. It's not a massive

impact on anything that the user is

going to do. So we can let it ride. It's

just we're investigating it and I'm

pushing it further to try and see what I

learn about my test process. So my aim

is slightly different than as if if I

was on a a proper project. And I've made

notes of the additional coverage items

that weren't actually on the the the

real plan. So, this is probably the

first um real testing that we've done,

which is interesting, right? We spent

quite a lot of time, but I haven't

really done any testing up until this

point. And the testing also wasn't

particularly

um exploratory. I mean, I've planned it

three times. Um, I in advance or in a

model, then in advance in the

handwritten notes, then in a mind map,

and then I adjusted the plan as I went

through. But the whole notion of

exploratory testing is that we're um

planning and executing at the same time

and expanding the plan. What I have to

be careful about is when I find things

that are far off plan that I don't

necessarily follow them immediately.

Sometimes I might if it's looks really

serious or the risk looks high. But in

my notes, whenever I do that, I will

make a note that I've gone off track and

I'll put the timestamp in. If I have an

idea that looks like it would take me

off track and I don't want to follow it,

I will make a note in my notes for

something to do later on and pick that

up in a debrief. Um, and I've gathered a

lot of evidence because it's an actual

testing session. I think it's really key

in exploratory testing to spend more

time explaining what you did and having

the evidence for that than necessarily

planning in advance. But the planning in

advance is important when it's a

relatively complicated thing to get your

head around and you want to make sure

that you're going to get the coverage

that's going to help you. Um, and but

what I haven't done is I didn't make my

notes in my mind map, right? Some people

use mind maps, make their notes in it. I

don't tend to work that way um simply

because I like to have linear notes. I

like to keep things running um based on

the the time stamps and have it work

that way. Um, and at this point I still

haven't really done exploratory testing,

loose testing, following ideas as we're

going through, generating a lot of um,

paths based on the observations I'm

making in the system. So, we're still

early in the testing phase. If I jump

straight into an exploratory session, I

may not add value into the project

because the project has no data on what

it's expecting

the system to do. So sometimes we start

with the acceptance criteria. This is my

notion of the expanded form of the

acceptance criteria. When I'm

comfortable that this works or doesn't

work, then I might move more deeply into

more uh technical observation,

exploratory testing, and following the

the work through. But that's why this is

different. This is why that's why the

notion is it's a coverage session

because I'm covering a

predefined scope fairly tightly. Um, and

the so the only other tool I've really

added in here is the mindmap tool to

support me with visual planning so that

I can see I can think through in advance

what I'm doing. Clearly could be other

tools I could use to do that. And you

haven't seen a lot of formal test

techniques in here, but the what we're

doing is we're exploring um states,

combinations, paths. We're doing all of

that kind of inherently, but we're still

working essentially from an acceptance

criteria point of view. And this really

represents combinations of what I feel

are acceptance criteria for this story.

We go beyond the acceptance criteria

when we start doing exploratory testing

in more detail.

So, the notes for the coverage session

are pretty thorough. There's a lot more

notes. I gathered much more evidence

during testing than I do during recon.

But the recon's actually good practice

because then I know what tools I need to

support me during the actual testing

session. So, I'm ready with my

screenshot tool. I'm ready with a way of

using Evernote to to document it. Um,

and I tend to use Freeplane. A lot of

people use Xmind for their outlining or

mind mapping tool. I use Free Plane

because it is a built-in scripting

language. And I value flexibility and

um, supporting me in more ways than I do

necessarily aesthetics. XMind does not

have scripting capabilities. It does not

have an API. In order to use Xmind, I

would have to export it into a certain

file format, then write code that will

then read that file, parse it, and do

things, which I could do, but I don't

need any of that boilerplate stuff with

free plane because I'm working in the

model. I'm scripting it in the the

diagram itself. So, people prefer tools

for different reasons, but I I like free

plane or um free mind because it has

built-in scripting and is more flexible

for me. Now, plans may not be complete.

plans are unlikely to be complete

because we won't have thought through

everything. So, we have to be aware of

new opportunities to follow. Now, with a

coverage session, I follow them if they

are within the scope of the coverage. If

there are areas that I missed in the

planning, if it's

more out of scope idea, then I'm

unlikely to follow. I'm likely to add it

down as a note or a to-do or an idea and

feed it into a planning session via a

debrief. But what I do is I keep notes

of the extra things that I do. So it's

obvious in my notes for the debrief that

we expanded the plan. Because I plan in

advance, I know roughly what I want to

do. So I know what tooling I need. I I

didn't really need any extra tools for

this type of testing. But I did have to

bring in extra tools for to support my

planning. Like I brought in free plane

to support my planning. And over time

you'll add to the tooling on your

projects to support the actual processes

and approaches that you're taking. But a

coverage session is an exploratory

testing session that has a very tight

charter which is do this coverage with

less permission almost to to explore or

go off peace. It's very important that

we rein it in at that point.

So, I should really talk about issues a

little bit simply because if this is a

real project, um, I would be doing more

about the issues and the defects, but

you're not really seeing that here

because this is a very, uh, small

example. So, I've made notes, but I

haven't really followed them up. So I

have

seen in the system that there is

supposed to be the ability to move a

context into a closed state. There's a

constraint where we should not be able

to move a context into a closed state

when it has open actions. But I can um

add an action to a closed context. When

I amend the action, I cannot create an

action on a closed context, but the

drop-down shows me the closed context.

So perhaps it shouldn't. So there are

ways of I've got ways of bypassing the

constraint, and there are functionality

in the tool itself that lets me hit that

constraint more easily, right? I can try

to add an action under closed context.

If that constraint was really important,

perhaps that shouldn't even be an

option. So, these are the kind of things

that I would probably want to discuss or

raise on a real project. But I don't

know if it's a real issue here because

um it's an open source project. It works

fine. I don't know what their intent was

behind the requirement. But what I do

know is that on a real project, I would

not have pursued this as far, right?

because I used essentially that issue as

the reason why I did the state

transition testing, why the reason why I

combined it with the action in more

detail. I think that's an important

combination, but if it's a real project,

I might not have done that as early and

I might not have explored it as

thoroughly because we may not have felt

that that was an important uh

combination or relationship. So, if this

is an issue, when would it be found?

Right? So if it was a traditional

structured or process, we're writing

test cases, scripts or whatever else,

hopefully when we're analyzing whatever

requirements or documents and we're

writing our test cases and conditions,

hopefully we would have found it then.

Hopefully we would have asked questions

then we would have gained clarification.

The issue with that process is it might

take weeks or days or months to get to

that point. Very often those kind of

projects you would track all your

questions and then every couple of weeks

have a meeting to clarify questions but

you'd be working on those assumptions.

So you'd have a lot of waste and rework.

Um so the one of the issues with a a

very tight scripted process is that

these kind of problems can take a long

time to find. Someone has to go away get

the answer then there's a big discussion

in a meeting. Should we do this? Then

some has to bring in the designers and

the analysts and it can take a long time

to get this stuff fixed. But that QA and

clarification process is essentially an

an exploratory process asking questions

um thinking through possibilities.

That's an exploratory process. Um we do

exploration during um traditional

structured uh test case test scripting

type processes. It's just that there's

it's in advance and there's a lot of

scope for waste. If it was an agile

project, then hopefully we would have

found it during a a story grooming

session or when the story is pitched to

the team, we would ask questions about

it. Um, if not, then would it be found

in an acceptance criteria? Well,

probably not because uh we didn't ask

the question. So we don't necessarily

know that one of the acceptance criteria

is the closed context should not be

shown in the list of contacts when

creating an action unless someone

prompted that by thinking it through

maybe the product owner someone did an

exploratory thinking process a modeling

process to work out the implication of

what they're doing. But again it's an

exploratory process. This is one of the

keys to do in exploratory testing.

Understanding that exploratory testing

doesn't just mean when we're hands-on at

the keyboard. It's the whole notion of

exploring the models, the requirements,

the interactions, the processes, how

we're building things, the technology

we're using. All of this that is where

we explore and that is how we find

things early.

But we have to be working in projects

that believe that that is suitable. Now

ultimately projects make decisions about

defects, right? To me this seems like a

mismatch in streams. If we have a

constraint and it's too easy to bypass,

then it's almost a worthless constraint

cuz it it it only triggers at one point

uh or two points. When I try and create

an action on a closed one, it says no.

But I should just not give people the

option. When I try and move a state to

closed when it has open actions, that's

valid. But then why don't I have the

same thing when I try and change an

action to move on to a a closed context

and surely I should receive the same

constraint. We're not enforcing it

enough. The project would have to decide

whether it's the problem for me in terms

of the impact on the system reliability,

the robustness, the data integrity. None

of that matters. The system was not

built with this constraint in mind. So

this has been added later. So we might

not push hard on this. If the user

decides, look, I I just want to stop

moving a context to closed when it has

open actions, right? That's cuz I keep

doing that and it's wrong and it annoys

me, so I just want a constraint there.

Fine. I just don't want to be able to

accidentally add an action to a closed

context. So, we put that in. That's

fine, right? Projects make decisions

about what is a defect and what is not.

We have to be very careful when we're

advocating for stuff that we're actually

fighting the right battles. It's not an

easy thing to resolve, but all of this

requires an exploratory process to

investigate whether the things that we

found are actually defects or not. And

we do that as a team. We do that as a

project.

Okay, so let's talk a little bit about

tooling. I mean, I know when I do this

session as a conference talk, people are

going to be much more interested in

well, what tools did I use? Um, and I

know that I'm going to be disappointing

them because for me, the important thing

is what are all the processes? What are

the thinking approaches that we use?

tooling is just something that we use to

support um the testing. But what's key

about tooling is well, what actually

drives it. So for me, what drives my

choice of tooling is I need to be able

to observe the system. I need to be able

to see what's happening. I need to be

able to monitor and receive alerts.

That's what observation means for me. I

need to be able to interrogate, which

means stopping what I'm doing with

testing, then specifically targeting an

area of the system and interrogating the

data that's there or looking back

through the previous locks and

interrogating it for more information.

Observation is something that happens at

the same time as testing.

Interrogation is something I have to

pause my testing in order to go and do.

Manipulation is where I uh change the

data where I create inputs for the

system. Can I manipulate the uh ATP

traffic? So I'll create something at the

GUI um breakpoint the traffic change the

traffic to try and bypass the GUI

validation and see if the server can

handle it. Can I manipulate this at an

API level? Can I manipulate this at

database level?

needs tools in order to do more than

just work with the graphical user

interface. And then I have all the

tooling that I need for my admin

support. And there's very often more

tooling here than anywhere else. So, I

need um tools for my process like

tracking my tasks, recording my time,

gathering the evidence, taking

screenshots, where am I going to put

this, is it under version control, how

am I going to take my notes, how am I

going to um draw my models, all of these

um admin type aspects are are what's

going to drive my tooling. So

observation is seeing stuff in real time

where I'll be doing some testing with a

proxy open and I'll see the traffic. So

I'll see whether there's any status

codes that are not what I expect. That's

observing

um and seeing in real time which allows

me to take action to quickly catch

issues JavaScript errors appearing in

the console. Observation is what

supports monitoring. We we can't monitor

unless we can observe. Observation is

key to creating alerts. Uh if there's

any errors in a log or if one of the

servers goes down as we're testing, we

have something in the background using

the observation approach to monitor and

alert us. So interrogation is deep dives

after doing something. So perhaps I

observed something, then I pause and I

interrogate the observation in more

detail. So I mean it could be just

seconds after but it's still after the

testing process. Um it's more we get

more data when we interrogate than we do

when we're observing because when we're

observing we can only we're observing

like the guey and the traffic and a

bunch of other things. So there's a lot

of information there. But when we're

interrogating we're zooming in and we're

seeing all the headers. We would never

be able to monitor or observe those

manually. But we can interrogate them to

see all the headers on a request and see

if they're what we expect. Manipulation

is our ability to change stuff which is

inputting data, creating uh values,

deleting things, using default static

data, repeating requests, using the API,

working at the database, sshing into the

box, all of these manipulation aspects.

And that's why we need more tools and we

need permissions cuz technology will

limit us in terms of our observation,

interrogation and manipulation and so

will our knowledge. So will our skill

sets and these are the things that are

very easy to work on as we are testing

applications because we want to improve

our abilities to test the application.

Other people want us to improve our

abilities to test the application. So I

think you rarely receive push back if

you're training yourself on the job to

interrogate, observe and manipulate in

more detail and to learn the technology

that the application is using. Now

evidence gathering is absolutely key. So

we need a lot of tooling for this. Um

normally I put this into my kind of

modeling type of in my Morim model the

modeling observation reflection

interrogation and manipulation. That's

it's kind of the modeling that this is

but I pulled out as evidence gathering

because a lot of people may not

understand the modeling part or may see

that as part of a session whereas

evidence gathering is a process that we

use in parallel with our testing. So

we're recording the observations we

make. Now I was using Evernote for most

of the note takingaking but very often

I'll use that in a text editor. The PDF

that you have of my notes was created

using marked which is a markdown viewer.

Very often I'll use pandoc. And to

generate these things I use the

livescribe ager pen to automatically

create the diagrams in Evernote. Two

screen capture tools snag and cloud app.

Snag it mostly for the static images and

cloud app for the animated gifts. But I

the to there's a lot of tools there to

support because notetaking and evidence

gathering is so important in the actual

testing later on I use a proxy tool and

I pick a proxy tool that allows me to

save the sessions again for evidence

gathering to support later interrogation

support later looking at and seeing

whether I did the testing that I want. I

mean, if you only included tools for

evidence gathering, um you'll be in a a

good place to um support your

communication processes, you may not be

in a good place to really go deep in the

system. But if you only have tools, then

put the tools in your evidence

gathering. So in summary to that is what

drives tooling? Well, it's the needs of

testing. But we have to identify those

needs. So I identify it from models of

uh observation, interrogation,

manipulation, and modeling. Other people

will have different ways of identifying

their needs, but they'll probably end up

in the same position. So specifically

for the web, I can test blind, which

means I don't have any way to observe or

interrogate or I can add tooling. And

with the web, we have tools out the box

like the dev tools in the browser

itself, or proxy tools. Both dev tools

and proxy tools support observation and

interrogation. I prefer proxy tools

because it's a lot easier for me to

observe the DOM and the traffic at the

same time with a proxy tool because I

can use the dev tools for the DOM and

the proxy tools for the traffic. So I

have multiple observation points at the

same time. In terms of interrogation and

manipulation, um the proxy tools do

that, some of the dev tools do that,

some don't. So I gain consistency cross

browser by using the proxy tools for the

HTTP traffic. But one of the keys that

we have is that the technology that we

use will require different tools. So we

have our needs for testing but then the

technology is what really chooses the

specific tool that we use to implement

those needs.

Okay, so back to the testing. So this is

uh an exploratory session. Um the

coverage is essentially an exploratory

session. It's just it's very

constrained. So I haven't described that

as an exploratory session. Really all

our sessions are exploratory.

So here what I'm talking about is

something where I have much more free

reign. Um, and I'm calling that a

technical exploratory session because

it's really being driven by technology

aims, technology constraints.

So, having done a coverage session, I'm

then going to say, "All right, well,

let's look at this in a bit more detail

at a technology level. So when I'm

looking at the changing of the state in

the context, I want to look at the

traffic that's involved. Uh so a much

more lower level to see whether that

triggers anything. I don't know what's

in there yet. I haven't seen it. So I'm

giving myself permission to look more

deeply at the system and explore it and

push it as we see fit at the time. So

this is a a real true in quotes

exploratory session. I don't really know

how long we're going to spend. I don't

know how long it's going to take. I

don't really know what I'm going to find

out. I'm going to have to extract

coverage from this later on in a

debrief. I think that's the real

difference between an exploratory

session and a coverage session. Coverage

session, we come in knowing what

coverage we're going to try and achieve.

Then we might achieve a bit more or we

might descope some during the session.

So the debrief is there. But with a real

exploratory session, we may not really

know what we're going to cover. So we

have to pull that out retrospectively.

So it's kind of like a spectrum from um

complete coverage to

complete free reign and exploratory. And

the closer you are to free reign

exploratory, the more debriefing you're

going to have to do to pull out the

coverage to see what you've done.

So the reason that I'm calling this a

technical exploratory session is because

I'm working specifically a technical

level. An exploratory session might be

more about the functionality, the the

users, um, how the GUI works. But a

technical one is basically saying, I

want you to go below the covers. I want

and I need tools for this. I'm going to

manipulate a much lower level. It's

going to resend requests. I'm going to

do DOM manipulation to bypass it. Now,

the testing can be driven by the

technical observations, right? Like,

well, what happens if I change that? I

look at it in the DOM. I see that it's

got a constraint. What happens if I

bypass that constraint? What if I remove

that constraint? What if I add more

fields in? What if I do things that I

can't do at the GUI? This is all what I

mean by a technical exploratory session.

And tools are mandatory at this. I could

do exploratory sessions in the GUI

without many much tooling, but for a

technical exploratory session, I

absolutely have to have tooling. So the

tools I'm using are Firefox. I don't

often use Firefox, but I do use Firefox

when I'm working with proxies because um

I have fewer plugins. I don't use

Firefox as much. So, and it's easier for

me to configure through a proxy. That's

the only reason. Um I do have a a Chrome

uh plugin to help me remove uh

extensions. Let's have a look. This is

the

extensity extension which lets me switch

extensions on and off which is quite

handy with Chrome. But because with

Chrome I'm often logged into Google,

there's a little bit more noise than I

have with Firefox. And I use the

Foxyroxy plugin to configure it to use

that proxy. One of the things I also

liked about Firefox is that it didn't

use the system proxy. Chrome very often

uses the system proxy, i.e. uses the

system proxy. Firefox certainly didn't

used to care. Unfortunately, all the

browsers are getting closer and closer

together because they're all getting

starting to get based on Chromium.

I'm using the OWAP Zap proxy because

it's completely free. Therefore, anyone

can repeat these this type of work. Um,

it allows me to observe, interrogate,

and manipulate an HTTP level. I'm not

using any of the security tools, but

OWAP Zap Proxy also allows me to save

the sessions, which I can't do if I'm

using Burp. Um, I think you can save the

sessions if you use Charles. You have to

pay for Charles. So, I'm I've used OASP

Zap Proxy because it's completely open

source. For this particular example, I

use the Firefox dev tools to interrogate

and manipulate DOM level and the proxy

is for the HTTP level. Um, I used a text

editor to help me view XML and I used

Sublime Text simply because um, I've got

a license for that and it starts up

fast. Um, and every time I type in text

editor, I'm so used to typing sublime.

So that's why I used that. I could have

used Visual Studio um, code, the

lightweight Visual Studio one, which is

my normal default text editor for doing

work. But for ad hoc stuff, I end up

using Sublime. Snag it for taking

screenshots and Evernote for note

takingaking. Not not taking, not taking.

Evernote for note takingaking. So this

is what my screen pretty much looks like

as I'm doing the technical testing. So

I've got the browser on one side with

the DOM up so I can see that and observe

it with the proxy on the other side and

then other tools floating around as I

need them. Uh and it's a fairly good

working environment. I tend to have a a

large monitor. If I'm working

exclusively on the laptop, then I'll set

it to a higher resolution than normal to

give me more screen real estate. Having

multiple monitors is an absolute boon

when we're testing because we can have

things that we're observing, things that

we want to interrogate in other windows

and drop down to them later on, have

multiple things to just scan around

with. So my basic process was to work in

the GUI, observe the impact of what I

did in the GUI at an HTTP level. So I

can see instantly if JavaScript is being

used. So when I was doing the

autocomplete for contexts when creating

a new action, I could see instantly

through the proxy that it was using

JavaScript autocomp completion to send a

message every time I typed a key cuz I I

didn't know how that worked until I had

something that let me observe the

traffic. I interrogate the traffic after

doing something to see what it is to

gain more information to build a better

model of the system and how it works. I

might manipulate the traffic, so resend

it to repeat the same action but with

some variables changed because it's

faster doing that than it is doing it in

the GUI. If I find anything interesting,

then I'll try and repeat it in the GUI.

And the reason I do that is it's much

easier to justify the raising of a

defect when you can recreate it through

the GUI than it is at an HTTP level.

Now, what I find is it's very important

to keep an eye on the the time when

we're doing this because an exploratory

session, you can often get carried away

and get so engrossed in it that if you

don't keep an eye on it, you might spend

too long. Um, it's one of the reasons

why people like time boxing um and to

keep tight rain on the scope of the

testing. I didn't time box this, but

what I did is I kept timestamps as I was

going through and I could see the

progress of time. Essentially, I stopped

not because of time, but because I

essentially I ran out of value that I

thought I could add in this process. So,

that's a basic overview of how I did the

technical exploratory testing.

just finished uh an exploratory session

which I'm calling a technical

exploratory session which is a kind of

sub class of exploratory session. So an

exploratory session is we we have much

more free reign for what we're doing.

We'll set up aims and objectives for the

session, but we're really being driven

by what we observe, by what we learn at

the time whilst trying to keep in in

line with the focus that we've set. I'm

a technical exploratory session and I

have much many more tools open. I'm I'm

going to be driven much more by the um

technical information, the the HTTP

requests, the messages that I'm seeing

rather than the the GUI and the

functional flow. So, it's just a

different level of observation that

leads me to a progress uh based on

slightly different information. So, much

more technical technically informed

information. That's what I mean by a

technical exploratory session. So I'm

using Firefox and I'm using Firefox

because I don't really log into Firefox.

So there's less traffic when I use it in

combination with a proxy. Using Foxy

proxy plugin to configure Firefox to

point to the proxy. I'm using Zap as my

proxy because it's completely open

source and allows me to save this

session. Um which some of the tools uh

like Burpuite you have to pay for that

functionality. And Zap is crossplatform.

So when I understand Zap on one

platform, I can use it quite easily on

another. I'm using the dev tools to help

me with the uh DOM. So essentially I use

the proxy for all the HTTP stuff. I

don't jump back and forwards between the

network tab. I use the proxy for the

network. The proxy is observing,

interrogating, and manipulating HTTP

requests. The dev tools are for

interrogating, manipulating the HTML

DOM. So if I want to make changes and

come into a context

and then go okay so I don't want to

submit closed I want to submit close

then I'll come in here and manipulate it

at the uh DOM level but I'm much more

likely to do that at the uh HTTP level

first I come in here I go close and then

that lets me send to bypass the um kind

of normal gooey the messages that are

being sent through.

So, I'm also using Snagit for

screenshots. I used a text editor

because I wanted to check the results

and as an export and Sublime seemed like

the easiest editor that I had installed

to use that. It could have been any

editor. And I'm using Evernote for my

note takingaking as before.

Um, so this is what it pretty much looks

like when I'm testing. I've got my HTTP

traffic. I've got my browser and I've

got the dev tools open. These aren't

open all the time, right? Because my

basic technical approach is I'm I'm

working with this. And what's good about

the um HTTP view is I can see things

like well this uh

has code completion. This has name

completion. And I I didn't know whether

this was cached or whether it was done

by messages to the server. But I can see

that as I type this in, messages are

being sent to the server. So that I know

that these values are not cached. I know

that they're coming in or I'm assuming

that they're coming in. So the way I

check is by looking at the actual

message and looking at the response and

I can see what's coming back. So I can

see that the things shown in that are

not um cached. But interestingly this is

a JSON request. A lot of the requests

that are coming back through the XHR

requests in tracks are not JSON.

um complete uh JavaScript with HTML

which again is a slightly different

interesting. So we have two different

approaches to handling H XHR requests

which I find interesting and it helps

build up a better profile of the

application in my head.

So basically I'm working in the GUI. I'm

observing the impact on the HTTP traffic

so I can see whether it's JavaScript or

actual HTTP messages. I'll come in, I'll

interrogate these in more detail.

Basically means I stop doing what I'm

over here. Then I'll come into here and

choose a message. Interrogate it by

looking at it, looking at more detail.

And I've kind of paused my work to

understand a bit more what's going on.

If I want to manipulate one of these,

say I want to, so this is a post on

to-dos. So this is creating a to-do. If

I wanted to create another to-do, I

would come in here and manipulate the

request and send it through. So I'm

bypassing the guey to work at a much

more technical level and create it from

the HTTP requests. I did learn that when

I'm manipulating things in Zap, I really

want to work at the text level because

then I can control the ordering and the

actual values because I found when I was

working at the table view, I didn't

quite get the output messages I was

expecting because this is sorted under

um

alphabetical parameter name here I

think. And uh this one I thought I had

typed in the value but I hadn't saved it

so I didn't quite get the result I

expected. But working at text although

it's a little bit more clumsy guarantees

that I'm getting the request. So we've

always got issues when we not issues we

always got the potential for risk when

we use extra tools that the tool itself

might get in the way. And uh when I was

using Zap, I had to upgrade the version

of Zap here um cuz I hadn't used it for

a while, reinstantiate the certificates,

load them into Firefox um because the

certificate was out of date. So it took

me about 20 minutes just to get started

because I was using tools. The tools

give me more insight, but there's

overhead and potential risks every time

we use them.

And it's very important that because I'm

working at a low level here, I've got

way more information than um I had

before. And here I can see there's a

default context name. When I saw this, I

went, "What is what is that?" And I was

hunting through the admin interface to

try and find the default context. I

couldn't find it. This was the first one

that I set up. Um which I thought was in

fact, it's not even Yeah. So that was

the first one I set up, but it's been

renamed. So, the original name is

changing over time, but I don't know how

to adjust this. I experimented a little

bit creating a um to-do without a

context to see if it used default

context. It didn't. So, I don't know

what this is for. I've now got questions

about what these things are, how they

are used, what is the impact on the

system, is there a potential risk there

that I would have to explore in my

testing. I wouldn't pick this up from

the GUI and get more information at

technical level. So there's interesting

things there. I also have to keep an eye

on the time because um with a lot more

information that's a lot easier to find

way more more more more test ideas uh

some of which might take me off scope.

So I have to be very careful which ones

I pursue. So I have to keep an eye on

the time very carefully when I'm doing

testing at this low level simply because

there's so much more I could be doing,

so much more I could be progressing.

There's so much more information to read

and understand and pull together. and

I'm getting a much much richer model of

the application which is good because

then that leads on to more testing later

on. Um so I found some issues with Zap

in that it wasn't able to save the

sessions in all the folders I wanted. Um

it was a high level folder. I don't know

if that's because I'm working on a Mac.

I don't know if that's cuz the operating

system has just been upgraded. Um I

don't know. Um, I had to switch off the

HUD when I started and because it was

too much information and I wanted to

reduce the amount of information

overload as I was testing and I'm not

used to the HUD in Zap yet. So, it's a

whole new set of technology that if I

had it working, it would possibly

distract me. Like the HUD on Zap is

here.

Then, okay. then refresh.

And now I've got a whole bunch more

stuff here. And there's also a risk that

this impacts my testing. I last time I

tried HUD, it was in beta mode and it

crashed my browser. So I switched off.

I'm sure it doesn't crash the browser

anymore. But there's a risk. It's extra

tooling. And if it's extra stuff that I

don't need, then I'm going to switch it

off so that it doesn't get in my way.

Options. HUD.

Okay,

bye-bye. So, I'm going to switch that

off to be as clean as possible in my

system. I found uh interesting things

that the tracks XHR messages um always

respond with a 200 even if there's an

error. Um some of these things are post

200

response error, right? So, there's an

error in there, but the the code

response is a 200. So, I would probably

raise that as a bug. Um, because

it's it's not obvious to me in the GUI

that the message has come back as a 200,

but it's obvious to me in the HTTP view

that the message is a 200. And really,

we want status codes that trigger that.

But this is an internal API. This is not

an API that you would use to interact

with tracks and because you're expected

to use the JSON API, which is a formal

API with status quotes. So, but this

might also suggest that there's two

implementations. There's a guey API

implementation and there's a backend API

implementation. So, perhaps there's some

risk there that I need to explore and

look into. And I wouldn't know that

unless I was investigating this at a um

a uh HTTP level. And I see a whole bunch

of stuff in here that uh some of which I

followed up. But when I do a post here

in the request, I can see that there's a

method and this method is a put, but the

HTTP method is a post. So I explored

that a little bit to see um what

interesting things happened. And I'll

have a look at my notes in a second.

We'll see that. So let's have a look at

the notes because the notes that I take

are not really any different when I'm

working at this level. I've still got my

aims. I've still got um the tracking of

what I've actually done. And I've still

got the timestamps. Um, and I do record

in here the actual messages and the

content that I'm sending through. So

even though it's being tracked by the

HTTP proxy, I don't necessarily rely on

that, right? I rely on my notes cuz this

is what I'm focusing on. This was of

interest to me. There's a lot of HTTP

messages in here. If I don't have it in

here, chances are I didn't really use

that to inform my testing. So there may

be scope for reviewing this and seeing

things that happen later on.

And you can see in here I explored the

um underscore method being different

from the verb the HTTP verb. And I I

found it interesting that when I put in

a post with a post method, I got a 404

which suggests that this is being used

for rooting that I did not expect. So

I'm not entirely clear in my head how

the application hangs together and

what's important for rooting. So there

may be scope for exploring verb and

method combinations in here to see what

happens to see if we can bypass

validation to be put into different

parts of the app. Don't know. So again,

it's one of those things that I might

want to look at. Um you saw that when I

put through some messages and got an

error, we get a 200 and code and I made

a note of that in my notes. Um I tried

um putting in

context states that are not uh valid. So

I used a closed state instead of closed

and saw a 200 response come back um

telling me that the context has been

saved. But when I actually went to view

it and because I don't have database

access

um I my tools didn't allow me to observe

everything. So I used the application

itself as an export to allow me to view

the internals to see whether um the data

on the back end had changed even if the

GUI wasn't reporting that. So I've got

limitations in how far I can observe and

interrogate with the technology because

even here I'm assuming that the system

is not lying to me. I'm not looking

directly in the database to see whether

the states did change.

Um, but it's it was interesting to see

that we get a positive response back

when it was actually an error and when

nothing was actually changed, but it

said that it did save it. So, I think

there was more exploration possible in

there. But one of the interesting things

with exploratory testing is you because

you're so focused on doing and observing

and learning, you only really get a feel

for the coverage when you debrief. When

you step back and look at your notes,

you reverse engineer your coverage

rather than following it as you go

through in a in a real exploratory

session, not a real exploratory session,

in a session that is prioritizing

exploration,

then coverage is important, but it's not

the main driver, right? The main driver

is the aims that we have and we try and

keep on scope and the observations that

we're making. at least when I do it. Um,

if I really want coverage to be

important, then I stress coverage in my

aims. And it's coverage that's really

driving me with a, um, well, I still

think of it as a true exploratory

session. Um, coverage is something that

I reverse engineer. Yeah. So, I think

I've mentioned some of this stuff.

Essentially, um, I'm what I do is I work

here. I see the stuff in here. I

manipulate in here. Then if I manipulate

in here and see something that is

interesting, I will then try and see

whether I can recreate that manipulation

in here because it's a lot easier to

raise defects against things you can do

in the guey than uh just at an HTTP

level. I have raised defects just at an

HTTP level. When you say, "Hey, look, I

can send this HTTP request and buy this

product for 0." Then people take notice

of it regardless of whether you can do

that at the GUIN or not. So, but I do

like to move back to the GUI because it

makes it easier for other people to

recreate the bugs. If I say send through

this request, they have to get a proxy

up. They have to come into the GUI. If I

say come into the GUI and open the form,

inspect it, change the value to X, then

hit the submit, then you see the error

that is a lot easier for um programmers

to deal with. So, using tools themselves

have risk and impact. So, it took me a

little while to set up um the proxy

before I started. That impacted my

testing, but I I couldn't do the work

unless the tool was working. So, that

was a necessary um tradeoff and side

effect. I have to be very careful when

I'm doing this type of testing that I

don't get carried away because there's

way more information. There's way more

stuff in here. I'm learning an awful lot

of things. So, I have to keep an eye on

the uh time. Otherwise, these sessions

can take longer than they need to. And I

have to keep really sure that I'm still

keeping on track with value. Is this

technical investigation I'm about to do

going to add value? So, I I have to be

careful that I don't spend too long

investigating the mismatch between the

verbs and the method. Um, I just do

enough to see is this interesting for

future work. Is there an immediate

problem here? But I've got it in my

notes so I can revisit it. when I do a

debrief on this, then I'll come out with

all my to-dos and

things after that to know where to go.

Um, I try to duplicate bugs at the guey

as well as just at the HTTP level

because they're easier to take

seriously. And the automated recording

here is not my main uh record, right?

This is a useful source of evidence.

I'll save it. We can review it later on.

But my main evidence is coming from um

my logs in Evernote. Where is you? Where

has Evernote gone?

There. So this is my still my main log.

That's why I copy the messages in here

because I thought this was important

enough. It's useful to come in here and

review it. Um and as we because I've got

fairly detailed notes on what we did,

there's some time stamps. I could look

at the 1223, look at the time stamp in

here, and I can be fairly sure that

round about this point is what we're

talking about in my notes. So, I can

start working through my notes in

combination with this to see what I did,

but it's really an after the fact

record. It's not my main um information

source. So, that's probably all I should

cover with the uh debrief here. And uh

it's interesting. It's good insight. Uh

the most important insight for me was

the notion that when I'm um focused on

coverage, the exploration is constrained

within the coverage and observation and

reflection identifies new opportunities,

but these don't need to be followed

immediately. But it's still very

coverage focused. The more that I'm

emphasizing exploration, the more the

coverage is reverse engineered from my

notes because it's not apparent when I'm

working there necessarily what I'm

covering on because there's a flow of

consciousness. It's very hard to um talk

as you're doing exploratory testing

which is why I'm not doing that doing

that for this sessions because if I did

the sessions would be much longer. there

wouldn't be as much uh indicative

process for how I actually work as a

tester because I'd be talking and and

talking things through that would

distract from the way that I make my

notes. So, I've done the testing clean

and made my notes and debriefing

afterwards to talk about it that way as

though this is a real debrief sitting

with someone, but it's not a proper

debrief because I haven't collated out

the coverage yet. That was a a good

insight for me, the the difference

between the the coverage focus and the

exploratory focus and really hammers

home why we need to do debriefs, why we

need to come through and summarize our

work afterwards because we just lose

that sense of coverage when we're

actually working through it. Or at least

I do. Maybe your notes and approach may

be different.

Okay. So, I should try and go meta a

little bit on the uh technical

exploratory session. And so, here's the

kind of notes. Again, you'll see this uh

I've my normal note takingaking top to

bottom. It's linear. It's time based.

I'm putting timestamps in, but I'm also

adding quite a lot of information about

the HTTP request. I see no issue with

copy and pasting from my work into my

notes. That seems perfectly valid and

sensible thing to me to do. I don't

really care if my notes get too long.

The notes don't have to be uploaded and

stored in Jer. They're just text files,

so they're very very light. Um, there

will be uh screenshots. They can be

zipped up. There's a whole bunch of

different ways of doing this, but

anything that I create in my notes is

much lighter than if I was doing it in a

word document or if I was doing it um on

the system and adding it into a

database. If I've ever have to work with

Jira or any system like that, I never

ever um type into the system directly. I

always copy and paste from my notes

because there been too many times when

I've written something up, hit save, and

lost it. um any any online system,

Gmail, Patreon, LinkedIn, I never type

in the system itself, I will always copy

and paste from my notes. And that's just

Allan's handy paranoid

tip for dealing with the real world. Um

because as a tester, things crash under

our fingers that don't crash under

anyone else's. So, I tend to rely on

stable technology like text editor.

there's so little can go wrong with it

and copy and paste into something else.

And if the system says, "Hey, you're not

allowed to paste in here," then that's

fine. I will hijack the DOM and send it

in or write it into a clipboard buffer

and type it in automatically. I have

ways of automating anything. So, there's

my notes, right? I'm putting in quite a

lot of information in there. And the key

thing I I want to pull out is the the

URLs that I'm adding in my notes when I

convert them into PDF via Pandock,

they're clickable, right? I'm not

formatting. I'm not doing anything

special for formatting here. I'm relying

on the tools adding formatting later on.

These um slides that I'm doing are

written in uh just text and it's it's a

system a tool called MARP which converts

my markdown um text into slides. I don't

even format my slides, right? I don't

the the aesthetic part of life is not

something that I'm really focused on.

I'm much more focused on the content and

the value. So, what I found so I found a

potential bug in Zap that I wasn't

expecting, but I don't know whether

that's um because I've just upgraded the

Mac operating system to this new horror

that um doesn't that keeps asking for

permissions to save stuff and perhaps

Zap didn't. So I wasn't able to save to

subfolders. Don't know possible bugin

tracks because uh when we do things that

trigger errors um it's not an error

response that comes back. Everything has

a 200 status code and that itself is not

an issue but it's triggers for me the

thought process that perhaps tracks

trusts the guey more than it perhaps

should um which would make me nervous if

I was running this as a multi-user

system on a server because I would want

to make sure that people couldn't mess

with it. the XHR messages, the HTTP

messages sent by JavaScript, which are

error messages are still a 200 status

rather than a a 400

status. Um, and there's a lot of stuff

that I saw that I found really

interesting that I might want to explore

later, but didn't really explore in

detail now because there's h method

types in the forms. So when I post

something to tracks in the form, there's

an underscore method equals put in the

body of the message. And I'm wondering

how does that differ from the actual

HTTP verb. I just find it interesting.

It's a difference. It's the kind of

thing that I would want to explore on a

technical basis. When I'm using the

tracks guey, it's submitting forms like

post requests with you uh encoded form

posts. It's not using the API, but I

know that tracks has an API. So, I'm

wondering if the forms API is different

from the JSON API, the REST API. So,

could there be a mismatch there? I

haven't explored the API in any way, but

seeing this difference makes me think

that I need to explore the API around

this area. I was limited in what I could

observe because I didn't include any

database tools in order to check whether

the system was actually not changing

data cuz it was telling every time I

change something it was saying yes I've

made this change and it shouldn't have

been. So I wanted to double check that

it wasn't actually making the change.

Like when I was putting a context into

an invalid state, it would happily go

yes, I have made your change. But it

didn't actually persist that change. But

the way I checked that was by doing a an

XML export of the data, but I still

don't know whether that's in the

database or not. I assume the system is

accurately reporting what it's exporting

from the database and not from a cache,

but I can't check that myself. if I

can't convince myself completely of that

because I don't have any way of

accessing the database at this point in

time. The tools that I did use

automatically record an HTTP session

which is great. I've got all the logs,

all the messages that I sent. I can

easily replay HTTP messages and it's

easy for me to see the traffic

separately from the dev tools. So, I can

use the dev tools exclusively for DOM

inspection and I can use the proxy

exclusively for HTTP. in terms of

lessons like tools

tools can impose a risk right every tool

that we add imposes some sort of risk in

terms of time to learn maybe it crashes

maybe there's bugs in that tool I lost

time using Zap because I hadn't used it

for a while so I had to set up the

certificates had to make sure it was

working on Mac had to and try and get it

upgraded to the latest version so it

cost me some time um and that's the kind

of minimal impact the tool has just in

terms of starting when we're doing this

kind of technical sessions, we have more

information than we can handle and

investigate at that point in time. So,

there's much more risk of going off

charter um or or taking up too much

time. So, we have to be very careful

about that. One of the good things about

pairing on testing is that there's less

risk of going off charter and there's

less risk of going off time because you

got someone else watching that as well.

Some test ideas are not possible without

tooling. Right? I was able to submit a

form with multiple fields in it. That

would not be possible without tooling.

So I was really trying to stress the

back end. I can't possibly do that

without tooling. Um I do make sure that

I duplicate bugs at the GUI as well as

HTTP just to make it easier to justify

or advocate for them. And even though

the proxy is storing all the traffic, I

still copy HTTP request and URLs into my

notes because in my notes, they have

much more weight. I'm saying this

specific thing is important and I rely

on my notes as evidence. I rely on the

session recording in the HTB traffic as

a a useful side effect that may be

useful in the future. But my notes are

my essential evidence that I really want

to focus in on.

So what did I not do? Well, there was so

many things I didn't do. I mean, I

haven't used the API yet. There was I

haven't checked state transitions in

API. This is such a tiny tiny amount of

testing over like four hours. But things

that I really didn't do in here, I

didn't do any automating, right? If I

was doing automating, that would be said

planned separately, possibly probably

not even as an exploratory session

because automating is uh most usually a

strategic activity. Therefore, it's

likely to be a story level task that we

automate the acceptance criteria,

not likely to be um tracked as a

session. If it was tactical automating

to support the testing, then it would

probably be a session, right? Like a

tooling session where I knock up a

script to do X. That would be an

exploratory session that would get

tracked. But automating when it's

tactical is like when it's strategic is

likely to be factored into the project.

When I automate, I start tactically. So

I start with crude code. Something that

basically just works. It might need some

uh massaging to make possible. Like I

might have to debug it because there's

no synchronization in place to slow it

down. I might have to but I'm I'm

automating with support.

having something that works, I then

refactor it to make it reliable to

create abstraction layers that we can

use later on.

So I start very ad hoc and move into

more uh strategic approaches.

I think it's very important that

um we learn some tools well and just use

those as our default. Um, as we move

between projects, the default may

change, but what we want is tooling that

we can use as experts to get value out

of quickly. The tools should not be

slowing us down. They're there to help

us. So, I'm unlikely to automate in PHP

unless the entire project is PHP, in

which case it's strategic.

But if I'm automating on a PHP project

for myself for tactical reasons, I might

well do that in Java because I know that

better.

So I didn't show any of the admin stuff,

none of the admin sessions, none of the

uploading my notes into Jira, none of my

adding testing tasks, none of the here

are my test conditions I think I'm going

to use, adding them in as ideas into

this story. I didn't raise any defects.

I just had notes that I should raise

defects. I didn't have any processes

where I went and spoke to the developers

and asked clarifying questions. I didn't

communicate anything to anyone during

this. So,

and lessons that I've learned in the

past are to try and keep this as

lightweight as possible. Um, very often

I'll see teams who have they're on agile

projects, they're doing agile things,

but then suddenly they have test tools,

a test tracking system, a test log

system, an add-on for tool X that

supports the exploratory testing

logging. I try and avoid anything like

that, particularly on agile projects

because the only people that are ever

going to look at test in quotes tool are

people on the test side. So no one else

like the scrum masters, the product

owners, the programmers, none of them

are going to look at the testing stuff.

Um they will look at the testing stuff.

If it's in the normal tooling, if it's

just a task, if it's another story, then

people will look at it. So I just try

and avoid it because it adds that extra

resistance into the collaboration.

So, you've seen this all the way

through, but this is the basic testing

flow that I used, right? I'd start, I'd

plan. Even if it was a what am I going

to do next and then have an idea, oh, I

better install the software. That would

then lead to a session. I try to capture

things as sessions because I'm trying to

manage my time. So I want to make sure

that my aims are as clear as possible

and I can do them in 5 minutes, 10

minutes, 20 minutes, half an hour, an

hour, right? I want to have very tightly

constrained activities because very

often in a working environment you'll

get interrupted. I mean, when you look

through the notes, you'll see that even

in my testing, I was interrupted, but I

had a I just got interrupted time stamp

and then a I'm back at work time stamp.

So, we can still use our logging to

handle interruptions. Each of the

sessions either formally or deliberately

or informally will lead to a debrief

session like just a thinking about what

I did, taking stock, collecting it,

tidying it up, uploading it. And that

will then lead onto a more formal admin

session where I will track the time of I

now have to raise all these defects. I

now have to um we have a a mandatory

process where my logs get uploaded into

a jer and I will do that and I'll track

that time.

But the main sessions that I used were

planning sessions um an environment or

an installer setup session a health

check session where I'm making sure that

things are right before I move forward.

a recon session like a modeling session

to help me understand what I'm doing. A

coverage session where I'm constraining

my testing to a particular coverage

scope which is different from an

exploratory session which has much more

free reign. But then there's different

types of um exploratory sessions. And

remember these classifications are

arbitrary, right? They're me making this

up to get a better handle on what I did.

This is not what I expect you to um

communicate. This is not how I expect

you to model. But and you you do this

for yourself. I'm just trying to make

the distinctions that I thought were

important. I could call each one of

these just session and I would be fine.

But the important point is I know what

my objective is and I take stock

afterwards. And there's a difference

between the actual value work and the

admin work. Right? the admin work

changes based on the environment and the

process we're using. These other things

are almost context independent, right? I

will always have some form of coverage

session where I'm tightly constrained.

Um, but the admin will vary from project

to project, environment to environment,

site to site.

Um, so take this as a a rough flow. Mix

and match them. Do them as you want. Do

planning. do debrief to check if you're

planning, plan some more. Um, then come

in and do a coverage session, then jump

to an admin session and include your

debrief in the ad. I don't really mind,

but whatever you do, chunk your work

into small chunks and be very focused as

you do it.

Okay. So as an end summary section, I

just want to describe the testing in a

bit more detail, go over tooling, bugs,

explanation, coverage, just try and make

some um summary distinctions. Now, this

was interesting to me, right? Of the

four hours essentially that I spent

testing, only one and a half hours were

hands-on. Um, which and if I if this was

a real project, there'd be another two,

three, possibly four hours of admin

raising the defects, talking to people,

investigating things. So the the real

point here is that a lot of what we do

in testing goes unnoticed and for a lot

of people goes unttracked. I track it in

my daily notes all the time. A lot of

people do not track their planning

sessions. They do not mark down when

they start going to a meeting what was

covered. They don't track things. So

their time gets lost. I think it's

really really important that we we know

where our time goes that we track it in

order to be more effective with our use

of time but also to help other people

understand what we do and the more that

we can make our testing visible just the

less hassle you get about are you adding

value because you're demonstrating I'm

doing these activities and I'm doing

these activities because this is the

value that comes out of it. So I

encourage you that's what partly why I'm

chunking down the work into different

sessions because it justifies the

existence. We know the value of a

modeling session is to help us think

things through and plan it. So because a

lot of testing goes unnoticed and

unttracked. I encourage you to make it

noticeable and to make it tracked.

So bugs are interesting. I if I had

found no bugs in tracks, I would be

happy, right? Because this exercise for

me was about the process that I'm using.

But in the real world, um, bugs are a

visible output. And for a lot of people,

they're the most important visible

output. Where testing itself is a

process of deliberate exploration and

coverage,

finding bugs is is simply a side effect.

Right? We find a bug because we try and

cover an area that we thought existed

and it doesn't. Or we try and pursue a

particular coverage path, but we can't

finish it because there's a bug, right?

Bugs essentially are a side effect from

our testing, exploration, and coverage.

But bugs are where people put in the

time and attention, and they can soak up

a lot of time, right? In the uh

exploration, I I note down here's a

potential issue. If I have to go then go

and raise that as a bug that can take me

another half hour an hour and then it

goes through a formal process. So an

agile processes that's why we encourage

people after your session when you find

an issue go and speak to someone about

it and see whether you can pair on them

and get it fixed. Sometimes you can't do

that because you have to plan uh to make

space for that activity. Um, but uh if

you do perhaps it's possible to stick it

on the board as a bug and then a note of

your session. But the less formal we

make it the less time it takes. So but

some projects if you find a bug in order

to justify the time for doing it you

have to raise it into Jer and then we

have to plan it and discuss it. If

that's the process, that's fine. In

which case, it's very important that we

track that time so that people

understand how much of our time is spent

on admin as opposed to actually testing.

So tooling comes from the needs of

testing, right? Our need to observe,

interrogate, manipulate, um do the admin

modeling and evidence gathering

processes.

That also comes from the technology that

we're choosing. And the technology can

hinder the tooling. We might be using

such advanced technology that the

tooling isn't there. Uh we might be

using a brand spanking new JavaScript

framework that has no debug tools that

has no way of actually working with it

which can impact the testing.

Consequently, we then have to rely on

our technical knowledge for HTTP

JavaScript the DOM. it it becomes a

risk. If we don't have the tooling that

supports the technology, we should flag

that as a risk and and tag it as because

we're doing this and it makes it a lot

harder to test because I can't gather

the evidence. I can't see the memory

usage. I can't observe this. I can't

pause it. I can't interrogate all those

things. So, it's worth flagging these

things up if you're in that situation.

So, a couple points on coverage just

before we finish. Um, because I made the

point that there was a difference

between an exploration section and a

coverage section in my head. Right now,

at all points in our testing, at all

points in the activities that we use in

testing, we are exploring. We're

exploring when we're communicating, when

we're asking questions in a story, when

we're in meetings trying to clarify

things. Um, as we're doing our testing

or any activity, we're observing and

we're trying to process that and reflect

on it so we can identify new

opportunities to test or investigate or

to-dos or questions to ask. We do not

need to follow those immediately. We can

add them in our notes and do them later.

And we should add them in our notes

regardless if we're going to do them

immediately or later. But the main

difference is when we're focused on

coverage, we constrain our exploration

within that coverage. And it's a lot

easier to understand what we did. And

our debriefs are mainly there to go,

well, what else did we cover? What did

we miss in our initial coverage

assessment?

Um, and it might be, well, what simply

could we not do? What did we have to

descope? And we might have to do that

later. But when we're focused on

exploration,

the coverage is is much less evident. It

has to be reverse engineered from our

logs, from an from our evidence, from

our notes because we are so focused on

the thinking process, the doing, the

working out what we're going to do next,

the modeling in our head, the working

out, well, what does that mean? What

should I do? All the questions. Um, we

don't track a summary as we are doing

our our exploration and testing. or I

don't right some people when they're

using mindm map tools um try and

organize their testing as they go

through into chunks and coverage areas I

don't do that because I'm a very linear

uh timebound

test approach and I will reverse

engineer that later I mean it's like

when I'm writing I don't want to edit as

I'm writing because I want to

concentrate on the flow and a lot of the

material that I create is first draft as

a result of that I'm focused on the

flow. Then I do a possibly minor um

spelling error. But very often I'll just

release it and if they see a spelling

error in live, I'll fix it in live. But

I don't edit as I'm writing because I'm

interrupting my flow. I'm interrupting

my modeling. I haven't got a handle in

what I'm doing. I'll go back later to

edit it. See if it covers everything

that I need to to add summary points. Um

I think that's the key point. Um

coverage. It's well defined. We

constrain ourselves. Exploration. We

have to be careful that we don't go too

far off scope, that we don't spend too

much time, but we have to do a debrief

to work out what we actually did.

That's pretty much it. Because this is a

case study, I don't really anticipate

adding more content to this. Although in

theory, I could do a a live session. The

the hard the reason I did it this way is

that it's very hard to test and describe

what you're doing at the same time. What

you get is a a slightly less

uh representative test session. So every

test session that I've done as a demo is

not completely representative of the

work that I do because I don't take as

many notes um because I'm communicating

and and rather than communicating via

notes, I'm communicating verbally. Um I

I tried to avoid doing that in this

exercise so that people can see the

exact things that I do, the depth to

which I make notes, the the amount of

questions that I add in the the

documentation. It's really important to

see that. Um and what I'm hoping you'll

do is you'll look you'll watch some of

these videos. You may not watch them all

if you got to this point. Maybe you

skipped them all. Who knows? But my hope

is that you'll look at the notes that I

made and look at the kind of notes that

you make and go right should I be making

more notes? Can I make more notes? How

is it possible to add more note

takingaking in? You might look at my

notes and go these are rubbish. There is

no depth or um these these notes just

aren't covered enough. Which is great.

Um, what I would like is if you add

questions or you add uh feedback in

here. Uh, possibly sharing your own

examples of how you test, writing blog

posts about how you test, putting the

links in here. That would be great cuz

what the world needs are more examples

of how people test and why they test

that way, what drove them to test in

that way, what made them take notes in

that form. Right? The more that we

analyze our own approaches, the more we

can um explain the benefits of doing

that and the more we take ownership of

our own test approach because I am

absolutely convinced every single person

has a unique approach to testing. And if

we do not learn to harness that, we are

um not taking full advantage of our

skill sets and abilities. So, my aim for

for putting all this together is to help

me get better, to try and make

distinctions in my thought processes for

how I test, but to not get hung up on

them. I'm not going to get hung up on

the notion that I have to plan, then do

recon, then debrief. My language will

change over time, right? My approach to

testing will change over time. what I

think is unlikely to change over time is

my focus on

um making sure I know what I'm doing in

advance. Planning, doing stuff and

capturing evidence in small chunks,

whatever that session is, looking at

what I've just done to reflect on it to

figure out what I'm going to do next and

what I've learned and what my modeling

is, right? debrief, then communicating

to other people, fitting in with the

project, the admin stuff. Those things

are unlikely to change. Um, so I think

this is a relatively good generic model,

but don't get hung up on it. Don't think

you need to do these sessions. Don't

think they need to be the same length

that I did. You need to model your own

processes. And I'm suggesting doing it

this way, like not not the actual

sessions, but thinking it through in a

session time in order to help you better

understand what you do. The more you

approach your work in small chunks, the

more you can look at why have I chosen

to do this now? What is this that I'm

doing? If it's small enough, you can

probably give it a name. Otherwise, it

just becomes testing. And testing is a a

good word, but it's so generic that it

really doesn't explain what we do. And

it's really hard to um convince people

that we're adding value when we just

say, "I'm doing testing." But if you

say, "I'm doing a health check," then

I'm debriefing to see what I need to do

next. I'm just doing some recon so that

I understand what it is I'm going to

test. I'm just modeling the system so I

can understand what coverage I'm going

to achieve. I'm just doing a coverage.

Now, all of these things help us explain

a bit better and hopefully when you do

your modeling, you'll come out with a

completely different set of words that

are unique to you that may add more

value to the external world than

anything like this does. So, thanks very

much for um spending the time going

through it. I found this useful exercise

putting together. I hope there was

something useful in there for you. Do

leave comments and ask questions and uh

thanks very much.