Whilst there are a wide range of medical classifications of neurodivergences, learning disabilities and impairments, and an incredibly broad range of abilities and preferences that need to be considered, WebAIM claims that functional classification is more useful as it ‘focuses on the user’s abilities and challenges, irrespective of their medical or behavioural causes’. (WebAIM 2020)
WebAIM groups these abilities and challenges into:
Memory
Problem-solving
Attention
Reading, linguistic, and verbal comprehension
Math comprehension
Visual comprehension
The first four of these are particularly relevant to information management in my context. From a different angle, but looking at the same things, the Neurodiversity Design System, Will Soward has tried to create a ‘coherent set of standards and principles that combine neurodiversity and user experience design for Learning Management Systems.’ (Soward, no date). These principles include:
Font
Typography
Colour
Buttons, links, inputs
Interface
Communications
Animations
The key items for me here are typography – use of headings, use of colour, buttons and links, clarity of the interface and communications.
Now I need to work out how to take this information and make it into something that works in the focus group. Essentially the barriers that I’m thinking of are of course all intertwined but can broadly fall into these categories:
Cognitive load:
Minimising the cognitive load needed to access the information on Moodle will benefit everyone, not only those who are neurodivergent or disabled. In the Neurodiversity Design System, Soward points out that ‘an LMS already presents learning content that has a high cognitive cost, having learners begin using a complicated system before the real learning begins creates cognitive fatigue and puts learners on the back foot…’ (Soward, no date).
Actions to minimise the cognitive load include:
Using a sans serif font which can be enlarged and colour combinations changed
Incorporation of white space and use of indents to indicate hierarchy
Using true headings (headings with background metadata, not simply large and bold), to indicate clearly the layout of content and reduce cognitive effort
Contextual information/instructions in plain English ‘concise, empathetic instruction that sounds human and addresses the person’ (Soward, no date). This should be sufficient to negate the need for substantial onboarding. The LMS should be intuitive.
Offering clear, simple instruction with the place of action e.g. next to a tutorial booking activity or assignment submission. People with dyspraxia often find it difficult to remember instructions or retain information from looking at one place to another. Offering instructions in the appropriate place can help with this.
Pairing graphics and text where possible to support dyspraxic or dyslexic learners who may benefit from a more visual representation of information.
‘identify and integrate information in to meaningful chunks’ webAIM
Use of colour to create a uniformity across the site and help with predictability and consistency as well as aesthetics. (Always with the proviso that contrast must be maintained and colour not used as the only source of meaning.)
Appearance:
Interface – how clear is the interface? It should be visually appealing without distraction or clutter. This is particularly for learners with ADHD and dyspraxia who may find it challenging to follow processes and patterns.
Navigation – having a clear system of navigation so that forwards and backwards arrows are labelled and ‘breadcrumbs’ are obvious. According to WedAIM, ‘maintaining consistency in design and presentation minimises memory requirements’ which can be an issue particularly for neurodiverse learners but also for anyone suffering from fatigue, illness, stress.
Functional accessibility:
One of the key issues for many people can be a frustration or relatively low resilience for issues with technology. This can me more prevalent if the cognitive effort is already high and then links don’t work or they don’t take you to where you expected to go, leading to frustration.
Regularly check links are active and up to date and that the page(s) work properly.
Provide a mechanism to get help quickly, for example who can help, an email or phone number or Chat service.
Buttons and links should be visually distinctive and large enough to click particularly for those who have low vision, dyslexia, ADHD or dyspraxia. For students navigating via a keyboard, it’s important that he ‘pseudo’ or temporary states of interactive elements are present e.g. changing colour when selected.
Ensure captions and transcripts are provided for visual elements to support users who are deaf, hearing impaired, or have English as an additional language. This will also support users who need to work/listen in a quiet environment (e.g. a library), or who have less private study space.
Ensure the page is set up with the appropriate structure and metadata to make it readable by a screen reader.
Conclusion
This reading has given me reassurance that the types of considerations we think about and try to put in place are appropriate, and that there aren’t huge new areas that are missing. The information about dyspraxia and the need for large enough spaces to click, plus the temporary states being present were new elements for me. It was interesting to see the explicit mention of instructions, partly to support dyspraxic learners, and also the contextual information to negate the need for substantial onboarding. This isn’t generally my experience, but I’m looking forward to seeing how the participants respond to these prompts and to what extent these elements are useful to them.
I’m really enjoying the process of trying to work out how to ‘do’ thematic analysis of my data, but it’s also very clear that I’m barely scratching the surface of what I’d need to know, do and understand to be able to do this in a really meaningful way. I am doing my best however, to be diligent and thoughtful and conduct the analysis in the best way I can, given the time constraints and my starting point.
I’m primarily focussing on Braun and Clarke’s approach to thematic analysis because it offers the flexibility to be able to incorporate multiple types of data and, as far as I understand, it isn’t attached to a particular theoretical model. It therefore allows me to work out where I am in relation to various criteria as I go along and try to make sense of my data. Using the slide below from Braun and Clarke’s lecture shown below as a framework, I understand my data coding to be primarily inductive in that I didn’t know what the participants would say from the start and I’m basing my codes on their contributions. However, I don’t think it’s purely inductive as inherent in my research question is the pre-conceived idea of what our Moodle template actually consists of, or its defining features. Therefore my coding is inevitably influenced by my research question i.e. to what extent these features offer support to students and staff who might otherwise experience barriers.
Slide from Braun and Clarke 2018
In addition, I believe I am taking an experiential orientation to the data in that my data is very deliberately based on the experience of participants (I started my interviews with a request to talk through what participants do on Moodle and what barriers or supports they encountered. This was followed by prompts, which were based on certain areas I was interested to discuss, but participants had a free rein to select the topics and avoid others.)
Familiarisation with the data:
My data collection techniques were: semi-structured interviews, a short questionnaire (mostly quantitative but with one free text box) and drawings of participants’ experience of using Moodle. Three participants gave me drawing and one gave me a ‘voice drawing’ an audio description of what it’s like to use Moodle from her perspective.
Participant’s drawing of their experience of using Moodle. It shows a process of starting with lectures or coursework, then being directed to Moodle, back to coursework, which is clear and then Moodle again. The black scribbles represent time spent on Moodle.Screenshot from my Miro board showing collation of data: The first frame shows each participant with colour coded sticky notes. On each sticky note is a pertinent quote or opinion expressed. This is my equivalent of highlighting parts of a transcript. In the second block, there are screenshots from the answers to the questionnaire, as well as the free text comments on sticky notes. Summaries of responses to the questionnaire, plus free text comments on sticky notes.
Coding:
My approach to coding is semantic as opposed to latent. Again however, I believe this isn’t cut and dried. As Braun and Clarke (2018) point out, these are all spectrums and I think I am somewhere between the two. My codes and categories are largely descriptive and meaning based (semantic end of the spectrum) but they are also partially formed (through the interview process) and informed by my existing knowledge of digital accessibility, inclusive practice and my research question, which I believe pushes my approach somewhat towards a latent coding approach.
Fryer describes the contrast between data-led and theory-led coding: ‘(data-led coding) can have the benefit of allowing the data to surprise us, to challenge our preconceptions, or to move beyond previous theorising—whereas theory-led coding has more of a tendency to encourage us to find what we seek.’ (Fryer, 2022). However, like Braun and Clarke, he says is it not a binary choice and an element of both approaches is almost inevitable.
In my process, it took me two or more attempts to establish the codes I wanted to use. The process of categorising (Lochmiller 2021) involved more code development. The image below shows part of my coding process using the platform Miro. Participants are given different coloured sticky notes as identifiers and codes are added as coloured tags. In most cases, related codes are given the same colour. E.g. Contextual information: usefulness, Contextual information: presentation.
The reason I used Miro for this purpose was to keep the data as movable items (individual sticky notes), but also because Miro will cluster items according to tags. In the screenshot below, each sticky note has been grouped according to the tag ascribed. At this point it’s possible to see if the codes include comments from more than one participant (by looking at the colour coded sticky notes), and also if there are codes which are too large or too small and should perhaps be split or combined. At this particular stage, I could also separate out the data I had yet to code (the ‘no tag’ group) and so I could add these to the existing groups.
In adding the additional data, I made some amendments to the codes. For example, a new code was introduced – site layout – and Finding what you need was divided up, along with appearance. Within the new code Site layout, there were some sub groups as shown below. Later, some of these data would be moved again, during the analysis phase.
This is the point where the categories came in, shown below in white boxes next to the groups of codes. This process helped me to manage the data and at the same to revisit it and check the coding and groupings. I changed the categories several times and tried to consistently refer back to the research question, shown on the top left in a grey box. At this point, I also added the drawings which I think best illustrated the points made in the category.
Generating and reviewing themes:
The process of analysis is obviously a thread through the different stages of TA, but the process of generating themes is one I found very difficult. Braun and Clarke are very clear that themes do not ’emerge’ and they have have moved away from describing the process are ‘searching’ for themes, as they specifically emphasis the active role of the researcher and as such have chosen the more active term ‘generating’ themes (Braun and Clarke, 2018). I like this distinction although it also makes me slightly uneasy as it’s a new way of looking at data from research for me.
Third attempt at generating themes. Pink sticky notes show previous attempt, based on criteria set by research question and data. The blue sticky notes show latest attempt, rooted more firmly in the data. The research question is in the centre with notes about definitions and there are further notes from Lochmiller and Braun and Clarke with prompts about generating themes.
In working out what the themes should be, I found this description helpful; ‘Patton‟s (1990) dual criteria for judging categories – internal homogeneity and external heterogeneity – are worth considering here. Data within themes should cohere together meaningfully, while there should be clear and identifiable distinctions between themes.’ (p. 20 Braun and Clarke, 2006)
Eventually I managed to generate 4 themes which I think are both distinct from each other and individually bring together different codes in a meaningful way. For a full description of the themes and the final report, please see Project findings: report and summary.
Reflections:
In doing this again, I would do the process of transcribing differently. At the start I listened to the interviews and wrote notes directly on the sticky notes. I didn’t realise that at this stage I was effectively transcribing and I felt it was ‘cheating’ to write actual quotes on the sticky notes as I listened to the interviews. However, it because clear that this was actually wrong and what I needed was a full transcription to be able to see some of the quotes I did have in context without having to listen over and over, and also the notes were not useful, I needed them all to be actual quotes.
As a result in the analysis stage, I had to keep returning to listen to the interviews, understand the context and also check the accuracy of the quote. This did mean I was very familiar with it by the end but it wasn’t efficient in terms of timing.
As I said at the start, I need to understand more about the context of thematic analysis but I found it a useful and enjoyable (up to a point…) process.
Bibliography:
Braun, V., and Clarke, V. (2006) ‘Using Thematic Analysis in Psychology’. Qualitative Research in Psychology 3, no. 2 (January 2006) pp.77–101 Available at: https://doi.org/10.1191/1478088706qp063oa.
Braun V. and Clarke V. (2018) Thematic analysis: An introduction. June 25 2018 Available at: https://www.youtube.com/watch?v=5zFcC10vOVY (Accessed: 29 December 2023)
Clarke V. (2017) What is thematic analysis? Dec 9 2017. Available at: https://www.youtube.com/watch?v=4voVhTiVydc. (Accessed 27 December 2023)
Fryer, Tom. ‘A Critical Realist Approach to Thematic Analysis: Producing Causal Explanations’. Journal of Critical Realism 21, no. 4 (8 August 2022): 365–84. https://doi.org/10.1080/14767430.2022.2076776.
Friese S, Soratto J, Pires D, 2018 Carrying out a computer-aided thematic content analysis with ATLAS.ti MMG Working Paper 18-02
Nickerson, C. 2023, Social Constructionism Theory: Definition And Examples. Available at: https://www.simplypsychology.org/social-constructionism.html (Accessed 29 December 2023)
So, I’ve just been reading about neurodiversity, disability and user experience design and Learning Management Systems e.g. Moodle. It’s interesting to get myself out of ‘tunnel vision’ about the range of features I feel are designed to make our Moodle design more inclusive, and to think about this more broadly and from different perspectives. This is helping to inform and expand the items I want to ask about in the questionnaire and focus group. One of the students responded to my email delightfully saying they were looking forward to seeing what I ask them about – now I feel a bit more pressure to make that interesting! Especially since they are design students.
I’m thinking, in terms of structure for the session, the first half I want to be about their experience, what they do and what are enablers and what are barriers about the Moodle pages. In line with ‘Standpoint theory’, I want to create some significant space for the participants to shape the discussion and identify the highlight the issues.
‘Standpoint theory views members of underrepresented or marginalized groups as epistemically privileged because they are able to see more clearly the institutional structures and systems that marginalize them.’ (Parson, 2005 p.23).
However, as we progress through the session I want to ask some more detailed questions, or actually to provide prompts for the students to pick up and discuss. This is partly because there may be other aspects of the design that students are not particularly aware of or which they may feel uncomfortable about bringing up, thinking they are too minor or because they don’t want to give criticism.
Parson argues that ‘If seeking to identify how an individual or group of individuals is challenged, research questions should guide the exploration of the experiences and challenges of the groups whose experiences we would like to improve.’ (Parson 2019, p. 22). I’m hoping that the use of prompts will allow sufficient scope for the participants to have this agency, without putting all the responsibility of the session on them.
Focus groups and semi-structured interviews
Although these are different research methods, I’ve put them together because there are some overlaps in the way I want to approach them, and I need to design a version of each that fulfils the same goals.
Whilst ideally I’d like to see what happens in the more open focus group structure, I also recognise that not everyone wants to discuss their neurodivergence or disability with classmates or peers and so I have offered 1:1 interviews as an alternative. In addition, as part of my contingency planning I’m also opening up the project to staff and 1:1 interviews seem like a more appropriate format for speaking to staff.
I am struggling to structure my interviews and focus group though, which is quite unusual for me. This is a part of planning that I usually really enjoy, thinking about the aims of the session and the different ways you might go about getting there. I think one of the reasons I’m struggling is that I want there to be space for the participants to talk, and if they do, that will be fine, I think I’ll be able to follow their lead. What I’m struggling with seems to be the structure, what are the questions that I can ask about this specific topic that aren’t so micro that they stall the conversation (e.g. to what extent is the size of these buttons helpful…? :)), or too personal (e.g. what affect does your neurodivergence have on your experience of Moodle?)
I am, however, reassured by Adams’ view, ‘Once developed, the interview guide, no matter how extensive its preparation, should still be considered a work in progress’. (Adams 2015 p.499). He is very clear that the structure of interviews and also focus groups, can and should change and be reviewed as the sessions progress. I had been worried that somehow this needed to be the same (a bit like a job interview), which would make it very unnatural and I think difficult to be in.
So, the upshot is that I am hoping to provide prompts on individual pieces of paper so that participants can select the different things they want to talk about. This should work either in a focus group situation or in a 1:1.
Another aspect of the session which I think is important is about creating a welcoming atmosphere in room and amongst participants and myself. This has always been something that’s important to me, from being a teacher, and I would like to continue it here. I will ensure that the room is set up to try and suggest an equal footing between us all (no teacher chair at the head of the room). If we are in a focus group scenario, I will be using individual, pair and group work to try and maximise the opportunity for those who are from less dominant social groups to participate fully and in the way they wish.
I’m interested in how Maha Bali and Mia Zamora have incorporated this apparently simple idea into a much wider and deeper concept of Intentional Equitable Hospitality (IEH) in an educational context. They say, ‘IEH begins with the notion that the teacher or workshop facilitator is a “host” of a space, responsible for hospitality, and welcoming others into that space.’ (Bali and Zamora 2022)
Whilst this is an apparently simple idea, it goes much deeper, ‘(IEH) prioritizes the values of social justice while fostering learner/participant agency within the learning space, while never forgetting the ways in which power and oppression work outside of that learning space, and how they influence it.’ (Bali and Zamora 2022)
A change of plan… semi-structured interviews
Since writing the first part of this post, it turns out that my plan for a focus group is likely to be replaced by 1:1 semi-structured interviews. Whilst no-one has explicitly said that a focus group would be uncomfortable, the practical considerations of trying to arrange to meet participants has meant that I will largely be conducting semi-structured interviews. I’ve kept the broad structure and principles of my plan, but amended it for the different scenario.
In the video below, I’ve tried to explain the rationale for my design:
Designing the online questionnaire:
In designing this form, I knew I wanted it to be short (easily completable within 5mins), and mostly tick boxes. I also wanted to ask about neurodiversity and/or disability but I was unsure about the wording for this question.
When considering who my participants might be, I asked for some advice from Carys Kennedy, who used to manage the disability service and she very helpfully sent me a sample question for how she asked about disability when conducting online feedback. I made some tweaks to this to suit my needs, but this was a great start for me to feel confident about the appropriate wording for this question. I had included an ‘Other’ box but changed it for ‘Prefer not to say’ and ‘None of the above’, with the free text box instead, after feedback from Carmen Fernandez, Assistive Technology Coordinator.
Question about neurodiversity and disability from the online questionnaire.
Carmen also helped me to see more clearly that I needed to remove a section at the start which attempted to explain why I needed the name of the participant and also their neurodiversity/disability. Inadvertently I was implying that there would somehow be a connection between what they said and their neurodivergence more broadly, even though I was actively trying to avoid saying this. It was unclear and I removed it.
I was concerned that this introductory text was too long, but after talking to my tutor group, I was reassured that it seemed ok and keeping it there was part of due diligence on my part. One of my classmates recommended changing the colour to dark green, which she said was easier on the eye. She also said that the use of tick boxes made it seem not too onerous. Mostly, from my tutor group, I was reassured that repeating topics in the questionnaire and in the focus group/interview was actually fine, and would give some valuable quantitive data and a different medium for participants to share their thoughts.
Excerpt from introductory text.
In writing the online questionnaire, I was concerned that my focus would either be too narrow (focussing on specific elements of design) or too broad, when probably the participants would not have considered their use of Moodle in detail and might not know how to answer. I read some more about UX design, specifically for neurodivergent people (see UX design for neurodiverse, visually impaired and disabled users post) and with more feedback from Carmen I was able to clarify what I wanted to ask, and to feel more confident about doing so.
Adams, W. ‘Conducting Semi‐Structured Interviews’ In Handbook of Practical Program Evaluation, edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 1st ed., 492–505. Wiley, 2015. Available at: https://doi.org/10.1002/9781119171386.ch19. (Accessed 22 November 2023)
Bali, M. and Zamora, M. (2022) ‘2: Intentionally Equitable Hospitality as Critical Instructional Design’ in Designing for Care, Pressbooks Available at: https://pressbooks.pub/designingforcare/chapter/intentionally-equitable-hospitality-as-critical-instructional-design/ (Accessed 7 January 2024)
Parson, L (2019) ‘Chapter 2: Considering Positionality: The Ethics of Conducting Research with Marginalized Groups’ in Strunk and Locke, eds. Research Methods for Social Justice and Equity in Education. Cham: Springer International Publishing pp. 15-32
I’ve found working out how to best plan and organise my notes, research, blog posts and schedules for ARP to be a bit of a task in itself. However, I think I’ve developed something that works for me. I use Trello as a scheduling tool, a mind map on Miro to organise my resources and OneDrive to keep track of documents. I also use Zotero for annotating secondary research.
In the short video below I’ve given a quick tour of how I use these tools:
Reflections: Planning, managing and organising my ARP
Mind map of notes on ethics guidelines from BERA, UKRI, and gov.uk
Introduction
I’ve focused my reading about ethics on the ‘Responsibility to participants’ as outlined in BERA’s (British Educational Research Foundation) ‘Ethical Guidelines for Educational Research’. I’ve also looked at the UK Research and Innovation (UKRI) guidance and read more broadly about ethical considerations in participatory research and in research which involves marginalised groups. I found that much of this reading links to both ethics and the practices and process which surround the research and also the research methods themselves, in choices of approach, facilitation and contingency planning. In this blog post I will look at the guidance and what it means more broadly and then look in more detail at my own ARP and additional considerations.
Ethics: Responsibility to participants
Consent
Right to withdraw
Privacy and data storage
Disclosure
Harm arising from participation in research
Transparency
Incentives
Consent:
‘In order to achieve genuine agreement to participation, researchers have suggested instead that consent should not be limited to a point in time at which a form is signed. Rather, an interpersonal process should take place in which interactions and negotiations will lead to a meaningful informed consent.’
Guillemin and Gillam (2004) quoted by Padan (2022)
The guidelines are very clear that voluntary, informed consent must be obtained at the start of a study, ideally prior to the research taking place, and most often in a written, formal format. The UKRI guidance clearly outlines what information the participants need in order to give their consent, which they describe as, ‘giving sufficient and appropriate information about the research, to allow participants to make a meaningful choice about whether or not to take part’.
The information for participants should include:
Name and status of the researcher
Brief rationale for the project (what is the purpose and value?)
An account of why the individual is being asked to take part
What will happen to the data
How the data will be shared
Whether they will be identified (ask for preferences)
In addition to the consent from participants, it’s also necessary to gain permission from the gatekeepers, which in my case are the Course leader and Stage leaders responsible for the students’ welfare.
So far, so good. It seems possible to share the above information with potential participants and to devise a simple online form to document written consent. I would be concerned that the amount of information and formality may be off-putting but I absolutely understand the need for transparency and avoidance of harm, all of which this seeks to avoid or mitigate.
However, I’ve chosen the quote at the start of this section because I feel there is more to consent than the transaction of information sharing and signing a form. Apart from the fact that the participants have the right to withdraw at any time, that consent needs to be in an accessible format, and that it’s possible to consent to some aspects of the project and not others, there is also a question of trust in the relationship between the researcher and the participants.
As Parson points out, ‘exploring one’s identity as it relates to the identities of the group whose experiences one hopes to improve is an exploration that acknowledges differences in order to consider how the intersection of power and privilege impacts a researcher’s ability to conduct research ethically.’ (Parson, 2019 p.18). In my case, I am an outsider to the course, and have no power over grades or assessment, which might make the power balance more even, but on the other hand I am not someone the participants already know and trust, and I don’t share the lived experience of being neurodiverse. I am also white, a native English speaker and middle class, and perhaps this will make it harder for some participants to ‘consent’ to give their honest opinions. I am thinking of the broader means of consent that makes one’s contribution greater or smaller, more honest or more exposing.
Padan talks about the issues associated with written, contractual, informed consent, and I agree that this is not really the full picture and a more nuanced approach to consent is needed which builds trust and addresses positionality. However, I think the ‘interpersonal process’ also raises issues of power, agency and othering.
Right to withdraw:
As the UKRI state, consent should ‘…in an appropriate balance with resources available and researchers’ responsibility to truth-telling, be open to revision and withdrawal throughout the research lifecycle of a project.’
The right to withdraw is of course interwoven with consent and the BERA guidelines are clear that participants have the right to withdraw at any time. The researchers contact details should be provided and there is a requirement to reflect, should a participant wish to withdraw, on whether this is connected to the actions of the researcher and whether the duty of care has been met, and potential risks managed. (BERA p.25)
This leads on to a brief discussion of disclosure and incentives.
Incentives:
The UKRI is clear about the use of incentives. Whilst it acknowledges that small monetary reimbursement or the use of incentives like book tokens or entry into a prize draw might be used, ‘Payment should not override the principles of freely given and fully informed consent. Participants should know before they start the research that they can withdraw from the study without losing their payment.’ (UKRI 2023)
In my ARP, there will be no monetary benefit for the participants, but they will have the opportunity to shape the design of their course Moodle page and have an input into the design of Moodle pages (and information management) across the college.
Privacy and data storage:
Confidentiality and anonymity are key in relation to privacy and data storage. The General Data Protection Regulation (GDPR) and Data Protection Act 2018 provide clear structure as to the management of data, including that gathered for research purposes. In order to comply with these regulations and to conduct research ethically and without causing harm to participants, there are some really practical guidelines, some of which are outlined on the gov.uk website relating to user research.
Their advice includes:
Planning carefully how information will be gathered, who will take notes and what will happen to the information.
Sharing this at the consent stage with potential participants.
Collecting the minimum amount of information needed.
Storing details securely, ideally using computer networks that are password protected.
Avoid unnecessary sharing or access and avoid sharing via email.
Special care should be taken in relation to any sensitive personal information.
Participants’ contact details should be deleted as soon as they’re no longer needed.
UAL GDPR guidance also includes this notice, ‘…all personal data within an academic research project conducted by staff or students should be processed on systems, storage and software provided by UAL. Personal email or storage must not be used.’
In relation to my project, I will use my UAL cloud storage, OneDrive, and consider using Panopto to record the audio during the session, which is also password protected. I will also plan how to take notes during the session and how this will be coded to protect the participants’ anonymity. This is particularly important as I will be asking them to disclose their neurodiversity and the impact this has on their use of Moodle (where applicable). The information regarding their neurodiversity will need to be separated from their consent forms so that the sensitive data is anonymous. When using online forms, I will use MS Forms, which are part of my UAL software and therefore protected by password and Multi-factor authentication. There should be no need to share the non-anonymised data, unless I ask a colleague to take notes and observe. In this case, I will ask my colleague who is on the PG Cert and ensure she is fully aware of the need for confidentiality.
Both Sarah and I will be introduced to the participants, our roles explained and the use of data made clear in an information sheet and verbally. It may be that there are participants who want to be identified and ‘own’ their contribution. This will also need to be thought through so participants can express a preference without feeling pressure.
Transparency:
Guidance regarding transparency very much relates to the topics discussed above and relates to the purpose of the project, methods used, rationale in approaching participants, consent, right to withdraw and ‘the confidential and anonymous treatment of participants’ data’ (BERA p.27). It also includes how the data may or may not be reused.
In addition though, I think there is the other side to this which relates to the positionality of the researcher. Who am I? Why am I conducting this research and why am I focussing on this particular course? For me, I think this will be important as I approach the students as a stranger and hope to recruit participants who are neurodivergent and may already feel ‘othered’ or marginalised and may not wish to disclose their situation. I will need to work towards gaining the trust of the students through transparency and integrity.
Disclosure:
In the BERA guidelines, disclosure is discussed as relating largely to the disclosure by participants of illegal or potentially harmful activity, and how this should be approached in relation to confidentiality and statutory obligations.
I don’t anticipate the discussions we will have in my ARP falling under these headings, but I suppose I will need to be prepared for any disclosure about the way the student participants view their course or tutors and what I can/can’t do in response.
From a lay perspective, which I appreciate is separate from the BERA guidelines, I would also include the disclosure of sensitive information, in the context of my ARP, in relation to neurodivergence and related issues. Confidentiality and anonymity are obviously key in this regard, unless participants actively want to be identified. As mentioned above, I will need to ensure I have relevant support information available should the project cause or trigger any distress, even that related to experiences at UAL or in education as a member of a potentially marginalised group.
Harm arising from participation in research:
‘Ethical research design and execution aim to both put participants at their ease and avoid making excessive demands on them.’ (BERA p.25)
Harm can obviously come from many directions in relation to research and all the areas covered above relate to the need to avoid harm to participants as a result of their participation. Some of the key areas not already discussed include:
Duty of care – as a researcher asking for participants there is a duty of care to minimise risk and if distress or discomfort occurs, to manage it appropriately. I feel there is a risk in my project that they participants may feel that their ‘neurodiversity’ is exactly that, and that their diverse experiences make the focus group feel forced and uncomfortable. I will need to think carefully about the way I manage the session and what I ask the students to do so that they feel that their individual contribution is valid. I will also ensure I have signposting to support services available.
Avoid favouring one group – there may be a risk that other students feel excluded by my request for neurodiverse participants. I will make it clear why I need to limit the size of this group, and also that I will be asking for feedback from everyone on the Moodle page. I will also be very open to running the session again for a different group of students.
Working with marginalised groups – ‘Researchers should consider the impact of their research on the lives and workloads of participants, particularly when researching vulnerable or over-researched populations.’ (BERA p.25). I have some concern that neurodiverse students are often the focus of research at UAL at the moment and they may feel they are carrying an extra load, as they are asked to participate in research projects. I will need to ensure the load is manageable and that the purpose is clear. I will also need to be sensitive that this and the point above may come up during the recruitment of participants and ensure I am clear about my rationale and planning.
Reflect on any withdrawals – As mentioned above, there is a clear need to reflect carefully if participants decide to withdraw and to question if there was an element of the project design that contributed to this decision.
In terms of potential harm, I’m concerned about focussing specifically on neurodivergent students, whether this will create a feeling of ‘othering’ and if it will inadvertently exclude some disabled students because they don’t identify as being neurodivergent. For example, the disclosure of disability is lower amongst our international students. In addition, there is the concern about anonymity and about asking students to participate in a focus group, rather than individually.
I feel there is a high risk I will have got this wrong in some way, but without wanting to ignore advice from knowledgable and experienced colleagues, I also really want to try asking, and see what happens. The quotes below from the LSE blog ‘How do we know that our research is inclusive’ is actually about challenging the norms of academic research and ensuring the voices of marginalised groups are heard. It’s not about ignoring the voices of your experienced colleagues! However, it helped to give me courage to try out my idea, in a way which I hope will not cause harm, and to see what happens.
‘There are no definitive rules on what inclusive research is, nor how it is done. However, the focus is on working with underrepresented groups, such as disabled people, to shift the balance of power away from the researcher so that those with lived experience can have an authentic voice in shaping the research process.’ (Read, Parfitt, Bush 2020)
‘This means we have to be willing to hear the voices of those who have been excluded, in order to understand which research processes work, and which do not. We need to be open to trying new things and be aware that we will likely get things wrong – but getting things wrong is important in allowing us to learn and improve.’ (Read, Parfitt, Bush 2020)
As I’ve mentioned above, I think a key part of ‘getting this right’ will be about trust. Whilst I want the participants to feel they are in safe hands, and that the project is structured and organised, and that their input will be valued and recorded and protected, I also think there will be space to be honest about my concerns in this area and to ask for feedback about my approach. I think this is part of the reflective process in any case, but also I hope it will enable a greater sense of agency for participants, as well as informing future projects for me. As Bunting and Hill say in their ‘Belonging through compassion’ blog,
‘Revealing our shared humanity shows we are on their side. By doing so we encourage others to have the courage to share their authentic selves; necessary for a sense of belonging to flourish. This requires our own courage to hold vulnerabilities and accept uncertainty.’ (Bunting and Hill 2023)
There is much more to discuss here in relation to potential harm, marginalised groups, positionality and power, and research methods. Please see my Social justice post and Research methods for more reflections.
Bibliography:
Bali, M. and Zamora, M. (2022) ‘2: Intentionally Equitable Hospitality as Critical Instructional Design’ in Designing for Care, Pressbooks Available at: https://pressbooks.pub/designingforcare/chapter/intentionally-equitable-hospitality-as-critical-instructional-design/ (Accessed 7 January 2024)
Banks S.(2016) Everyday ethics in professional life: social work as ethics work, Ethics and Social Welfare, 10:1, 35-52, DOI: 10.1080/17496535.2015.1126623 (Accessed 20/10/23)
British Educational Research Association [BERA] (2018) Ethical Guidelines for Educational Research, fourth edition, Available at: London.https://www.bera.ac.uk/researchers-resources/publications/ethical-guidelines-for-educational-research-2018
Parson, L (2019) ‘Chapter 2: Considering Positionality: The Ethics of Conducting Research with Marginalized Groups’ in Strunk and Locke, eds. Research Methods for Social Justice and Equity in Education. Cham: Springer International Publishing pp. 15-32
I visited all three stages of the BA Course in person and used this PowerPoint presentation to describe the project and ask people to take part. I had hard copies of the Information sheet, QR codes and an online/accessible version to try and make it as easy as possible to take part.
For the second course, I sent the emails below to staff and students.
Email to staff:
Hello everyone
I hope you’re well. I’m currently doing a research project as part of my PG Cert, which involves taking a deeper look at how the Moodle design we use in (…), and also now elsewhere, works for different groups of staff and students.
For this project, I’m looking for neurodivergent, visually impaired or disabled staff who would be willing to attend a 30minute interview with me to just talk through how you use Moodle, what is helpful, and where any barriers are, as well as any changes that would improve it. The outcomes would help to inform the development of a Moodle template across CSM.
Participation would be confidential, and the outcomes would of course be anonymised. All data will be kept securely and you could decide to withdraw at any time. Full details are on the Participant information sheet.pdf.
If you’re happy to take part, I’d be really grateful for your input. We could meet online or in person at a time that suits you. Please just reply to this message or send me a message on Teams.
Many thanks for your help.
Best wishes
Amy
Email to students:
Hello everyone!
Could you help me with my research project on Moodle design and how it impacts different groups of students?
If you are neurodiverse, visually impaired or disabled, could you join a small, friendly discussion (focus group) to share your experiences? You wouldn’t need to disclose any personal details to your classmates and your contributions would be confidential.
If you can spare an hour to join the group, please complete this Consent form, or just get in touch for more information: a.urry@csm.arts.ac.uk. I will fit the time and date around you.
It’s still too long! I need to say less about thematic analysis and also reduce the text on my Research Question slide. I’m also aware I need to be clearer in what I want to say about my findings.
I’ve had a few goes at creating this presentation, but I tried out this version with my tutor group. It’s unfinished (no references and still some areas to sort out), but this was the very useful feedback:
It’s too long! 16mins – I need to cut it down.
Reduce the time spent on the Rationale slide and make the start more succinct.
Remember to mention the report – perhaps in final version of the action research cycle.
Reference participant/own drawings in reference list.
Avoid vague language – …and so on, …that kind of thing. – specifically in relation to ethics and coding.
Signpost ethics research explicitly.
Remember to mention Miro as the platform used for Thematic Analysis.
Make the themes and quotes more clearly linked – all on one slide?
In conducting the thematic analysis of data from the participant interviews, questionnaire and drawings, I have summarised them into 4 themes, each organised around a Central Organising Concept (Braun and Clarke 2018), or description. Each theme includes key takeaways and the conclusion addresses the research question.
See below for the:
Summary of findings.
Conclusion: addressing the research question.
Full report: Making Moodle more inclusive
Summary of findings:
Please find the summary of findings below. An accessible version on Word is also included.
To what extent does our Moodle design template support inclusive practice in relation to neurodiverse and disabled students?
Perhaps inevitably, the answer to this research question seems to be ‘partially’. Based on this data, there are practical elements that work well; headings, spacers, colour contrast, bullet points, and the use of distinct images for individual pages. When we work with course teams on their Moodle pages, we encourage them to think of Moodle as a member of the team, an active part of learning and teaching, and the data suggests that this has some traction – material is reliably available and Moodle is seen as important part of learning and teaching. Participants talked about the improvement from previous years and the overall usefulness of Moodle, suggesting (although not proving, of course) that this a step forward, rather than a reflection of Moodle without our current guidance.
‘Firstly, the redesign is better overall and more successful.’ Participant 1
‘I would say compared to last year’s Moodle, this one is a lot easier and a lot better to use.’ Participant 4
However, there are clearly significant areas where we need to improve. I would say the priorities here are clarity and consistency in the naming of documents; more training and support for staff, in a variety of easily digestible formats, to ensure they are in a better position to support students; and a push for lecture recordings to be made available. In addition, I think in the Digital Learning team we can look into potential for colour-coding, use of graphical images and further testing of the grid/collapsed topics format
One area we have not really touched on here is about accessible documents (the uploaded material on Moodle), and how to encourage best practice across the college. This is another significant area where we need to improve.
Finally, I’m aware that this project is very small and has focussed on one particular group of students. It’s interesting that relatively few of the comments made by participants specifically related to their neurodiversity or visual impairment. These were largely concerned with use of images or colour, a preference for a different font/background colour, use of a screen reader or the requirement to have less text, all of which are likely to be helpful to a wide variety of students. Having said that, I am also interested to run similar projects with other groups, in particular, students with English as an additional language, to see if their experience is substantially different. I will also incorporate some of the prompts used in this project into our Moodle evaluation forms in order to try and understand more about students and staff from a broader range of backgrounds who might not wish to participate in 1:1 interviews or focus groups. Speaking to students and staff has been a really pleasurable and valuable experience and I’m very grateful to all the participants. As one said:
‘So it’s good that they’ve got people thinking about this and, you know, meeting with students is very, very big step. And it’s a personal step, you know? So it’s good.’
‘Interwoven, neurodiversity and the creative mind’ Chris Kelly, Artist and workshop lead
In my project I hope to hear in detail about the views of people who are neurodivergent and so it makes sense to use a dialogic research method, where questions can be posed and follow up questions asked, based on responses. Using an interface such as Moodle is also a fairly intuitive exercise for most people and so it may be that the participants haven’t articulated or considered carefully their behaviour on the platform and what does and doesn’t work. Of course it’s equally possible that they have become frustrated with it or conversely, have appreciated changes in organisation. I feel that given the potential diversity of experiences, plus the range of potential participants, the opportunity to have an in-depth conversation is a key element of my project.
Focus groups and interviews: The main alternative to a focus group would be to do semi-structured interviews with each participant. This would of course offer benefits in terms of confidentiality and anonymity and it may allow participants to feel more comfortable to talk more freely. Talking about the advantages of semi-structured interviews, Adams says they are suitable, ‘If you need to ask probing, open-ended questions and want to know the independent thoughts of each individual in a group.’ He also mentions it suits ‘uncharted territory’ which I can’t claim this is for everyone else, but I feel it is for me. I would like to open up the conversations and see what happens.
However, my overall feeling is that given the nature of the topic and the diversity of participants, there is more to be gained from a group discussion. We can’t make Moodle perfect for every individual, what we can do is make it good enough to be inclusive and flexible enough to meet everyone’s needs. Although there are many definitions of focus groups, I feel that this one describes the tone of what I’d like to achieve.
‘Agar and MacDonald (1995) suggest that a focus group lies somewhere between a meeting (reflecting the fact that it is specifically organized in advance and has a structure) and a conversation (reflecting the fact that the discussion has nonetheless a degree of spontaneity, with individuals picking up on one another’s contributions)’ (Sim and Waterfield, 2019, p. 2)
There is also the additional element that the ‘outcomes’ of the focus group should include both dialogue and interaction, that the act of the group conversation leads to something more than the sum of it’s parts. I feel that this could be beneficial in discussing information management on the course as hearing about how others experience and interact with the page, may prompt participants to realise or articulate something about their own behaviour and preferences.
As I mentioned above, it is partly for this reason that I am proposing a focus group rather than individual interviews, despite the risks (see below). In addition, I watched the video above of Chris Kelly’s account of a crochet workshop he held with neurodiverse students where several of the participants talk about the, sometimes unexpected, benefit of coming together as a group, even though neurodiversity is explicitly about diversity, it was still possible to find things in common and to feel a benefit of being in a community. Obviously I can’t be sure this will happen, but it feels like a really positive possibility and it will be interesting to see if there is any sense of this that comes from the focus group.
I think for myself also, the information I hope to discover, isn’t totally clearly defined, it’s not an account and nor is it quantifiable. I feel that the ability to be flexible and responsive when facilitating the focus group will be important to the development of my thoughts, as well as meeting the as yet unknown needs/wishes/preferences of the participants. Furthermore, this flexibility feels like an important element of inclusive research. Although I can’t realistically manage full participatory research for this project,
‘… allowing participants to shape the direction of data collection allows them continued access to the research process.’ (Larson 2019, p.28)
I wonder if there is scope to offer a little more flexibility in the research methods. A simple, but limited, way would be for participants to choose questions from a selection, at least enabling participants to focus on the aspects they chose to.
Kerschbaum and Price write about ‘centring disability’ when conducting qualitative interviews. This has made me think about whether I can be more flexible about the methods I use. For example, I have been agonising about whether to invite students to a focus group, with an interview as an option, but maybe I need to frame this more as a positive choice, rather than a way of ‘dealing with’ privacy. For example, they say ‘Centring disability is about much more than simply compensating for or including disabled researchers and participants’ (Kerschbaum and Price 2017, p.98). It’s not an awkward thing to deal with, it’s a choice.
I see more also how my methods of recording and the interviews and gathering data, might need to be more flexible, but that also might bring about something more positive, for example, using captions or interviewing online. Kerschbaum and Price’s argument is that thinking of these arrangements as something to fully pin down in advance, reduces the chance of an opportunity for extra insight. As they say, ‘we recognise our strengths that stem from disability and and we make access a higher priority than maintaining consistency across different interview modalities’ (Kerschbaum and Price 2017 p.100). For example, perhaps it’s possible to keep some elements of the modality open, depending on participants’ responses, thus making space for greater agency and accessibility.
Risks: There are risks to using a focus group, including that participants may not want to disclose their diagnosis to peers, or any difficulties they may have in accessing information. There is also the risk that participants are less open about what they experience and may follow along with more dominant participants. There is of course also the risk that confidentiality is not maintained and therefore even after the focus group, there may be repercussions that are difficult to control and mitigate.
Strategies: In order to address some of these issues, I will start the focus group with a ‘briefing’ which outlines the purpose, the idea of a confidential space to talk and the importance of trust between participants. I’ll also briefly remind participants of this before they leave. In addition to this, I’ll plan in some time for participants to talk in pairs before sharing with the whole group, to make space for everyone to contribute, and also incorporate some individual responses, before asking the group to discuss. I’ll also try to create a friendly, welcoming space through arranging the room nicely, having snacks and drinks available and having things on the table for people to doodle, fidget or whatever they like to relax. I’ll also start the session with some gentle chat to try and generate a relaxing atmosphere.
Another strategy here would be to make use of an online questionnaire to offer an opportunity to participants to contribute confidentially. As well as sharing their experiences this way, they could also disclose their neurodiversity on the form, meaning they wouldn’t need to do this in front of their peers. Finally, it would also give me the opportunity to ask some targeted questions without interrupting the flow of the discussion. This might also help to make the focus group more relaxed, more of a conversation than a Q&A session.
Drawing: One thing I am interested in doing is asking participants to draw their experience of Moodle. I think this is because I often think of experiences visually, through a metaphor or a rough drawing e.g. scribbles for confusion. I also looked at the ‘draw and write’ technique described by Helen Kara as ‘a tried and tested technique of gathering data that enables children to express their views and opinions in their own terms’ (Kara, 2015, p.89). Although Kara describes using the technique with children, which is not relevant to me, I was drawn to the idea of participants expressing themselves in their own terms. After listening to Dr. Jenni Good talk about using drawings as a form of data collection in her Cross-Programme talk, I feel more confident in pursuing this. As my topic is quite dry, I think it might open up a different perspective on how it actually feels to use Moodle, and also, potentially, might offer participants a medium they could be more comfortable with using.
I am concerned though that this might make participants feel uncomfortable or self-conscious, so I will make it very clear that the level of drawing is unimportant and also they are free to do it in any way they choose. Of course it will also be fine for anyone to say they’d rather not participate in this element.
References:
Adams, W. ‘Conducting Semi‐Structured Interviews’ In Handbook of Practical Program Evaluation, edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 1st ed., 492–505. Wiley, 2015. Available at: https://doi.org/10.1002/9781119171386.ch19. (Accessed 22 November 2023)
Kara, H. (2015) Creative Research Methods in the Social Sciences: A Practical Guide. Bristol: Policy press.
Kerschbaum S. and Price M. (August 2017) Centring disability in qualitative interviewing Research in the Teaching of English, Volume 52
Parson, L (2019) ‘Chapter 2: Considering Positionality: The Ethics of Conducting Research with Marginalized Groups’ in Strunk and Locke, eds. Research Methods for Social Justice and Equity in Education. Cham: Springer International Publishing pp. 15-32
Sim J. and Waterfield J. (2019) Focus group methodology: some ethical challenges Quality & Quantity Available at: https://doi.org/10.1007/s11135-019-00914-5 (Accessed 6 Nov 2023)
