At the sumptuous RSC Library at Burlington House, we gathered Methods in Chemistry Education Research 2017, a day of lectures, activities and catching up with friends and colleagues. From school teachers to a Professor Emeritus, we gathered with a common purpose – to spend a day thinking about methods in chemical education research.
The day started with Dr Suzanne Fergus (@suzannefergus), Principal Lecturer in Pharmaceutical Chemistry at University of Hertfordshire (also 2016 RSC Award Winner for Higher Education Teaching). Through the context of her journey into ChemEdRes, Suzanne discussed the difference between anecdote of what works in our own teaching situation, and what constitutes genuine research. Critical features included i) contextualization within the current literature, ii) robust data collection and evaluation, and iii) novelty of work. While replication of others work in our own context can help increased generalisability of ideas, the new learning from such replication needs to be made explicit.
We worked through an exercise in formulating a RESEARCH QUESTION, central to ensuring high-quality research, and ultimately in getting our studies published. In my previous teaching of A-level sciences, I have come across research questions in Biology fieldwork, but their use in Chemistry research are not common. The worksheet proved a useful structure to start the challenging process of formulating high quality and usable research questions. Benefits of starting the research process with the research question include i) helping connect with the literature; ii) influence on the methods used; iii) focus on the presentation of the work and iv) focus on the discussion of the conclusion.
One of Suzanne’s papers (DOI: 10.1021/ed2004966) was highlighted as a useful example of how ChemEdRes can be written. The ‘New Directions’ journal was also suggested as a good starting point for those looking to get into academic publishing. Suzanne also suggested other less formal (more ‘social’) ways of publishing to help build one’s confidence in sharing our thoughts with a wider community. This included speaking at TeachMeets, small conferences, engaging in Twitter conversations, writing personal and professional blogs and writing for institutional publications. On a personal level, Suzanne’s talk gave me that last little push to start a personal blog!
Suzanne’s colleague Dr Stewart Kirton (@skirtonUH), Head of Pharmaceutical Chemistry, University of Hertfordshire, then took us through the use of Likert scales in providing an assessment of the impact of our interventions. While analysis of attainment in assessments is a major source of such information, surveying students’ perceptions is an increasingly used source of information. I used such surveys throughout my time in secondary teaching, and their use is becoming more common at university level with the ‘Teaching Excellent Framework’.
Stewart took us through a process for developing valid questions and Likert-scale responses, including:
- the importance of trialling the questions with peers
- trialling with the subjects of the questions (usually your students
- ensure each question is only examining one idea
- avoiding jargon
- think carefully about the possible responses – including ‘Don’t Know’ is acceptable
- phrase questions positively (if possible)
- sticking to around eight questions – using many more than this and the students will likely run out of steam!
Our activity involved drafting some questions to help evaluate a programme run by final year students to help second-year students prepare for interviews for industry years. A particularly useful online app was used to share our ideas (www.mentimeter.com) – a virtual notice board where you can send in your responses via smartphone/laptops.
Stewart finished with a clear exhortation on NOT taking the average of responses when using numerical responses on Likert scales (e.g. 1=strongly agree to 5=strongly disagree). Simply put, these numbers are not interval data, where the difference between successive values are identical and meaningful, rather they are ordinal data, i.e. can be ordered but the differences between them are meaningless. Stewart’s suggestion was to present the relative ratio of each response to each question and analyse pre- and post- intervention where appropriate.
After coffee, and meeting up some friends I have made on Twitter over the last year, Dr Orla Kelly (@orlakelly5), Senior Lecturer in Social, Environmental and Science Education, Dublin City University, discussed the evaluation of classroom practice, with a focus on ‘Classroom Action Research’. Orla started with a definition from the Open University, of ‘systematic and collaborative collection of evidence on which to base reflection’. She provided a summary of the cycle of action research as ‘Plan / Act / Observe / Reflect’. Orla’s extensive use of Problem-Based Learning in undergraduate labs provided a context for the talk, and had strong resonances with Suzanne’s earlier talk.
Prof Graham Scott (@grahamscott14), Professor of Bioscience Education at University of Hull then expanded the speaker repertoire beyond chemists to a biologist helping bring a perspective from a related relevant field. Graham’s key message was of the advantage of moving away from our ‘science comfort blanket’ and embracing the discomfiture of collecting and using the more qualitative data derived from interviews. Graham took us through his research journey of using interviews in studying various different areas, from an analysis of student’s and teacher’s perceptions of a course, to barriers to using biological fieldwork in primary schools.
Key ideas in making effective use of interviews included i) establishing a suitable dynamic between the interviewer and interviewee (location, time available, consideration of any prior professional relationship); ii) clearly constructed questions that will elicit the information required (including the use of trialling) and iii) the importance of audio/video recording and the processes of transcribing and analysing the data.
I have used interviews in previous research and as part of evaluating the effectiveness of my previous school departments. While I experienced many of the problems that Graham described, I wholeheartedly agree with Graham that the quality of information you can derive makes them well worth the effort.
After a much-needed lunch (energy levels were flagging by 1.10pm!) we had time to chat and look at posters. This was a nice aspect of the conference, starting off the conference a good month before the get-together, and get feedback on our ideas online. My particular interest right now is in how microscale chemistry can be integrated into my teaching, and whether it has sustained benefit to students’ learning.
Prof Keith Taber (@DrKeithSTaber) took us on a tour of ethics in educational research. It has been many years since my MEd days with Keith as my supervisor, but his erudite and rigorous style continues to shine through and it was a pleasure to be part of the audience.
Starting with a brief tour of various ethical frameworks, including deontology and utilitarianism, we discussed the importance of voluntary informed consent from the subjects of our research, and the responsibility we bear as researchers. These include reporting our findings as completely and fairly as possible, including not selectively reporting our findings, and highlighting the known limitations. We discussed the issues around anonymity and confidentiality, and the particular problems that the easy access to worldwide information via the internet can provide. We discussed the particular cases of the Milgram studies and the Tuskegee Syphilis experiment, highlighting areas of real contention in the ethics of research.
The key message from this double session was that while the rules of ethics can be relatively easily stated, the actual decisions we have to make as researchers can be very nuanced and rightly deserve careful consideration before, during and after our studies.
The day finished with Prof Georgios Tsaparlis, Professor Emeritus of Science Education in the Department of Chemistry at the University of Ioannina, Greece, winner of the 2016 RSC Education Award. Georgios’ work on problem-solving has spanned decades, and a whistle stop tour was presented in this final session. Ideas around how the limitations of our cognitive architecture, with a particular focus on working memory, were discussed. The importance of scaffolding and exercise, as well as success for students, in developing problem-solving skill was clearly emphasised.
MICER17 proved to be all I had been looking forward to, and a great venue to meet new people, make connections and expand my professional network. Most of all, it has helped put the human face to the world of ChemEdRes. Reading articles in CERP or J Chem Educ can be a little daunting to those new to ChemEdRes, and the barrier to entry to the world can seem impossibly high. However, it really is an inclusive and welcoming community, one that I look forward to contributing to in the coming years. Many thanks to Micheal Seery (@seerymk) and Claire McDonnell (@clairemcdonndit) for all the hard work in bringing this together.