Integrating ethics training into any quantitative course

Today we have a guest entry authored by Rochelle Tractenberg (Georgetown University) about integrating ethics training into any quantitative course. She writes:

This blog post focuses on encouraging instructors to use structure in order to facilitate the integration of ethics training into their courses. In particular, five structural features are briefly introduced.

Instructors can use professional ethical practice standards, not issues, to guide instruction/learning objectives.

This assumes (or, also encourages) the use of instructional and/or curriculum development guidelines–all of which will place the articulation of realistic and meaningful learning outcomes in the primary decision making position–such that all later instructional decisions follow from the learning objectives. Using issues, instead of practice standards, makes the articulation of learning objectives difficult; it can also lead to the incorrect perception that it is issue resolution, and not ethical practice, that is the purpose of the training. These suggestions are motivated by (and explicated in) a new white paper, “Ten simple rules for integrating ethics into statistics and data science instruction” (Tractenberg, 2020).

Two other resources can also provide helpful structure: an outline of stakeholder analysis, which can be used to orient students to harms and benefits associated with decisions that they make throughout the process of data analysis; and the knowledge, skills, and abilities of ethical reasoning (which utilize professional practice standards and stakeholder analysis in order to support defendable decisions about ethical practice or in cases of ethical challenges or dilemmas). Finally, considerations of seven tasks that all statistics and data science can/do follow/recognize means that the integration effort will leverage–and accommodate–the complexities of practice. This means that, rather than adding “ethics content”, instructors can truly integrate ethics training as they engage students with authentic, practice-based, approaches to using statistics and data science methods ethically and responding to ethical challenges as they arise throughout their careers.

Practice Standards

The ASA Ethical Guidelines for Statistical Practice include 52 items under eight general areas:

A. Professional Integrity & Accountability (7)
B. Integrity of data and methods (11)
C. Responsibilities to Science/Public/Funder/Client (5)
D. Responsibilities to Research Subjects (7)
E. Responsibilities to Research Team Colleagues (4)
F. Responsibilities to Other Statisticians or Statistics Practitioners (4)
G. Responsibilities Regarding Allegations of Misconduct (6)
H. Responsibilities of Employers/Clients Employing Statistical Practitioners (8)

In one sense, this means there are 52 different ways to add ethics content to your course. In another sense, this means that the ethics content really cannot be memorized – because it is a lot to remember; fortunately, “memorize the ASA Ethical Guidelines” is a weak learning outcome that few instructors would bother to formulate – because memorization, the lowest (least complex) cognitive behavior in Bloom’s taxonomy (Bloom et al. 1956), does not enable the learner to utilize what was memorized.

Curriculum development guidelines

Integration of ethics content, and ethical reasoning in particular, should follow established curriculum and instructional guidelines (described and discussed in Tractenberg et al. 2020), for example:

A. Identify and follow a formal paradigm for curriculum or instructional design.
B. Focus on Learning Outcomes (LOs) first, to inform all other decisions about the curriculum and instruction.
- B1: Leverage LOs to explore and identify appropriate learning experiences (LEs).
- B2: Leverage LOs to select content that is appropriate for the LEs and promotes the LOs.
- B3: Assess learning based on achievement of LOs using formative and summative assessment, as appropriate.
C: Plan and execute an actionable evaluation of the curriculum and instruction.
D: Document and share the features of the curriculum or instruction – including criteria for their success – with learners.

As guidelines for curriculum and course development in higher education and training will suggest, realistic and actionable learning outcomes are essential to ensuring that the effort required to achieve the target integration (i.e., of ethics training) is actually effective. Moreover, if there are improvements to implement, an actionable evaluation will help identify them.

Knowledge, skills, and abilities (KSAs) of ethical reasoning

One way to conceptualize the purposes of integrating ethical training is to focus on preparing students to use ethical reasoning (Tractenberg & FitzGerald, 2012), because applying ethical guidelines can be complex. Learning to reason with (i.e., utilize) professional practice standards and stakeholder analysis will allow students to make and support defendable decisions about ethical practice, or, in cases of ethical challenges or dilemmas. Importantly, the ethical practice standards can help both general practice and in identifying and overcoming ethical challenges or dilemmas. This is lost when instruction or training focus solely on issues. The articulation of learning objectives can be facilitated by a focus on ethical reasoning (i.e., on the initiation and future growth of these specific KSAs along the pre-specified developmental trajectory outlined by Tractenberg & FitzGerald, 2012; Tractenberg et al. 2017).

Leverage – and accommodate - the complexities of practice.

Since “application” of ethical practice standards is a considerably more cognitively-complex behavior than “recalling” those standards (Tractenberg et al. 2013), instructors should be prepared to encourage students to move beyond memorization on Bloom’s Taxonomy of Cognitive Behaviors (Bloom et al., 1956).

Image released under a Creative Commons Attribution license from the Vanderbilt University Center for Teaching.

It is essential to recognize that Bloom’s level 4 (analysis) is required to use ethical practice standards effectively; since the ASA Ethical Guidelines are just that, guidelines, and not rules, Bloom’s level 3 (application) is not a complex-enough level of performance to aim for. Rather than create de novo materials for existing courses that might then appear to compete for attention during the available time, consider seven tasks that all statistics and data science can/do follow/recognize (Tractenberg in review A; in review B):

plan/design
collect/munge/wrangle data
analysis – literal for statistics & data science, “evaluation” for computing
interpret – always for statistics & data science, never for computing
document
report & communicate
work on a team

These seven tasks are essential in the practice of statistics and data science, and the ASA Ethical Guidelines pertain in each of these tasks. Thus, rather than adding “ethics content” to an existing course that is presumably already “full” of content, instructors can truly integrate ethics training as they engage students with authentic, practice-based, approaches to using statistics and data science methods ethically and responding to ethical challenges as they arise throughout their careers. Again, leveraging these tasks (and/or any of the other structural elements described here) can help instructors to focus on learning objectives – not issues – and to optimize existing course/teaching materials so that articulated learning objectives - plus specific learning objectives about ethical reasoning for statistics and data science - have the greatest chances of being achieved.

Bio

This is a guest post by Rochelle Tractenberg. Dr. Tractenberg is a tenured professor in the Department of Neurology, with secondary appointments in the Departments of Biostatistics, Bioinformatics & Biomathematics and Rehabilitation Medicine at Georgetown University. She is also a Research Fellow at the National Rehabilitation Hospital in Washington, DC. She is a research methodologist specializing in designs and analyses with “difficult to measure” outcomes in biomedical and educational studies, with PhDs in psychology/cognitive sciences (1997) and measurement, statistics, and evaluation (2009); she also earned a doctoral level certificate in gerontology (2006). Her areas of interest beyond biostatistics and psychometrics include higher education curriculum development and evaluation; statistical methodology and statistical literacy for effective stewardship of the discipline in PhD students/holders; and effective instruction and mentoring in research ethics. She was elected a Fellow of the American Statistical Association in 2016 and a Fellow of the American Association for the Advancement of Science in 2017. She was the Chair (2017-2019) of the American Statistical Association Committee on Professional Ethics, serving as Vice-Chair 2014-2016. Internationally she is involved in educational and training initiatives that feature the curriculum development, evaluation, and revision tool she created, the Mastery Rubric (https://osf.io/preprints/socarxiv/qd2ae/). In addition to supporting effective curricula (i.e., where intended and actual learning goals are aligned), the Mastery Rubric construct is the first way to bring psychometrically-defined validity to curriculum decisions, and has supported the first-ever evidence that training in research ethics can be sustainable beyond the end of instruction (http://www.mdpi.com/2227-7102/7/1/2).

About this blog

Last summer we wrote a series of blog entries designed to start conversations around teaching data science, Teach Data Science. We covered topics such as data science software, data ingestation, data technologies, data wrangling, visualization & exploration, communication, and key reports and findings on data science.

One key element that was lacking on our 2019 blog was a discussion about and a commitment to teaching the ethical aspects of data science. We have now found ourselves in the summer of 2020, overwhelmed by the state of the world and re-committed to the ethical challenges which can help data science be a positive force for change.

Although none of us are experts in ethics, we have all included ethics discussions in our classrooms for many years. In the weeks to come, we will share some of the ways we engage our students in these important topics. We will provide resources for readings, examples, datasets, and exercises. We believe that data ethics are part of every data science analysis and classroom experience, and we hope that this summer’s blog will entice you into presenting ethical dilemmas and related conversations to your students early and often.

During the summer of 2020, we plan to write a dozen blog entries on data ethics. We’re nearly there! We hope that you bookmark the site and check in regularly. Want a reminder? Sign up for emails at https://groups.google.com/forum/#!forum/teach-data-science (you must be logged into Google to sign up).