Learning Analytics Toolkit: Potential Outcomes

Learning analytics, the “collection and analysis of usage data associated with student learning…to observe and understand learning behaviors in order to enable appropriate interventions,”[i] employs data to improve learning contexts and help learners succeed. Learning analytics seeks to 1) help educators discover, diagnose, and predict challenges to learning and learner success; 2) enable instructors to identify and enact necessary changes to improve and customize educational content, delivery; 3) empower learners with insights into their own learning;[ii] and 4) point the way to successful and active engagements and interventions that benefit students. In a library context, learning analytics can help librarians share library data with students to enable them to have agency and control over their own learning journey; uncover systemic hindrances to student learning; make decisions and take actions to support students as they overcome those hurdles; change policies, procedures, and practices to dismantle structural obstacles to student learning, and develop and deepen collaborations with other members of the educational support team (e.g., faculty, advisors, etc.) in service of student learning and success.[III]

As librarians determine whether or how to engage in learning analytics, they may find the development of “user stories” a useful tool in guiding potential learning analytics development work.  The LIILA report (section 4.1) presents the 95 user stories brainstormed as a part of the LIILA project as well as 14 categories ranked as "high impact" by LIILA participants (section 4.2). Each user story focuses on a user group, such as students, librarians, faculty, academic advisors, institutional researchers, or senior leaders. In each story, the “user” is followed by a “want” statement. Want statements represent the potential outcomes of learning analytics effort may include the ability to do an activity, build an awareness, or accomplish a task requiring library/institutional data. When library or institutional data is necessary for the “want” to be achieved, that data is visually separated into two categories (“library” and “institutional”) for clarity. To conclude each user story, a reason, intent, use, or goal for the “want” is listed; in general these focus on achieving an outcome, solving a problem, and/or meeting a need. In this way, user stories guide the potential outcomes of learning analytics efforts. [IV]  A few examples from section 4.0 of the LIILA report are listed below.

• As a student, I want to know whether using resources provided by the library will save me money on textbooks so that I can afford to stay in school or accumulate less debt.
• As a student, I want to know whether my use of the library is more/less/equal to other students so that I can match my library use with other students who are successful.
• As an academic advisor, I want to know if students who are contacted by or referred to librarians for consultations or instruction attain more learning outcomes, earn better assignment grades or course grades, are more engaged, are retained, graduate/complete on time, etc. so that I can make more referrals and get students the help they need.
• As faculty, I want to know whether my inclusion of library resources in student assignment requirements correlates with student learning outcomes attainment, assignment or course grades, or teaching evaluation measures so that I can change the amount of my inclusion of library resources in student assignment requirements to increase student learning or receive better teaching evaluation scores.
• As a librarian, I want to know whether students who interact with library reference services attain more learning outcomes (i.e., learn more), earn better assignment or course grades, are more engaged, are retained, transfer successfully, graduate/complete on time, get jobs, and/or earn more money so that I can advocate for more (or more appropriate) reference resources, encourage more faculty and students to interact with reference librarians, and improve reference services.
• As a librarian, I want to know whether the amount, degree, or relative rank of student library resource use or other library participation impacts learning outcomes attainment, assignment or course grades, GPA or test scores, engagement indicators, and/or semester-to-semester retention, transfer success, employment rates or earnings after graduation/completion so that I can encourage faculty to require use of more library resources in their teaching content and assignment design, and encourage students to increase their library resource use.
• As a librarian, I want to know whether any relationships between the use of library services/resources and institutional outcomes (described in other user stories above) vary by student population/status/characteristics so that I can tailor library services/resources to meet the needs of populations with specialized needs and engage in appropriate instruction, outreach, etc. and help the institution prepare for changing student demographics.
• As a librarian, I want to know whether any relationships between the use of library services/resources and institutional outcomes impact learning outcomes attainment assessed at the assignment level (or some other more granular level that may actually get at learning directly) so that I can tailor library services/resources to maximize and facilitate student learning.  Learning outcomes might include: locating, evaluating, using information; thinking critically, analytically; analyzing, solving problems; applying information skills to the real world; disciplinary information skills; etc.
• As an institutional researcher, I want to know whether and to what degree librarian interactions with identified at-risk student populations influence their short- or long-term student success so that I can facilitate connections with librarians and improve student outcomes.
• As a senior institutional leader, I want to know whether (which) library expenditures impact student success measures, faculty research productivity, faculty grant productivity, institutional brand/image/prestige, institutional affordability, and/or institutional accreditation so that I can make resource decisions that will maximize institutional outcomes.

Potential outcomes for library engagement in learning analytics may also be expressed in a research question format; sample research questions are also included in the LIILA report in section 3.2.

Share your feedback and suggestions

Please let us know how we can improve the toolkit via this anonymous feedback form.