Learning Analytics Toolkit

Learning analytics, the “collection and analysis of usage data associated with student learning…to observe and understand learning behaviors in order to enable appropriate interventions,”[i] employs data to improve learning contexts and help learners succeed. Learning analytics seeks to 1) help educators discover, diagnose, and predict challenges to learning and learner success; 2) enable instructors to identify and enact necessary changes to improve and customize educational content, delivery; 3) empower learners with insights into their own learning;[ii] and 4) point the way to successful and active engagements and interventions that benefit students. In a library context, learning analytics can help librarians share library data with students to enable them to have agency and control over their own learning journey; uncover systemic hindrances to student learning; make decisions and take actions to support students as they overcome those hurdles; change policies, procedures, and practices to dismantle structural obstacles to student learning, and develop and deepen collaborations with other members of the educational support team (e.g., faculty, advisors, etc.) in service of student learning and success.[III]

As librarians determine whether or how to engage in learning analytics, they may find the development of “user stories” a useful tool in guiding potential learning analytics development work.  The LIILA report (section 4.1) presents the 95 user stories brainstormed as a part of the LIILA project as well as 14 categories ranked as "high impact" by LIILA participants (section 4.2). Each user story focuses on a user group, such as students, librarians, faculty, academic advisors, institutional researchers, or senior leaders. In each story, the “user” is followed by a “want” statement. Want statements represent the potential outcomes of learning analytics effort may include the ability to do an activity, build an awareness, or accomplish a task requiring library/institutional data. When library or institutional data is necessary for the “want” to be achieved, that data is visually separated into two categories (“library” and “institutional”) for clarity. To conclude each user story, a reason, intent, use, or goal for the “want” is listed; in general these focus on achieving an outcome, solving a problem, and/or meeting a need. In this way, user stories guide the potential outcomes of learning analytics efforts. [IV]  A few examples from section 4.0 of the LIILA report are listed below.

For students, faculty, and academic advisors

• As a student, I want to know whether using resources provided by the library will save me money on textbooks so that I can afford to stay in school or accumulate less debt.

As an academic advisor, I want to know if students who are contacted by or referred to librarians for consultations or instruction attain more learning outcomes, earn better assignment grades or course grades, are more engaged, are retained, graduate/complete on time, etc. so that I can make more referrals and get students the help they need.

As faculty, I want to know whether my inclusion of library resources in student assignment requirements correlates with student learning outcomes attainment, assignment or course grades, or teaching evaluation measures so that I can change the amount of my inclusion of library resources in student assignment requirements to increase student learning or receive better teaching evaluation scores.

As an institutional researcher, I want to know whether and to what degree librarian interactions with identified at-risk student populations influence their short- or long-term student success so that I can facilitate connections with librarians and improve student outcomes.

For librarians

• As a librarian, I want to know whether students who interact with library reference services attain more learning outcomes (i.e., learn more), earn better assignment or course grades, are more engaged, are retained, transfer successfully, graduate/complete on time, get jobs, and/or earn more money so that I can advocate for more (or more appropriate) reference resources, encourage more faculty and students to interact with reference librarians, and improve reference services.
• As a librarian, I want to know whether the amount, degree, or relative rank of student library resource use or other library participation impacts learning outcomes attainment, assignment or course grades, GPA or test scores, engagement indicators, and/or semester-to-semester retention, transfer success, employment rates or earnings after graduation/completion so that I can encourage faculty to require use of more library resources in their teaching content and assignment design, and encourage students to increase their library resource use.
• As a librarian, I want to know whether any relationships between the use of library services/resources and institutional outcomes (described in other user stories above) vary by student population/status/characteristics so that I can tailor library services/resources to meet the needs of populations with specialized needs and engage in appropriate instruction, outreach, etc. and help the institution prepare for changing student demographics.

Potential outcomes for library engagement in learning analytics may also be expressed in a research question format; sample research questions are also included in the LIILA report in section 3.2.