Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Learning Analytics Toolkit: Leading Libraries

At the Forefront

The following libraries have made great strides working with library learning analytics. A brief description of their work can be found on page 81 of the LIILA report (i). This list will be growing. 

 

University of Michigan

University of Minnesota

Lewis & Clark Community College

University of North Carolina - Charlotte

 

[i] Oakleaf, M. Library Integration in Institutional Learning Analytics (LIILA. Nov. 15, 2018), available online at: https://library.educause.edu/~/media/files/library/2018/11/liila.pdf

Library Learning Analytics Project - Grant Abstract

We propose a project centered at the University of Michigan (U-M) to study how libraries impact learning. We define learning as the process of acquiring knowledge and/or skills through formal study or instruction (course instruction), or experientially through laboratory research or clinical activities (research). However, learning is also the outcome of this knowledge acquisition process, including the ways or means by which this knowledge is disseminated to society (publishing). We use analytics – the discovery, interpretation, and communication of meaningful patterns in data (https://en.wikipedia.org/wiki/Analytics) – to address the research question: How does the library impact learning, specifically in the areas of course instruction, research, and publications? Libraries should strive to improve learning outcomes for the communities they serve, just like how healthcare institutions should ideally improve individuals’ health outcomes. This type of work is best described as library learning analytics (LLA), which entails embedding library data within institutional learning analytics ecosystems, and is guided by a more encompassing definition of learning that extends beyond the classroom. Our goal is to establish the groundwork for a group of diverse institutions to use a common analytics framework. The three-year project, which will commence in June 2018, has two main goals as follows:

Goal 1: To understand how the U-M library impacts learning, specifically in the areas of course instruction, research, and publications. We plan to achieve this goal through modeling and analyzing both deidentified and identifiable library use data including: website server logs, library catalog server logs, proxy server logs, circulation history and related data, and campus status & affiliation data. These datasets are linkable through both strong and unambiguous identifiers that are unique to each individual user, as well as IP addresses and timestamps. First, using deidentified data, we aim to apply clustering algorithms to identify typologies of library users on the basis of library interactions e.g. user type A (high degree of use), user type B (low degree of use), etc. Second, we aim to replicate the clustering analysis using the identifiable data, using the deidentifiable clustering findings for robustness checks. Third, our goal is to use clusters from identifiable data in the sequence mining, and predictive and prescriptive modeling of the links between learning outcomes and library user types. For privacy reasons, results will be shared using only aggregated and anonymized data.

Goal 2: To develop tools, scripts, and protocols that serve the library analytics needs of our community of project advisory group (PAG) institutions. We plan to achieve this goal through two modes of engaging with the PAG community. First, we plan to develop and share with PAG institutions data dictionaries (names, definitions, and attributes about data elements that are in a database) of the datasets we will be using in the project. This will facilitate data harmonizing across PAG institutions (and with U-M data) and thus smooth the exchange of ideas and best practices over the course of the project. We will maintain a repository of tools and scripts developed for data cleaning and database construction and grant PAG members early access to this repository. We also plan to regularly update PAG members via email on the latest status of data management and analysis activities with a view to making it easier for them to voluntarily replicate our work or perform their own LLA studies. Thrice a year, we will hold virtual research meetings with PAG members for more thorough debriefs of project activities. Lastly, throughout this process we will stay in regular phone and email contact with PAG members in order to help them resolve replication issues as they arise. Second, we will give PAG members access to aggregated and anonymized findings (from Goal 1, using U-M data) through a webbased dashboard running on data that will be located on a secure virtual data enclave. Further, in addition to the dashboards, we will hold annual PAG Workshops where member representatives will be tutored and immersed in training tasks to ease voluntary replication of the project, but using their own data.

Impact: This project will provide guidance to PAG institutions and other libraries on how to best design and implement empirical, holistic LLA studies of the links from library usage, to learning outcomes such as course instruction, research, and publications. The project will produce a set of tools, scripts, and protocols that will be freely available to all libraries. The study will serve as a template for other libraries with respect to: 1) studies that collect and store library use data with individual identifiers while maintaining the privacy of individuals; 2) designing and implementing a holistic LLA study of the link from library use to multiple learning outcomes, and; 3) creating a secure cyberinfrastructure (data repository, virtual enclave, and dashboard) for LLA research that facilitates collaboration in a community of diverse institutions in the US and Canada. 

More information can be learned at the project website, https://libraryanalytics.org/.

 

 

 

Connecting Libraries and Learning Analytics for Student Success (CLLASS) - Grant Abstract

CLLASS is a one-year grant designed to perform preliminary planning activities to pioneer the integration of library data in institutional learning analytics and develop detailed proofs of concept and models to guide academic libraries preparing to engage in this emerging and important use of data to support student success.  The lead applicant, Syracuse University, is joined by advisory group members and project participants from ACRL, Blackboard, CNI, DePaul University, EDUCAUSE Learning Initiative, IMS Global Learning Consortium, Jisc, Lewis and Clark Community College, OCLC, Susquehanna University, the University of California, Berkeley, the University of Michigan, the University of Minnesota, and Unizin.

The foremost purpose of higher education is to educate students, and a key component of any educational endeavor is assessment.  As active contributors to the educational mission of their institutions, academic librarians use assessment to expand student access to learning; ensure students are able to persist and attain their goals; scaffold student experiences to aid attainment of independent learning capacity; and develop productive self-awareness, metacognition, and self-actualization in a variety of contexts, including their immediate learning environments, the broader community, and the world around them.  Now, as institutions of higher education commence and commit to the next wave of assessment capability in the form of learning analytics initiatives, it is time for librarians to explore the opportunity to engage with emergent institutional learning analytics tools, systems, and strategies as well.  Learning analytics “is the measurement, collection, analysis, and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs.”  Essentially, learning analytics employ data to improve learning contexts and help learners succeed. Learning analytics help educators discover, diagnose, and predict challenges to learning and learner success and point the way to successful and active interventions to benefit students. 

The CLLASS project will analyze feasibility, solidify partnerships, develop work plans, and design prototypes in order to create proofs of concept that can guide academic libraries seeking to support student learning and success by connecting library data with institutional learning analytics.  The project will be enacted by participants in three task teams working together at two face-to-face meetings; progress and documentation will be shared with the academic library and higher education community via a formal white paper and conference presentation proposals.  Through this process, the CLASS project seeks to achieve four goals:

  1. cement sustaining partnerships and collaborations among academic librarians and learning analytics lynchpins, including institutional information technology and library systems professionals as well as library and higher education technology vendor communities;
  2. design three library prototypes that serve as proofs of concepts and models for future projects connecting library data with institutional learning analytics;
  3. as a part of prototype planning, develop library data profiles for a common interoperability standard, enabling the integration of library data with institutional data repositories; and
  4. recommend ways in which drafted prototypes can enable the use of library data to expand library support for student learning and success in ways that are achievable, scalable, actionable, and ethical.

The CLLASS project coalesces a diverse group of library and higher education leaders and experts to develop models for library inclusion in institutional learning analytics, anticipate strategies for bringing the models to fruition, develop technologies to support library-enabled learning analytics, and anticipate ways in which this work will increase library impact on student learning and success.  Through these actions, CLLASS will:

  • advance the role of libraries as anchors within their higher education communities,
  • enable libraries to provide indispensable data and contribute to a complete picture of institutional student learning, and ultimately,
  • facilitate student learning and success by contributing to the identification, development, and assessment of the curricular and instructional improvements resulting from learning analytics initiatives.

Data Doubles - Grant Abstract

The research project will conduct a student-centered, three-year research agenda into student perspectives of privacy issues associated with academic library participation in learning analytics (LA) initiatives. Led by the primary investigator at Indiana University-Indianapolis (IUPUI), the team consists of research collaborators at the University of Wisconsin-Madison, the University of Wisconsin-Milwaukee, the University of Illinois at Chicago, Northwestern University, Oregon State University, Indiana University-Bloomington, and a site facilitator at Linn-Benton Community College. Six scholars and practitioner experts in the areas of assessment, library analytics, diversity, and information ethics and policy will support the team as they develop research protocols and disseminate findings.

Learning analytics (LA) is the “measurement, collection, analysis, and reporting of [student and other data] for the purposes of understanding and optimizing learning and the environments in which it occurs.” With LA, institutions are more prepared to describe (what is happening?), diagnose (why did it happen?), predict (what is likely to happen?), and prescribe (what should we do about it?) student learning by identifying factors that impede or promote success. Libraries are pursuing LA insights to evaluate the impact of library services, collections, and spaces on student learning. The success of LA depends in part on an institution’s ability to connect campus information systems—including those under the purview of libraries—to aggregate and analyze student data. But as institutions continue to surface granular data and information about student life, the risk to student privacy grows. It is unclear what rights to students have in relation to the data, and there is little evidence regarding student perceptions of LA—especially when libraries are involved.

Very little research has addressed LA and student privacy issues from a student perspective, and extant research suggests that the student voice is missing from LA conversations. To the team’s knowledge at the time of this writing, no scholarship currently exists that specifically considers student perceptions of their privacy when libraries are actively leading or contributing to LA initiatives. In fact, in Connaway et al.’s OCLC-sponsored study, the authors argue that “this topic is particularly fraught in the areas of assessment and academic libraries since there is a lack of established effective practices and standards addressing the methods and contexts that may threaten the privacy of students.” Because of these indicators, the team believes there is a national need to study library LA and the privacy issues from a student perspective.

The team seeks to answer this overarching research question: How do LA initiatives align with and run counter to student expectations of privacy; and with these insights, how might libraries maximize the benefits of LA while respecting student expectations?

Three iterative research phases structure this project. During phase one, the research team will conduct preliminary interviews with students to identify themes about library participation in LA and LA generally with regard to privacy. During phase two, the research team will deploy a survey to undergraduate and graduate students at each researcher’s respective institution. In the third and final phase, each team member will run a series of scenario-based focus groups with students to explore possible applications of LA that respect and break expectations of privacy. All three phases will lead to peer-reviewed scholarship, practitioner-focused conference presentations, workshop materials, and a toolkit for informing library practitioners about student privacy and LA.

More information can be learned at the project website, https://datadoubles.org/.

Prioritizing Privacy - Grant Abstract

The University of Illinois at Urbana-Champaign is the lead applicant, in partnership with Indiana University-Indianapolis, for this Laura Bush 21st Century Librarian Community Catalysts Grant Proposal for Prioritizing Privacy: Training to Improve Practice in Library Analytics Projects.

Prioritizing Privacy is a three-year continuing education program that will train academic library practitioners to comprehensively address privacy and other related ethical implications of learning analytics projects (e.g., autonomy, agency, and trust). The training program will guide participants to explore learning analytics, privacy theory, privacy-by-design principles, and research ethics and then present participants with case studies. Participants will develop a plan for a learning analytics project prioritizing privacy protections.

The project plan will be carried out in three phases: Curriculum Design; Training and Evaluation; and, Dissemination. A team of content experts will contribute to the curriculum development and an advisory board will provide guidance and feedback. The primary deliverables of Prioritizing Privacy are: (1) face-to-face training for an estimated 200 participants; (2) online training for an estimated 200 participants; (3) an open educational resource packet consisting of the training curriculum, guidelines for facilitating the training, and recommendations for incorporating the materials into other training programs and library science courses; and (4) at least two peer-reviewed conference presentations and one peer-reviewed research publication.

As a result of Prioritizing Privacy, academic library practitioners will be better prepared to consider fully the privacy implications of library analytics projects and to improve the design of such projects in order to strengthen personal data protection practices. Participants will have expanded knowledge of the interplay and tensions between learning analytics and library values as well as improved ability to navigate these tensions. They will specific skills related to designing learning analytics projects with attention to privacy and be prepared to use various methods and tools that can be deployed to protect privacy and provide for better data management.

Prioritizing Privacy will directly train up to 400 academic library professionals. Each of these individuals will bring the knowledge and skills that they gain through the training to their workplace setting as they apply them in learning analytics projects. As library learning analytics work tends to involve teams as well as engagement with campus partners, an estimated additional 1600-2000 people will be impacted indirectly by Prioritizing Privacy. Ultimately, though, the impact of Prioritizing Privacy will be on how library learning analytics projects are designed and the resultant protections for students.

Given the “big data” nature of learning analytics projects, the impact of this is tremendous. For example, if each participant conducts a learning analytics project with only 2,500 students, which is relatively small size for a learning analytics project, that means that Prioritizing Privacy training will impact the privacy protections offered to 1 million students. The impact on students is only amplified once use of the OER packet and other deliverables from Prioritizing Privacy are considered.  

More information can be found at the project website, https://prioritizingprivacy.org/.

Jisc has undertaken learning analytics work since 2014 and has collaborated with the libraries at the University of Huddersfield and the University of Gloucestershire. More information is available at https://www.jisc.ac.uk/learning-analytics.