The following libraries have made great strides working with library learning analytics.
Reid Memorial Library
Dennis Krieb, Ed.D., Director, Institutional Research and Library Services | email: dkrieb@lc.edu | phone: 618-468-4300
To better understand any correlational relationships between student success and library services/collections, Lewis & Clark Community College has developed a technology platform for capturing student usage data within the library. Three components of the library are tracked: reference questions, attendance of library instruction classes, and checking-out library material. How it works. Any student using one of these three library services is asked if they would be willing to share their student ID. This serves the purpose of providing an informed consent. If the student agrees to share their student ID, the number is scanned and sent to a data warehouse. Within the data warehouse, course, grade, retention, and completion data from the Student Information System are connected with library usage data. Using a reporting tool, student success measures and be correlated with library usage data. In terms of the technology, student IDs are captured using a scheduling program called SARS Trak. SARS Trak then passes the student ID to a Blackboard Analytics data warehouse. The Blackboard Analytics data warehouse connects to an Ellucian Student Information System (SIS). A Pyramid Analytics reporting tool is used to query the data warehouse. Pyramid is able to perform calculated measures such course grade averages, student GPA, and retention rates with student use of library services. Lastly. No personally identifiable student information is shared. All data are kept within Lewis & Clark’s SIS. All data are only accessible by college personnel with permitted authority.
The Open University Library
Selena Killick, Associate Director, Student and Academic Services | email: selena.killick@open.ac.uk | phone: +441908 659209
Learning analytics is a key organizational strategic driver at The Open University and the institution is known as a leader in this research field internationally. In 2014 the University worked in partnership with the Student Association to develop and agree on an Ethical use of Student Data for Learning Analytics Policy. In line with the wider organizational strategy, the Library embarked upon research into Library Learning Analytics in 2015, initially focusing on the relationship between library use and student performance and retention. The study analyzed online library resource data from access logs from the EZproxy and OpenAthens systems. A data set of 1.7 million online resource accesses was combined with student success data for around 90,000 undergraduate students and a series of analyses undertaken. The study found a pattern where students who are more successful are accessing more library resources. In 2017 the team went onto explore the relationship between attendance at library tutorials and student performance, combining Adobe Connect attendance data with student success data. Again, the study identified that students who attend library tutorials are more successful in their studies. The research in both areas has been repeated annually and we have continued to find a correlation between student attainment and engagement with the Library. The research is communicated with the academic faculty to inform service planning and curriculum developments. Our work is continuing in this area as we explore the opportunities our data provides us to support student success.
Visit The Open University's Library Use and Student Success webpage for additional information.
University of Michigan Library
Ken Varnum, Senior Program Manager and Discovery Strategist | email: varnum@umich.edu | phone: 734-615-3287
As part of the University of Michigan Library’s effort to be transparent about the ways it collects, analyzes, and uses data that connects U-M users and library resources, the library participates in the campus-wide ViziBLUE information site. ViziBLUE, a service of U-M’s Information Technology Services, provides a clearinghouse for campus units to describe the way they collect, store, share, and use data about U-M affiliated individuals.
The library has also participated in two IMLS-funded grants related to learning analytics: Connecting Libraries and Learning Analytics for Student Success (CLLASS), which produced a library profile for the Caliper learning data standard, and the Library Learning Analytics Project, which developed a proof of concept for collecting and analyzing library transaction data from a range of sources and connecting it with other campus data.
University of Minnesota - Twin Cities University Libraries
Shane Nackerud, MLIS, Interim Director, Content Services | email: snackeru@umn.edu | phone: 612-625-7880
Since 2011, The University of Minnesota Libraries have worked in this area through our Library Data and Student Success project (LDSS) which attempted to correlate student success measures such as GPA, retention, and 4 year graduation rates to library use. The UMN Libraries have also been enthusiastic participants in a number of grants spearheaded by Megan Oakleaf, including the Library Integration in Institutional Learning Analytics (LIILA) grant, and the Connecting Libraries and Learning Analytics for Student Success (CLLASS) grant which produced a Library Profile for the Caliper learning data standard. The UMN Libraries are now investigating using the Library Profile to share library data with institutional learning record stores for participation in campus learning analytics projects.
J. Murrey Atkins Library
Becky Croxton, Ph.D., Head of Assessment | email: racroxto@uncc.edu | phone: 704-687-0480
Anne Cooper Moore, PhD., Dean | email: amoor168@uncc.edu | phone: 704-687-0145
To assess which engagement factors significantly contribute to student success at UNC Charlotte, a large, public research university in the southeastern United States, the university library, along with representatives from Academic Affairs, Student Affairs, and other academic support units across campus have agreed to contribute their co-curricular and extracurricular student data to a repository that is enabling multifaceted and evolving longitudinal study. This joint project, led by the library, is allowing library and university leaders to identify key resources, services, and activities within their units that are positively associated with student success. Alignment of student engagement data with measures of student success not only involves identifying key student success and engagement metrics, but also requires careful consideration and protection of student privacy. The findings from this study are helping the library and other support units and services across campus to help students succeed and graduate.
This project is part of an ongoing, longitudinal study of undergraduate student engagement and success data of students who matriculated in summer/fall 2012 through the present. The dataset includes yearly student engagements with 17 different co-curricular and extracurricular partners across campus at the "type of activity" level of specificity. The library collaborates with the university's Office of Institutional Research to align the engagement data with pre-college data (e.g., HS GPA, ACT/SAT Scores, # of Incoming Credits), demographic data (e.g., Race, Ethnicity, Pell Status, College of Enrollment, On/Off Campus Residence, Learning Communities), and measures of success (e.g., Year-to-Year Retention, GPA, and Graduation Rates). Recent research publications related to this work include:
UW Oshkosh Libraries
Joe Pirillo, MLIS, M.Ed., Information Literacy/Online Learning Librarian | email: pirilloj@uwosh.edu
Since the Fall of 2017, Polk Library at the University of Wisconsin Oshkosh has been integrated into our University's Navigate platform (An EAB product). Early on, Polk Library demonstrated the good fit between library services and the value in contributing to our Institution's learning analytics platform. Through EAB's Navigate, we log such things as reference transactions, workshops and all other types of instructions events. We have also, and continue to, work with our institution's Office of Institutional Research in order to identify trends and correlations between students, retention, and section participation. This information has helped us plan and make adjustments to our information literacy program, in order to be of greater service to our students. As of spring of 2023, Polk Library is investigating the use of Navigate’s Intervention Effectiveness tool, which will allow Polk Librarians to further evaluate the outcomes of their interventions.
University of Wollongong Library
Margie Jantti, BA LIS, MBA, Director of Library Services | email: margie@uow.edu.au | phone: +61418445791
An exploration of ‘big data’ generated by the University of Wollongong Library (UWL), and the ability to con-join Library usage data with other institutional data to create relational datasets sets, resulted in the Library Value Cube, initiated in 2010. UWL, through the development of the Value Cube, had established the necessary architecture for the full-scale, systematic capture and reporting of students’ use, i.e. logins or borrowing of Library resources to test correlations with academic performance (grades). The UWL Cube demonstrated a positive and persistent correlation in student use of Library information resources and improved academic performance outcomes as evidenced in their grades. The reporting cycle for the Value Cube is, however, sessional as the original intent was to produce reports associated with grades. Regardless, Library usage data is uploaded weekly and can be decoupled from grades. This means it could be readily harvested and contribute ‘near real-time data’ mapped to teaching weeks. Library Cube data was used for the early iterations of learning analytics dashboards. Over time, reporting has become more sophisticated and embedded within the learning management system platform (Moodle), with an array of reports and vizualisations available to teaching staff and students. Library data in the form of interaction with etexts remains an element of monitoring student activity within the learning management environment. This remains an important and significant milestone in terms of how interaction with library resources can contribute to the ‘whole of student’ learning experience and the success goals of the institution.
[i] Oakleaf, M. Library Integration in Institutional Learning Analytics (LIILA. Nov. 15, 2018), available online at: https://library.educause.edu/~/media/files/library/2018/11/liila.pdf
Share your feedback and suggestions
Please let us know how we can improve the toolkit via this anonymous feedback form.
Library Learning Analytics Project - Grant Abstract
We propose a project centered at the University of Michigan (U-M) to study how libraries impact learning. We define learning as the process of acquiring knowledge and/or skills through formal study or instruction (course instruction), or experientially through laboratory research or clinical activities (research). However, learning is also the outcome of this knowledge acquisition process, including the ways or means by which this knowledge is disseminated to society (publishing). We use analytics – the discovery, interpretation, and communication of meaningful patterns in data (https://en.wikipedia.org/wiki/Analytics) – to address the research question: How does the library impact learning, specifically in the areas of course instruction, research, and publications? Libraries should strive to improve learning outcomes for the communities they serve, just like how healthcare institutions should ideally improve individuals’ health outcomes. This type of work is best described as library learning analytics (LLA), which entails embedding library data within institutional learning analytics ecosystems, and is guided by a more encompassing definition of learning that extends beyond the classroom. Our goal is to establish the groundwork for a group of diverse institutions to use a common analytics framework. The three-year project, which will commence in June 2018, has two main goals as follows:
Goal 1: To understand how the U-M library impacts learning, specifically in the areas of course instruction, research, and publications. We plan to achieve this goal through modeling and analyzing both deidentified and identifiable library use data including: website server logs, library catalog server logs, proxy server logs, circulation history and related data, and campus status & affiliation data. These datasets are linkable through both strong and unambiguous identifiers that are unique to each individual user, as well as IP addresses and timestamps. First, using deidentified data, we aim to apply clustering algorithms to identify typologies of library users on the basis of library interactions e.g. user type A (high degree of use), user type B (low degree of use), etc. Second, we aim to replicate the clustering analysis using the identifiable data, using the deidentifiable clustering findings for robustness checks. Third, our goal is to use clusters from identifiable data in the sequence mining, and predictive and prescriptive modeling of the links between learning outcomes and library user types. For privacy reasons, results will be shared using only aggregated and anonymized data.
Goal 2: To develop tools, scripts, and protocols that serve the library analytics needs of our community of project advisory group (PAG) institutions. We plan to achieve this goal through two modes of engaging with the PAG community. First, we plan to develop and share with PAG institutions data dictionaries (names, definitions, and attributes about data elements that are in a database) of the datasets we will be using in the project. This will facilitate data harmonizing across PAG institutions (and with U-M data) and thus smooth the exchange of ideas and best practices over the course of the project. We will maintain a repository of tools and scripts developed for data cleaning and database construction and grant PAG members early access to this repository. We also plan to regularly update PAG members via email on the latest status of data management and analysis activities with a view to making it easier for them to voluntarily replicate our work or perform their own LLA studies. Thrice a year, we will hold virtual research meetings with PAG members for more thorough debriefs of project activities. Lastly, throughout this process we will stay in regular phone and email contact with PAG members in order to help them resolve replication issues as they arise. Second, we will give PAG members access to aggregated and anonymized findings (from Goal 1, using U-M data) through a webbased dashboard running on data that will be located on a secure virtual data enclave. Further, in addition to the dashboards, we will hold annual PAG Workshops where member representatives will be tutored and immersed in training tasks to ease voluntary replication of the project, but using their own data.
Impact: This project will provide guidance to PAG institutions and other libraries on how to best design and implement empirical, holistic LLA studies of the links from library usage, to learning outcomes such as course instruction, research, and publications. The project will produce a set of tools, scripts, and protocols that will be freely available to all libraries. The study will serve as a template for other libraries with respect to: 1) studies that collect and store library use data with individual identifiers while maintaining the privacy of individuals; 2) designing and implementing a holistic LLA study of the link from library use to multiple learning outcomes, and; 3) creating a secure cyberinfrastructure (data repository, virtual enclave, and dashboard) for LLA research that facilitates collaboration in a community of diverse institutions in the US and Canada.
More information can be learned at the project website, https://libraryanalytics.org/.
Share your feedback and suggestions
Please let us know how we can improve the toolkit via this anonymous feedback form.
Connecting Libraries and Learning Analytics for Student Success (CLLASS) - Grant Abstract
CLLASS is a one-year grant designed to perform preliminary planning activities to pioneer the integration of library data in institutional learning analytics and develop detailed proofs of concept and models to guide academic libraries preparing to engage in this emerging and important use of data to support student success. The lead applicant, Syracuse University, is joined by advisory group members and project participants from ACRL, Blackboard, CNI, DePaul University, EDUCAUSE Learning Initiative, IMS Global Learning Consortium, Jisc, Lewis and Clark Community College, OCLC, Susquehanna University, the University of California, Berkeley, the University of Michigan, the University of Minnesota, and Unizin.
The foremost purpose of higher education is to educate students, and a key component of any educational endeavor is assessment. As active contributors to the educational mission of their institutions, academic librarians use assessment to expand student access to learning; ensure students are able to persist and attain their goals; scaffold student experiences to aid attainment of independent learning capacity; and develop productive self-awareness, metacognition, and self-actualization in a variety of contexts, including their immediate learning environments, the broader community, and the world around them. Now, as institutions of higher education commence and commit to the next wave of assessment capability in the form of learning analytics initiatives, it is time for librarians to explore the opportunity to engage with emergent institutional learning analytics tools, systems, and strategies as well. Learning analytics “is the measurement, collection, analysis, and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs.” Essentially, learning analytics employ data to improve learning contexts and help learners succeed. Learning analytics help educators discover, diagnose, and predict challenges to learning and learner success and point the way to successful and active interventions to benefit students.
The CLLASS project will analyze feasibility, solidify partnerships, develop work plans, and design prototypes in order to create proofs of concept that can guide academic libraries seeking to support student learning and success by connecting library data with institutional learning analytics. The project will be enacted by participants in three task teams working together at two face-to-face meetings; progress and documentation will be shared with the academic library and higher education community via a formal white paper and conference presentation proposals. Through this process, the CLASS project seeks to achieve four goals:
The CLLASS project coalesces a diverse group of library and higher education leaders and experts to develop models for library inclusion in institutional learning analytics, anticipate strategies for bringing the models to fruition, develop technologies to support library-enabled learning analytics, and anticipate ways in which this work will increase library impact on student learning and success. Through these actions, CLLASS will:
Share your feedback and suggestions
Please let us know how we can improve the toolkit via this anonymous feedback form.
Data Doubles - Grant Abstract
The research project will conduct a student-centered, three-year research agenda into student perspectives of privacy issues associated with academic library participation in learning analytics (LA) initiatives. Led by the primary investigator at Indiana University-Indianapolis (IUPUI), the team consists of research collaborators at the University of Wisconsin-Madison, the University of Wisconsin-Milwaukee, the University of Illinois at Chicago, Northwestern University, Oregon State University, Indiana University-Bloomington, and a site facilitator at Linn-Benton Community College. Six scholars and practitioner experts in the areas of assessment, library analytics, diversity, and information ethics and policy will support the team as they develop research protocols and disseminate findings.
Learning analytics (LA) is the “measurement, collection, analysis, and reporting of [student and other data] for the purposes of understanding and optimizing learning and the environments in which it occurs.” With LA, institutions are more prepared to describe (what is happening?), diagnose (why did it happen?), predict (what is likely to happen?), and prescribe (what should we do about it?) student learning by identifying factors that impede or promote success. Libraries are pursuing LA insights to evaluate the impact of library services, collections, and spaces on student learning. The success of LA depends in part on an institution’s ability to connect campus information systems—including those under the purview of libraries—to aggregate and analyze student data. But as institutions continue to surface granular data and information about student life, the risk to student privacy grows. It is unclear what rights to students have in relation to the data, and there is little evidence regarding student perceptions of LA—especially when libraries are involved.
Very little research has addressed LA and student privacy issues from a student perspective, and extant research suggests that the student voice is missing from LA conversations. To the team’s knowledge at the time of this writing, no scholarship currently exists that specifically considers student perceptions of their privacy when libraries are actively leading or contributing to LA initiatives. In fact, in Connaway et al.’s OCLC-sponsored study, the authors argue that “this topic is particularly fraught in the areas of assessment and academic libraries since there is a lack of established effective practices and standards addressing the methods and contexts that may threaten the privacy of students.” Because of these indicators, the team believes there is a national need to study library LA and the privacy issues from a student perspective.
The team seeks to answer this overarching research question: How do LA initiatives align with and run counter to student expectations of privacy; and with these insights, how might libraries maximize the benefits of LA while respecting student expectations?
Three iterative research phases structure this project. During phase one, the research team will conduct preliminary interviews with students to identify themes about library participation in LA and LA generally with regard to privacy. During phase two, the research team will deploy a survey to undergraduate and graduate students at each researcher’s respective institution. In the third and final phase, each team member will run a series of scenario-based focus groups with students to explore possible applications of LA that respect and break expectations of privacy. All three phases will lead to peer-reviewed scholarship, practitioner-focused conference presentations, workshop materials, and a toolkit for informing library practitioners about student privacy and LA.
More information can be learned at the project website, https://datadoubles.org/.
Share your feedback and suggestions
Please let us know how we can improve the toolkit via this anonymous feedback form.
Prioritizing Privacy - Grant Abstract
The University of Illinois at Urbana-Champaign is the lead applicant, in partnership with Indiana University-Indianapolis, for this Laura Bush 21st Century Librarian Community Catalysts Grant Proposal for Prioritizing Privacy: Training to Improve Practice in Library Analytics Projects.
Prioritizing Privacy is a three-year continuing education program that will train academic library practitioners to comprehensively address privacy and other related ethical implications of learning analytics projects (e.g., autonomy, agency, and trust). The training program will guide participants to explore learning analytics, privacy theory, privacy-by-design principles, and research ethics and then present participants with case studies. Participants will develop a plan for a learning analytics project prioritizing privacy protections.
The project plan will be carried out in three phases: Curriculum Design; Training and Evaluation; and, Dissemination. A team of content experts will contribute to the curriculum development and an advisory board will provide guidance and feedback. The primary deliverables of Prioritizing Privacy are: (1) face-to-face training for an estimated 200 participants; (2) online training for an estimated 200 participants; (3) an open educational resource packet consisting of the training curriculum, guidelines for facilitating the training, and recommendations for incorporating the materials into other training programs and library science courses; and (4) at least two peer-reviewed conference presentations and one peer-reviewed research publication.
As a result of Prioritizing Privacy, academic library practitioners will be better prepared to consider fully the privacy implications of library analytics projects and to improve the design of such projects in order to strengthen personal data protection practices. Participants will have expanded knowledge of the interplay and tensions between learning analytics and library values as well as improved ability to navigate these tensions. They will specific skills related to designing learning analytics projects with attention to privacy and be prepared to use various methods and tools that can be deployed to protect privacy and provide for better data management.
Prioritizing Privacy will directly train up to 400 academic library professionals. Each of these individuals will bring the knowledge and skills that they gain through the training to their workplace setting as they apply them in learning analytics projects. As library learning analytics work tends to involve teams as well as engagement with campus partners, an estimated additional 1600-2000 people will be impacted indirectly by Prioritizing Privacy. Ultimately, though, the impact of Prioritizing Privacy will be on how library learning analytics projects are designed and the resultant protections for students.
Given the “big data” nature of learning analytics projects, the impact of this is tremendous. For example, if each participant conducts a learning analytics project with only 2,500 students, which is relatively small size for a learning analytics project, that means that Prioritizing Privacy training will impact the privacy protections offered to 1 million students. The impact on students is only amplified once use of the OER packet and other deliverables from Prioritizing Privacy are considered.
More information can be found at the project website, https://prioritizingprivacy.org/.
Share your feedback and suggestions
Please let us know how we can improve the toolkit via this anonymous feedback form.
Jisc has undertaken learning analytics work since 2014 and has collaborated with the libraries at the University of Huddersfield and the University of Gloucestershire. More information is available at https://www.jisc.ac.uk/learning-analytics.
Share your feedback and suggestions
Please let us know how we can improve the toolkit via this anonymous feedback form.
