Skip to Main Content

Learning Analytics Toolkit

This toolkit was drafted by a Value in Academic Libraries' subcommittee on Learning Analytics and Privacy. It is meant to assist academic librarians as they consider responsibly engaging with campus learning analytics at their respective institutions.

Introduction to Issues

The following are some areas of privacy that institutions beginning to explore data analytics should consider along with some questions that can begin a fruitful dialog. These areas and questions are taken from the LIILA project. The general areas of privacy can also be further explored by referring to the extensive bibliography at the end of this toolkit. 

Anonymity, Confidentiality, and Privacy

  • What policies, statements, or requirements are relevant to discussions and decision-making with regards to confidentiality and privacy?
  • At the library level?
  • At the professional association level?
  • At an institutional level?
  • At a governmental level?
  • Do these policies, statements, requirements, or statutes require re-examination or updating?

Institutional Data Sharing and Storage

  • What elements are timeless?

Personally Identifiable Information

  • What data is being collected?
  • How long is collected data retained?
  • How secure are the underlying systems in which data is maintained?
  • What known vulnerabilities exist in the systems that maintain data?
  • How are permissions handled and who has (what level of) access to these systems?
  • Are there differences between the stated goals of learning analytics and real-world practice?
  • Are the practices and policies with regard to the gathering, use, and retention of personally identifiable information shared with the public in a transparent manner?
  • How is data in vendor-controlled systems negotiated?
  • Who owns it?
  • Who has access to it?
  • Are there additional costs associated with access or analysis?
  • What problems need to be solved?
  • What research questions need to be answered?
  • What data would librarians need to solve those problems or answer those questions?
  • How granular and identifiable might that data need to be?
    • For example, do librarians need to know that a user accessed a library resource, service, or facility?
    • Or do librarians need to know the specifics of that access (resource type? subject? title?), detailed information about a service transaction, or the precise time or place of facility use?

Data Privacy and Security

  • What data is the overarching institution, or individual units within the institution, maintaining?
  •  What systems are included?
  • What policies are involved?
  •  What practices are employed?
  • What safeguards are present?
  • What governance structures are in place?
  • Who controls collected data?
  • Who is responsible for decision-making with regard to the gathering, use, and retention of data?
  • Who has access to the data?
  • What is the process for others to gain access?
  • Institutional Data Sharing and Storage
  • What library data can help complete an institutional picture of student learning and success?
  • What data is relevant at the library level but may not be significant at the institutional level?
  • What benefits can the inclusion of library data at the institutional level provide to student learning and success?
  • Are there other advantages to consider?
  • What disadvantages might occur as a result of including library data at the institutional level?
  • What access controls does the library possess in terms of controlling the visibility and use of library data at the institutional level?
  • Which entity has more robust data security and/or policy protections, the library or the institution?

 

Risk Mitigation Practices

"LIILA participants developed an initial list of potential practices to reflect upon, which are organized below into three categories: 1) investigating current practices, embracing transparency, and educating others; 2) increasing connections and engagement at the institutional level; and 3) being parsimonious with any library data under consideration for inclusion in learning analytics:" [i]

  1. Investigate Current Practices, Be Transparent, and Educate Others

Investigate current data collection, use, security, and retention policies and real world practices within the library and among systems used by the library (i.e. campus-based, vendor-controlled).

 Uncover default settings in systems used by the library. What data is automatically logged? Are systems opt-in or opt-out by default? What happens to data if a user opts-out?

Craft transparent statements about library data collection, use, and retention for students and other library users. Provide rationales for data use. Ensure that the statements are accurate, understandable, and findable.

Educate students and other stakeholders (e.g., parents, faculty) about institutional data collection, use, security, and retention.

        2. Become Connected and Engaged at the Institutional Level

            Become involved in data governance at an institutional level.

Examine and/or improve institutional policies around the ethical collection, use, and retention of data.

Discover practices used by other institutional units engaged in collection, use, and retention of data with special attention to those with similar privacy concerns, such as student counseling services and student health services.

Investigate access to institutional data warehouses and library data storage. Who has access? At what levels? Are best practices and policies followed? Are improvements or changes needed?

 Develop shared requirements for vendor licenses and advocate for their use across institutions.

        3. Be Parsimonious with Library Data

Consider the level of granularity required for any library data shared at the institutional level.

Be parsimonious. What is the minimum necessary specificity, amount, or type of data needed to solve problems, answer questions, empower students, support institutional student learning and success initiatives, etc.?

Are specific details related to student-library interactions important and necessary to support student learning and success? If so, which ones?

If student-interaction details are unimportant or unnecessary, how can they be removed from data collection?

Data Managment

NISO Consensus Principles on User’s Digital Privacy in Library, Publisher, and Software Provider Systems and data management best practices are a key resource for academic institutions and libraries when addressing proper data handling strategies. Briney (2019) evaluated a number of recent studies performed by academic libraries and found an inconsistent application of these best practices. [i]

The NISO principles addressed in the study included: 

Shared Privacy Responsibility

Transparency and Facilitating Privacy Awareness

Security

Data Collection and Use

Anonymization

Options and Informed Consent

Sharing Data with Others

            Access to One’s Own Data

[i] Briney, K.A., 2019. Data Management Practices in Academic Library Learning Analytics: A Critical Review. Journal of Librarianship and Scholarly Communication, 7(1). DOI: http://doi.org/10.7710/2162-3309.2268