Learning analytics is the measurement, collection, analysis and reporting of data about students and their contexts, for the purposes of understanding and improving the quality of learning and teaching and the environments in which these actions occur. 

Student data for learning analytics is defined as that data which is created, collected or stored by the university and which could pertain to students’ academic activity. 

Learning analytics can be applied to individual students as well as to defined groups of students (identified via combinations of characteristics and/or study behaviours), and to academic courses or programs (through observing behaviours and/or results of students in courses or programs).

Context

At the University of Saskatchewan, our approach is to gather information about our students’ academic performance as well as demographic and activity data that could pertain to learning success.  In line with the institution's learning charter commitments, this data will be used to connect students with appropriate academic supports and services, to personalize their learning experience, and to improve our academic programs.

Learning Analytics work is conducted under the leadership of the Office of the Vice-Provost Teaching and Learning in partnership with Information and Communications Technology, Student and Enrolment Services, and the Gwenna Moss Centre for Teaching Effectivenes. 

Policies and procedures to ensure proper use of student data and to protect learner privacy are being put in place. 

A learning analytics advisory body, consisting of faculty, staff, and students, is being put together.  This advisory body will help to guide policy and procedures, discuss innovative projects, and assist with communication to the university community and beyond.

Projects

Student Advice Recommender Agent (SARA)

SARA is a computerized Student Advice Recommender Agent that has been delivering personalized advice messages since fall 2014 to students in BIOL 120.  Based on each student’s individual context, preferences, achievement levels, and academic activity, SARA attempts to craft messages that can offer each student in this large multi-section course some useful academic advice every week.  SARA collects information about students’ online activity in BIOL 120 as well as their grades on quizzes, labs and assignments in order to connect each individual with the most helpful resources to maximize overall performance in Biology.  

Every week in each student’s BIOL 120 Blackboard site a new message from SARA is presented.  The message may point you toward online resources, peer learning opportunities, academic advising options, and study tips.  By clicking on the SARA link in the left-hand menu of Blackboard students can look back at all of SARA's messages to them in the term so far.  If they wish to explore further, students can also see what kinds of messages SARA is sending to other students.

SARA is a software agent and sometimes SARA's advice might not be perfect – perhaps more suitable advice could be given.  For that reason, students can browse the advice SARA gives to others – or they can ask a question to SARA if they so choose.  When questions are sent to SARA, a real human instructional support specialist in Biology will respond.

SARA is an innovative project being piloted for the first time in 2014 at the University of Saskatchewan in Biology 120.  The SARA project team includes experts in Biology, Computer Science, Education, and Artificial Intelligence – who are all working together in partnership with the Department of Biology and Student Learning Services to create a better learning experience for you.

For more information about the SARA project see:

RibbonTool Vizualization

In partnership with universities associated with the BayView Alliance tools are being developed to track the movement of students in academic programs with an interactive visualization tool.  Using this tool, academic units can discover patterns and flows of students as they move from year to year in the university.  Student attrition and retention can be tracked, as well as student migration from one program to another.  Filters can be applied to determine differences and effects associated with demographic variables to discover, for example, whether the flows through a program are the same for female versus male students. 

Data sets for various clusters of students can be developed for exploration with this innovative tool. Administrators can use this tool to identify issues, concerns, and possible solutions to problems associated with student success and academic choices.

For more information visit:

Early Alert Project

Much has been written about the importance of early identification of students at academic risk as a way to alleviate problems with student retention.  It is important to encourage selected students to connect with academic advisors before their academic future is in peril.  Early alert solutions, such as Elucian Pilot use activity data to identify students who are not engaged or who are falling behind their classmates.  Predictive models based on learner demographics and known risk factors can also help identify students who might benefit from some additional support or advice.

The College of Engineering is working with the Learning Analytics group to experiment with early alert tools and technologies.  Tracking first-year student academic activity and early success (or lack of success) has helped College advisors to alert and reach out to students. At this time the university is considering the purchase of a commercial early alert solution.  Our initial pilots to date rely on locally developed statistical models and data integration from various university sources.

For more information about early alert systems and the early alert project, contact Sean.maw@usask.ca or Jim.greer@usask.ca

NSSE Analysis

Every three years the University of Saskatchewan participates in the National Survey of Student Engagement, NSSE.  The rich data available from NSSE have been underutilized in recent years.  An initiative to help turn the information derived from NSSE into actionable change is being led by the Learning Analytics team. 

College-specific NSSE data is particularly useful to units planning program revisions or teaching innovations.  Data-informed and evidence-based decisions are easier to take with greater access to data.

The chart above shows just a few of the many information tidbits available in NSSE for the university overall and for many of our colleges.

Know Your Class Infographic

Beginning in the Fall 2016 term, this infographic has been available to instructors inside Blackboard Learn courses. It allows instructors to get a general sense of the demographic makeup of their class sections each term. It has been designed to provide this overall demographic information while protecting individual student privacy. To further maintain student anonymity, the infographic will only display relatively proportionally-sized bars without a vertical-axis and it will only display for class sections with 20 or more students enrolled. This type of information is helpful when designing and planning courses, especially during the Learner Analysis.

An example of how this information could be used by instructors is that they may learn that they have a large proportion of students registered from two colleges. In this case, it would be helpful to include examples and cases relevant to students in each of these colleges.

Privacy

Student data are securely stored in the University Data Warehouse. Student data may be created by faculty, instructional staff, advisors, administrators, administrative and instructional systems, and by students themselves.  Student data can include academic performance data, advising data, survey data, admissions and demographic data as well as activity data associated with university academic and administrative services (such as learning management systems, student response systems, network services, and card swipe systems). In addition, student data may include derived interpretations based on statistical models and patterns. Student data may be used for learning analytics with permission of the University Data Steward.

Student data for learning analytics does not include contents of email messages or other personal communications, discussion forum or social media postings, notes or written materials for which the student could claim copyright.  Data of this type will not be used for learning analytics work without expressed written consent of the student.

Instructors in courses are not privy to their students’ personal or demographic data, nor to their students’ academic history outside the course or predictions of success within the course unless permission is given by the student.  Instructors are not to discuss or disclose personal or academic information about students to other students or instructors. As such, instructors are limited in the scope of learning analytics work that they can do with their own students. 

Academic advisors are granted special access to personal, demographic and academic history data about students, and they follow a strict code of conduct to ensure that students are protected. Academic advisors frequently participate in learning analytics work.

Get Help

Office of the Vice-Provost, Teaching and Learning

Jim Greer in the office of the Vice-Provost, Teaching and Learning can provide consultation and support if your college or department is interested in using learning analytics to improve student outcomes.

Project Team

Ryan Banow
Instructional Design Specialist


Stephanie Frost
Coordinator - Online Support


Jim Greer
Senior Strategist for Learning Analytics and Professor of Computer Science


Craig Thompson
Learning Analytics Data Analyst