Case Study: Knowledge Base for SVSU Chatbot

Problem

C

oop , the chatbot at Saginaw Valley State University (SVSU), offers the University a unique form of internal communication with students. “Coop is a great tool for sending updates and important information, providing a friendly connection, and giving a face to SVSU’s mascot,” says Cara Deschermeier, Assistant Director for the Academic Advisement Center, who has been instrumental in personifying SVSU’s mascot through the chatbot tool.

Nicholas Wagner, Executive Director of Institutional Effectiveness at SVSU, believes the mascot is a convenient way to communicate with students amid the information overload they face on a daily basis. “Students in particular are getting inundated with so much information,” Wagner says. “They are receiving emails, notifications on mySVSU, and even phone calls. It’s so much easier when they’re able to chat back and forth with someone just like they would in a text message. It seems to resonate well with a lot of students.”

In 2019, the chatbot served primarily first-year students and new transfer students. However, when the pandemic hit in March 2020, the chatbot team quickly expanded to all degree-seeking undergraduate students, and now the chatbot reaches the vast majority of SVSU students. The chatbot can target students by year of study, major, registration status, and more so that students receive only the updates that are relevant to them. “We use targeted messaging,” explains Deschermeier. “Sometimes I send a message that relates only to students of a particular major, only to students who live on campus, or only to students who haven’t made a payment. We can easily filter out students to whom the message doesn’t apply. I want to make sure that students are receiving only the information they need.”

Deschermeier and Wagner became interested in launching a chatbot at SVSU after looking at some of the outcomes achieved by different universities that have used a chatbot, especially outcomes related to retention initiatives and financial literacy. They say the bot has cut down on students’ frustration because they no longer have to search the web or look up a phone number when they have a question. Students value the simplicity, ease of use, and their familiarity with a texting-based tool. Moreover, Coop has alleviated pressure for a variety of offices that previously allocated more staff hours to answering students’ phone calls.

Coop’s knowledge base has doubled since Deschermeier got involved with the project. “Coop is built on artificial intelligence,” she explains. “When a student texts the chatbot, it uses its knowledge base to try to find the right answer based on the keywords in the student’s question. If it finds a match within a range of 80% to 100% accuracy, it will give an answer. If it can’t find a match, the bot will just say, ‘I’m not sure. Let me get a human to help.’ In that case, the question is flagged, so I have to go in and train the bot on how to answer it. I not only give the student the right answer, but I get the item in Coop’s knowledge base so that the next time a student asks that question, the process will be automatic.”

Undergraduate students who are seeking a degree at SVSU are automatically enrolled in the chatbot service. However, they receive directions on how to opt out with their first message from the bot. The opt-out rate has stayed very low—between 6% and 8%—suggesting that most students find the chatbot tool easy and enjoyable to use.

Team

Cardinal Solutions collaborated with the chatbot team to launch Coop to the SVSU community. “We relied on a lot of the students in Cardinal Solutions to test the bot,” says Wagner. “They posted questions to help us build up our knowledge base.” In addition to testing, the Cardinal Solutions team members created a series of Gifs for the chatbot, and they checked information for accuracy and timeliness.

Joe Lourenco, a Graphic Design major, worked with Cardinal Solutions on the chatbot project during two semesters in 2019. Lourenco was responsible for creating new Q&A entries based on popular questions students asked the chatbot that were not yet included in Coop’s knowledge base. “In Cardinal Solutions, I learned how to take direction on a project, which will benefit me in the job market,” says Lourenco, who plans to pursue a career path in graphic or web design.

Lourenco answered some questions by looking up information on the SVSU website, while others were common knowledge to him. When populating Coop’s answer bank, he remained mindful of best practices for maximum character limits on text messages and aware that students like to use the tool to get accurate information quickly and easily.

Lourenco says the chatbot is beneficial because it serves to give the mascot a voice. Coop also saves staff members hours of phone calls per semester and offers a convenient alternative to SVSU students, some of whom may feel less nervous sending a text message than making a phone call.

Client: (SVSU Chatbot)

SVSU Chatbot

David M. Callejo Pérez
Associate Provost

Student Specialists: Research & Chatbot Proposals

Garrett Shovan
Graphic Design Major

Ashley Ames
Graphic Design Major

Kevin Markuszewski
Graphic Design Major

Student Specialists: Chatbot Knowledge Base

Skylar R. Miller
BFA Major

Joseph Lourenco Jr.
Graphic Design Major

Jessica Larsen
Graphic Design / Marketing Major

Noor Alam
Computer Science Major

Student Specialists: GIFs/Video

Alexis M. Stacy
Graphic Design Major

Staff/Faculty Advisor(s):

Janet Rentsch
Director Sponsored Programs

J. Blake Johnson
Professor of Art / Graphic Design

Solution

The chatbot team uses a conversational, peer-to-peer tone and fun images such as Gifs to help personify the SVSU mascot. “I have to think about it in the sense of, I’m actually texting my friends,” says Deschermeier, “not that I’m SVSU emailing students. That’s not what students want to read in a text message from the chatbot. They’re turned off by messages with an overly formal tone; they don’t even bother looking at them. I want Coop to act like he’s your friend, like he’s at your level, but he’s the knowledgeable friend. He can get you access to everything.”

Wagner credits Deschermeier for the tone of Coop’s messaging, which makes first-year students in particular feel comfortable seeking Coop’s help when they experience academic, financial, and/or emotional challenges in their transition to college. He has found that when users need someone or something, they don’t hesitate to ask the chatbot. “Coop isn’t only an informational resource,” explains Wagner. “He is also part of a student’s peer group. The chatbot is an important tool in light of the struggles students face while acclimating to higher ed.”

Although Coop most frequently sends reminders about key events, initiatives, and deadlines, the chatbot occasionally checks in with students about how they are doing overall. “I’ve definitely talked to some students who only want the chatbot to send them the important information,” says Deschermeier, “but a majority of the students like the fact that Coop checks in. He isn’t only about ‘Here’s the information you need.’ He is also about ‘I’m your friend. I’m here for you. How are things going?’”

Deschermeier recently sent out a message asking students how happy they were feeling, for example. She followed up with the students who responded that they were unhappy, sharing mental health resources with them. Although Deschermeier recognizes that she cannot serve as a professional resource in every area in which students are struggling, she finds it rewarding to connect them with the right people on campus who can serve as those resources: “I don’t have counseling experience, but I do have friendship experience.”

SVSU offers a variety of support services for students, including the Mental Health and Wellness Center, SVSU’s Employee Assistance Program, the Student Counseling Center, the Academic Advisement Center, the Campus Financial Services Center, and multiple tutoring centers.

Result

Among the most important roles Coop plays, he reaches out to degree-seeking students who have not registered for courses within the expected timeframe. “The bot is able to reach out to students who are not registered yet,” says Deschermeier. “This is what contributed to our increase in retention, which was absolutely amazing. Last year I sent out a non-registered message to upperclassmen specifically since we had expanded to them. I received a 50% response rate out of sending messages to 670 students.”

The bot also improves students’ financial literacy. Through Deschermeier’s efforts, Coop has reached out to students who still need to fill out the FAFSA, for whom a tuition payment deadline is approaching, and who may be eligible for grant opportunities such as the CARES Act.

“The chatbot isn’t some type of revelation,” says Wagner. “It’s just a normal conversational tool, but people have responded better to text messages, and we get great data back from those exchanges. When we talk now about the best way to reach students, the chatbot always rises to the top.” ■