Name: From Detection to Dialogue: What Two Institutions Learned About AI, Writing, and Trust
Uploaded: 2026-06-09T18:30:09.428Z
Duration: 1 h 29 min 8 s
Description: From Detection to Dialogue: What Two Institutions Learned About AI, Writing, and Trust

Transcript for "From Detection to Dialogue: What Two Institutions Learned About AI, Writing, and Trust": The chimes of a cock ticking on the tabletop. Will I remember that sound? The breaking of a bus driver come. Welcome, folks. As I attempt to share my screen again, apologies. My name is Cliff Archie. Welcome on behalf of the Superhuman team. For those of you who don't know Superhuman, you are forgiven. Superhuman is the now parent brand of Grammarly. We acquired Superhuman Mail, which is an AI first email client about a year ago, and we took their name as the overarching company. But for those of you who are likely here for Grammarly and who know it well, fear not. Grammarly is not going anywhere. Grammarly is still very much our most important product in our evolving portfolio. And today, we're gonna talk about this journey from detection to dialogue. I'm joined by two very wonderful guests, doctor Jenny Billings from Roan Cabarrus Community College in North Carolina and doctor Brian Harfy from the University of Florida. I will introduce them in more detail in just a second. But before I do, just some housekeeping items to take note of. I think everybody, it looks like, has discovered the q and a function. You can feel free to, raise any particular questions that you have in that q and a panel. Looks like we have folks from all over the globe today, so thanks everybody for joining us. You can also ensure that this meeting will be recorded. This presentation will be recorded and shared after the fact. So if you miss anything and you wanna share it out, feel free to do so. And, last but not least, thank you for submitting a lot of really good questions in advance when you signed up for this webinar. We will be saving time for those questions, very likely at about forty five, past the top of the hour, fifteen till the the next hour, depending on where you're joining us from. But feel free to address any additional questions that come up as you hear Brian and Jenny's wonderful wisdom, throughout the course of this conversation. So, just to get into a little bit of a roadmap from what we'll cover today, we're gonna start with kind of the questions that instructors have been asking and the evolution of those questions since the advent of generative AI in 2023 over the last few years. We'll then get Brian and Jenny's perspectives on how they are tackling those particular questions and how they've gone on similar journeys, for the idea of detection and student learning in the age of AI. Then we'll get into some data that comes from both of their implementations at their respective colleges and then discuss potential next steps. As we get into just the context to set the stage for what, Brian and Jenny will share with us today, I think it's important to go back in time to November 2022, something we often do when we talk about AI in education, and talk about the initial questions that instructors were rightfully asking at the time. The initial way was really based around this idea of we have this new technology that is incredibly disruptive to how we have traditionally run assessments and how we've traditionally taught writing. So the first question was understandably, did AI write this thing that I am now reviewing as an instructor, or was it written by my student authentically? And that is a very black or white question. And it was one that was obviously the the lowest level question that you could and should ask at the time when all of this was being very disruptive in the space, but it was a black and white question that lacked a black or white solution. And so increasingly, while plenty of professors and instructors are still asking that question, over time, more and more started asking the question of, did the student use AI in this particular assignment? And while that seems like the same question just phrased differently, it's a meaningfully different point of emphasis. It's a question from a binary of did AI write this to how much AI was used, if any, at all, and why did they potentially use AI to write this particular assessment? Was this in line with my particular instructional guidelines? Was this in compliance with how my department or my institution advises AI being used? It's a more nuanced question, but it is still one that is in the Maslow's hierarchy of needs and the Bloom's taxonomy a little bit lower down the rung, and it doesn't get at the core question that increasingly instructors will be asking as we continue down this AI native world we're living in, which is ultimately, did the student learn? And it's interesting to think about how cyclical that question is. Obviously, if you're an instructor, you work in education, this has been the core question that you've been asking your entire careers. It is an age old question, but how we answer it is very different in 2026 and moving forward than it used to be pre AI. It is one that requires really high quality tooling and sophistication and a willingness from instructors to actually ask hard questions and dig under the covers a little bit from beyond just reading the output that they are being handed in by students. And so, in order to accommodate that journey for instructors, they need reliable tools so that they can ask that higher level question of did students learn in the AI era. And while no tool is perfect, we need a collection of tools that can go beyond just a simple detection score or similarity score. Detection is certainly part of the story, but it is an incomplete part of the story if it's leaned on independently. There there needs to be more tools that show how AI was used, where AI was used, at what part of the assignment creation process and the revision process it was used, and it complements the how much question. So, as we'll hear in a moment, you may have certain assignment designs that say, I want you to use AI entirely, or I want you to use AI in the brainstorming process or solely in the revision process. Without tools that really document that, it's very difficult to assess how well students are aligning to those instructions and ideally avoiding the cognitive offloading that we know happens very frequently in the modern era. And with that point, we need tools that showcase process beyond just the output or the product that students are handing in. So tools that show how a end artifact ultimately came to be, tools that highlight where students may have run into difficulty, and if they are using AI, how they used AI to complement the output that they ultimately decided on. Not just I put in a prompt, I got a output, and I inserted that output, but I put in a prompt that was a lot more about inquiry, a lot more about research. I asked a follow-up question. I asked three more follow-up questions. I looked at a source that the chatbot sent to me, and then I incorporated that directly into my paper after reading it and analyzing and understanding how it related to my main points. Just some ways in which tools that really can showcase the full idea of process can be really illuminating to that question of did a student actually learn? Did they actually do what I asked them to do on this particular assignment? And that is is the last point around end to end visibility. As an assignment is a lot more than just typing out the letters on the keyboard, it starts pre actually doing that to how they're researching, to how they're planning out their particular papers if it is a written assignment, what sources they're leaning on, what questions they are asking both their instructor and potentially an AI, how they are reading to inform the points that they're making, and how they are revising collaboratively, with either a peer, an instructor, or increasingly in the modern age in AI. So, we are working on tools that will provide that full visibility. They are not ready today, admittedly, but one of them has been in market for the last two years that at least gets on some of these particular things. And that's what we're going to hear Jenny and Brian talk about in addition to their own approaches to this, evolution from detection to dialogue over the last several years. So, a more detailed reintroduction of our two guests. Jenny, would you like to introduce yourself? Hi, everybody. So my name is doctor Jenny Billings. I'm over the associate of arts degree, the English department, and ACA, which is study skills at Rowan Cabarrus in North Carolina. I have been working with Grammarly now. I think it's almost four years, but I've been in the community college sector for almost sixteen years. So thank you so much for having me today. Absolutely. Yeah. Jenny's been with us, pre Gen AI, pre all of the discussions we're gonna have today back when, the questions were just how do we support students with basic writing fluency. Still an important question, but, one that has been crowded out by a lot of naughtier questions in the years since. So appreciate you sharing your insight with us today. Absolutely. And then, Brian, would you like to introduce yourself? You have to hit the, enter stage button, Brian, as I can see. Yep. Hey. Yep. So hi, everybody. So I'm Brian Harvey. So I am a professor here at the University of Florida in the College of Medicine, genetics and developmental biology. And then I spent a number of years as a associate dean, over a decade in a different college, liberal arts and sciences, and then the last three years as associate provost. But none of that is the reason that I think Cliff invited me here. So I really like to teach. So I teach a very large, course that has used some of the stuff, from Grammarly that I'll discuss in a few minutes, I think. Indeed. I'm actually gonna pass it to you now, Brian. So, I I would love your take on, some of the journey that we had, that I had walked through before. I I think just for context, we have folks joining from, tier one institutions here in The United States, from community colleges, from global institutions. We have some k 12 folks coming from different continents today. So you have a very, diverse constituency here. So it would be helpful just to give, in addition to your intro about your own experience, sort of your own perspectives of, how you viewed the assignment journey, as AI really started to take roots at the University of Florida. Yeah. So, I'll talk it from a kinda teaching standpoint. So I I've always encouraged people to use generative AI technology in particular, but AI just overall. And as I think as Cliff knows that, so my wife and I, we actually have twins that are 19, and and, they have varying uses of generative AI and various feelings of how to use generative AI. So in the context of this course, though, so I I had developed this. Cliff said, you know, we all talk about 2022 and that fall and everything changed. So I actually developed a course that I'm gonna talk about a few months before that. And those of you that have developed courses from scratch realize how much time that takes and how much effort you put into there. And then to realize that your final assignment could be relatively easily answered by ChatGPT at the time, prior to me actually giving the course was kind of an eye awakening moment. So what do you do at that point? And then I know a lot of people listening here are in the same exact boat. It's like, what do you do when when your final assignment in your course can be, answered by a generative AI chatbot? So I'll give you kind of just a little bit really brief background here. So this is a general education course, so there's a little over 600 students, every semester. I teach it every fall and spring, so about 1,200 to 1,500 students a year, academic year, take the course that I teach. And it's on genetic engineering and human cloning. It's actually called, can we design better humans? Should we? So I have, music performance majors in the course, and I have biomedical engineers in the course. So you have to teach the course at a level where everybody is not only engaged, but also can understand the content, which I know a lot of people listening to this have a heck of a lot more experience doing that than I actually do. So the final assignment was, what are your thoughts on what you learned in the course, which was, human cloning, genetic engineering, and that sort of stuff. And since it's not really a hard course, and it shouldn't be, most students do really well. Again, it's it's kind of an intro course, and some students haven't taken a biology course since they were sophomores in high school, and this would probably be the only biology course they'll overtake in college. So I didn't want it to be really difficult. So the chatbot at the time, which was, 3.5, ChatGPT could easily answer the final question. So then you're in the boat where many of us are. It's like, what do you do? You can do detectors, and as Cliff kind of alluded to, and it's certainly my feeling also is detectors are not the best, to actually use to say, you know, don't use generative AI. And if you do, I'm gonna catch you and go down that route. So I chose a different route, which is to encourage students to use generative AI. So, again, this is not a writing class. This is not a course on how to write an essay. I'm not really scoring the students on grammar, so there's a lot of caveats in what I'm about to tell you and the approach I used here. So this is what I did is I gave them a generative AI essay. So I gave them a couple different versions written by different chatbots and said, here's an essay written by generative AI, and you have to then modify it to reflect your views. And then after you do that, they had to write a reflection essay describing what they changed and why they changed it. But the main part of the assignment was to to edit a generative AI text down to what their views on that topic would be. There's a lot of interesting things you can do at this point, and, you know, we'll talk about the Grammarly product that came into play, which is why I'm here. But in general terms, what I said is that if a student wanted to a 100% use the generative AI essay because it perfectly reflected their views, that was perfectly fine. They could just turn it in, and they would get a 100%. They had to write a reflection essay after that, justifying their you know, why they didn't change anything, but that was perfectly fine. And I'll tell you as an aside here, and a lot of people find this really interesting, including myself, is that of the thousands that I have looked at at this point, there is probably less than half a percent of the students that actually turn in their essays without changing a single thing. So you're giving the students the opportunity to turn in something that's a 100% generative AI, and they choose not to. And I think that's a really kind of gives you a little insight into how students are thinking about assignments where you give them a justification of what the assignment is really about, which is their opinion. It's not about writing an essay again. And, again, if you're in a writing class or some other kind of assignment, this may not work. But when you give them an idea of why they're doing the assignment, they actually do not choose to just mail it in, as I've said in various places. They actually choose to modify it and, then reflect their opinion. And some students change it a lot and some don't. And if you're wondering what's the average, it's around 25% is human written. It's about a thousand word essay or about 1,500 word essay. About 25% is human written, and about 75% is generative AI written, of the text that they started out with. So that's a really kind of interesting approach that's worked very well in the course. Where authorship comes in, which is a product, that some of you, I think, listening to this, have played with and I think Cliff can go into detail about. But it allows you to track how much they've changed and how much they haven't changed. And then you can do some kind of cool scientific stuff, which is ask the question, do just for example, do science majors or engineering students in particular change more of their essay than journalism students? Because, again, I have every major in the course. So and I have thousands of students. So I can ask the questions of what majors will go in and change more of the text than other majors. And it turns out that students that and it doesn't actually matter what the major is. It actually turns out how well they're doing in college. So it turns out that students that are doing academically better, and if anybody's interested how I define that, it's not just strictly GPA. Those students actually will go in and change more of the text, which may be surprising to some people, may not be surprising to other people. But academically talented students will actually change more text than students that are struggling a little bit in college. And I'll caveat this many, many different ways by saying in this course, with this assignment, and at the University of Florida, which can be very different than some other location and some other type of course. I have not been able to convince anyone else to do this. So if anyone out there is listening, contact Cliff, because we would love to try this out, say, in a history course or a music theory course or some other sort of topic. So that is very briefly. And, Cliff, I know you have some follow-up questions, but of what I've done. So we're gonna get into the data in a second and, spoiler alert, Brian was able to get, his experience published as a study, and we'll share the link to that in just a moment. Before we do that, I already see some skepticism in the chat about several different pieces. So one is you already called this out. You know, University of Florida, it's a different student composition than a lot of other regional colleges may be. Another is your course was explicitly not a writing course. So do you see this as applicable in other disciplines or in spaces where, potentially the learning objectives are different than it was in your course around human cloning and and yours was very opinion oriented. Yeah. So great questions. So I'm gonna kinda punt on the writing course because I think Jenny's much better positioned to talk about that, than I am. I will tell you about other courses that are not science courses because I've had a lot of discussions around. And, again, I'll caveat this by saying my campus. And if you don't know University of Florida, we have about 40,000 undergraduates physically on campus here in Gainesville. So it's a large place, which generally very large classes, especially at the intro level. My class also I've taught it face to face, but recently, all 600 students actually have 650 or so students this semester is online, which also has some differences. So I've talked to my colleagues. I'll give you an example. So, you know, the history department, totally different world, but they are thinking about, at least, trying out a similar approach. So you can imagine an approach where you have a history essay where then the student has to edit it down to a certain amount of human generated text. And you you can talk about the authorship product, which would do this, and I I will fully admit even though this is a Grammarly venue that there are other tools out there too. I used authorship. It worked great, where you can start off with a 100% generative AI text, and then you can have a student, in the instruction of the course have to only send something in that is, say, 10% AI generated and 90% human generated. And there are tools out there, including authorship by Grammarly, which will allow you to do that. So in that case, it will still work, but there are also lots of caveats. Right? So you that's also not teaching you how to write an essay. That's teaching you how to edit AI generated text to reflect your opinion on the topic that the instructor gave you with guide rails around it. So I think this works in certain situations. I think it actually works really well outside of hardcore science classes, but it's not gonna work everywhere. I I I think it's a, it it's worth noting just a little bit more about authorship. And, you know, this isn't gonna be like a product oriented webinar, but I will just briefly state that because some of the questions in the chat are around this. Authorship is, a product that really focuses on process tracking. And so what that means is, students enable authorship when they have Grammarly installed. They can do it in Google Docs. They can do it in Word. And all they do is they click a button. And when they write, we will be able to attribute, was this typed by a human? Was it modified with Grammarly's traditional grammar and error correction support? Was it AI generated from Gemini, from Chachibuty, from Grammarly itself? Was it revised with AI tools? And we have different designations for each of those categories. And then instructors get a full report of that when a student turns in their assignment. And that's what, Brian reviewed and what we'll go over in a second. But it shows not just a likelihood that AI was used. It actually shows it came from this AI generated source because it was pasted in from that source and Grammarly, by virtue of running across your desktop and your device is able to attribute that accordingly and provides more clear cut insights into how things were used in addition to having a replay that shows certain moves that students made. So in Brian's example, they started out with an AI generated output, and then he can actually see student by student how much they went in and modified. Did they type things in themselves? Did they build off ideas that were in that AI generated output and include their own insight that came from the course or from their own ethical compass? There's a lot of really interesting moves that you can do when you have a tool that goes beyond just an algorithmic probability and actually looks at the process of which, a student used when they created an assignment. And I think that's a good segue to talk about the data that you were able to analyze using authorship, Brian. And again, this is a published study that we'll link to, but would love just, for you to walk us through a little bit more of the, data insights that you're able to glean. Yeah. Sure. So I'll go really quickly here because you guys can all read the study at the end if you want if if you're interested. So this was the first pass. This is only one semester of data, and you'll notice it's not 600. It's three ten. The reason is is because we had to compare what we wanted to compare apples to apples. So exchange exchange students. I run exchange programs. Sorry. That was on the side. So, students that were transfer students are not included on this. I had some dual enrollment high school students. They're not included in this. I had some students that were nontraditional. This is only essays, from freshmen, so first time in college students at the University of Florida. So that's why the number is low. It was also one semester. I'll tell you, after looking at another 1,500 essays, the numbers actually hold. And that's also an interesting thing because you can do this longitudinally. So I don't know what this data is gonna look like if I do exactly the same thing two years from now because those students would have started using this technology when they were in in junior high, and I don't know if that will change things, which is a great question. But as I said, about 75, 76%, of their text was still generative AI. Remember, a 100% at the beginning was, about 25% or so they changed to human text. And then, the top ranked students actually wrote more human text, than the lower ranked students. And actually, this is irrespective of major. So if you're interested, it doesn't matter if you're looking at non science students, if you're looking at business students, journalism students, engineering students. We can break it down by any student population, and it still holds that the more academically, talented students or students doing better at the time they took the course are actually gonna change the text more. I think last last question before we move to a very different experience, is just what was most surprising? You've shared this with me several different times. Like, there's a lot of really interesting cuts that you did, but you shared one of them around, different major breakdown. What what was another really surprising insight that you got from this? So I think another one is, and I I mentioned this a little bit, is that students actually change the text. So, I don't know if that's what you're referring to, Cliff. We've talked about this so many times over the last year or so, year and a half. Students could have just sent everything in without changing a single thing and still would have gotten full credit as long as they justify it in the reflection essay. That's that really shocked me. Maybe I had a slightly lower opinion of some students, I guess, but they didn't. They actually changed the text. I'll also say that another kind of really surprising thing is that the amount of time the students took to do the assignment. So it's not shown here. You guys can look in the paper if you're interested, but the authorship program will track the amount of time a student takes on the assignment. And it actually, maybe not that surprising, tracks really well with the amount of text that they're changing, which is a nice correlation. It's kinda if you're going into the study at the beginning, you might have thought that, but it actually is pretty linear, which is also nice to see when you're doing this sort of study. Yeah. And and last question just because it's it's an interesting one that people are posing is, you know, I I mentioned this at the beginning. The the key question to ask is are students learning? And that is not a simple question to answer. It's not, a uniform answer, discipline by discipline, course by course. You already highlighted, Brian, that your learning objectives for this particular class were meaningfully different than learning objectives that you might have in a first year writing course, in a history class, in a science class. But how did you define success in this assessment going in? Because I think some people are skeptical. They're like, it's still 75% AI generated. Why is that successful? Yeah. Yeah. So, Cliff's kinda doing a leading question. So the next paper that we've written and unfortunately got rejected and has to rewrite it. But you can actually measure learning in different ways. So Cliff's exactly right, and I'm sorry. I'm not following the comments. They're going so fast on my screen here. But that's a great whoever's asking those questions is absolutely right. You gotta measure learning. Most of the students do really well in the course. Right? So you can't really correlate the student that writes 2% AI or 2% human, how well they do in the course versus student that writes 25% human. Or there are students in here that write 90% of the essay, they actually change. But there are other ways to do that. So one of the things we've actually done is we do a pre and post course survey, which has been really kind of a interesting eye opening experience. If those of you have, experience writing survey questions and then, the kind of the pedagogy of trying to prove that those survey questions are actually addressing what you want, you'll understand how you can't just write a question and then ask it and assume that it's actually answering the question that you want. So we do have pre and post course validated statistically survey questions addressing the student learning outcomes in the course now, And they will change, and you can correlate that by how much the students change, their essays. So a little bit preliminary data here, but you can actually look at the students that really don't know much about the topic before they take the course. Then they change their essay, and then you can correlate that to how they answer the questions at the end of the course. And the questions are actually the same at the beginning and at the end of the course. So it's a way to measure learning that's independent of grades and also independent of this final assignment. And that's kind of a little bit of a little bit of hand waving because I'm not showing you any of the data, but hopefully that will be coming out relatively soon in another, publication. Yeah. I I mean, it's it's a fascinating use case just around a course that is a general education course and is on human cloning of all things. Like, I would imagine very few students do come in with a lot of preconceived knowledge and, perspectives on the topic. So, I think how you employed our tool, in a very innovative way is, you know, really interesting to take take a look at. As I mentioned before, we will be sharing the link to the study, in the q and a at the very end, so stay tuned for that. But I wanna bring in Jenny to talk about a very different use case and a very different context, particularly when it comes to how, she and her team employed Grammarly authorship. So Jenny, you come with a very different context than, I do. Ryan does. Can you just talk a little bit about, your journey on, using authorship, particularly the problems you were trying to address, and why you were interested in this to begin with. Sure. First of all, can you hear me okay? I saw some folks. we're having. Okay. Awesome. So hi, everybody. So as I said earlier, I've been with Grammarly for four years now. The state of North Carolina redesigns developmental education quite often, And so we wanted to make sure that we had a new tool in order to help our students with revision and in order for them to really keep track of different drafts, where they would know where all of their documents were so they wouldn't be searching on their computers for those, and to assist them with formatting, downloading it into Word, you know, item submission, things like that. So that was the first reason that I went to Grammarly, sent a message, you know, didn't know if I was gonna get to partner with them, and I did, and it's been a beautiful thing ever since. Well, then, you know, months into my partnership, AI came into the conversation. And, obviously, the first thing that, you know, my faculty did was freak out. We had a lot of anxiety. The focus was absolutely on the enforcement and policing of AI was absolutely the original conversation. How are we gonna catch them? How are we gonna prevent the use of AI? And so we really had to come together and figure out what was gonna be in the best interest of not only my division, my faculty, but of my students where the world was headed with AI and were these going to be skills, durable skills, not just soft skills that they were gonna need in the future. So that kind of helped us reframe that conversation early on when we formed the first committee, from the English department that then led to conversations across the college. We then started putting in assignments into our English courses, into the writing courses in order to show students both the benefits of using, Grammarly at that time, but also some of the, you know, some of the things that were less beneficial if they use something outside of that that was generative. And then, of course, authorship came two years into our partnership, and that was when we adopted that. But it was really mostly about first refocusing faculty from a policing standpoint to not necessarily embracing, but we had to figure out a way that we were going to, you know, teach students how to use it in an ethical way, in a way that would benefit them as students, as learners, and as writers, but that was also not gonna completely remove them and their voice. So that's something that we've been actively, working on for years. And so instead of policing final products and things like that, faculty can now see students struggle. So this has really reframed the conversations that we've had with our students. Why are you defaulting to AI in this situation? And in many cases, it has been a confidence thing. So a lot of our students are not confident not using the tool, and so we have had to reframe lesson plans, grammar lessons, things that we were not necessarily doing day to day in order to increase that confidence so they don't feel so reliant upon it, but that they also know where they can and cannot use it. But also making sure that we understand the struggle that we can intervene at that point, whether that be with or without Grammarly and just making sure that we're having more honest conversations with our students so that we can, you know, make sure that we are intervening in the right time and therefore creating more learning opportunities. So I I wanna just step back and and get a little bit more in the weeds of who your students are at Rowan Cabarrus. and, Yeah. what they tend to struggle with, Sure. and particularly, how they and you at the institution approach writing in 2026. Like, how foundational. is it to their success, and what outcomes do you typically want to see in writing intensive coursework? Sure. So North Carolina has 58 community colleges. We are we kind of falter between top six top eight depending on the year, but we have been growing. So we are a fairly large community college. We serve a lot of high school students, so our CCP, or Career and College Promise program, population is very large. We have three early colleges that are on-site, and then, of course, we have traditional and nontraditional students, that are either coming in for an actual degree, just a credential, couple classes, and reverse transferring. So our population, you know, is very, very diverse. Most of our students, are not taking math and English as soon as we need them to, so that's a battle that we are constantly fighting, trying to get them into the courses as soon as possible. My area, the one that I'm in charge of, is study skills that will teach them everything they need to know about being a student. Grammarly is involved there too. In our English courses, both developmental and curriculum, where we're teaching them, about the institutional access that they now have to Grammarly, about the overall score reports that we use to monitor how much our students are revising and their drafting processes and their writing process in general and then of course authorship as well. So, we're really trying to get them into these courses that are serving truly as gateways not only to the institution but to the technology that will make them successful at the college, so that's why we embedded them throughout those courses. As for what writing looks like now, we're actually about to do a revamp. So when Grammarly, joined my family a couple years ago, we were teaching a different developmental course than we are now. So a lot has shifted over these four years. So the developmental course changed from English two to English 25. Rowan Cabarrus removed faculty withdrawal, so we no longer withdraw students. Students have to withdraw themselves. So as you can imagine, that's impacted data because we have less withdrawals, but we thought we would have more f's because of that, but that has not been the case either. So we've seen cool gains with that. And then, of course, the introduction of authorship and institutional license for Grammarly where our students have it across all of our courses. And, North Carolina is moving LMSs, so we are going to a streamline LMS to Canvas in the fall. Rowan Cabarrus is going first. You know, all of this has led to a lot of revamping for what writing looks like now and what it will look like in fall of twenty twenty six. I was looking in the questions, and somebody said, you know, that it's all faced to think that we only teach grammar and writing courses. Absolutely true. We do not. We we conquer a lot in English courses because I'm preparing them for the technology that they're about to use. We're teaching them how to use word, MLA format, APA format. We're teaching them how to submit things, you know, and, unfortunately, grammar has fallen kind of to the backstage, in a lot of cases in a lot of my courses, but we are very excited to bring that back. So in order to complement Grammarly and what our students are already doing with their revision process and teaching them how to use AI to plan and draft and things like that, we're gonna be taking more of the grammar lessons back into the classroom, so that's what fall twenty twenty six will look like. I think we're even gonna bring a novel back. So we are going back to the basics in our instruction and in our classroom because our technology and our assignments are so much in the future, and we're really looking forward to that. We think that coupling is gonna be beautiful. Something we've seen over the past couple of years is that our students are really, really, really tired almost of some of the things that they've seen with AI. Mhmm. We thought our students would stay online, you know, and they wouldn't come back to the classroom. That has not been the case. They're coming back to the classroom more than ever. And so we want to make sure they know the benefit of things like authorship and Grammarly and and AI and how it can be used if they're allowed to use it. But we also want them to know, you know, why Grammarly is making that suggestion for a comma there and what a semicolon, you know, replaces and things like that. So we're looking forward to that partnership and, really bridging that gap, from an old school perspective in a very, very new school technology world. Yeah. What what I think is interesting about your case is you you had really, a minimum of two arguably more problems to solve. You had. the fact the anxiety and with that came. academic integrity violations that were very real with AI, right. That's right. you had the developmental angle, like the idea of both, That's right. right and writing to learn. That's think. that are being a very enduring skill that will, I'd hopefully persist through this AI era. And authorship was able to help you with both Right. as a. tool, not as a perfect. silver bullet solution. But can you talk. a little bit about the initial results and then how you've implemented it to go beyond just the academic integrity challenge, Yeah. I will. your current implementation? Absolutely. So when we first, agreed to be one of the beta pilots for author ship, that simply mean that we put it in certain classes that were not going to, have technology issues. So what I meant by that is when it first came out, it had to be downloaded. It was only, you know, kind of hovering over Google Docs. We had to make sure the students had access to download or enable this feature. You know? So we wanted to make sure that we really had the right audience to pilot with. And so what happened was we had authorship in some classes, and then we just had the traditional usage of Grammarly and the overall score reports in the others. We actually require that in all of my developmental courses and my gateway English courses. They have to run reports to show us how much they're interacting with their text, what they're revising, what they're editing, what they learn, what their score was, that kind of thing. So that was a way that we kind of combated our students not wanting to draft or revise and just submitting a rough draft as a final draft. So thank you Grammarly for that. So when we put authorship in those classes, the, we had an average academic integrity violation score of 27, 27 violations per semester. That semester that we institute or there that we put in, authorship, it went from 27 to one. So that's why you see the 96% drop, so we had just one. And the reason for that, I think, is because these reports, authorship is student facing, So they are able to watch their video. They're able to see what you're about to see. They can see, you know, where it's showing that they wrote, where they lifted it from, what they, you know, how long they were actually, you know, interacting with the text. And so so many of our students would tell on themselves, and they would write us and say, you know, oh, doctor Billings, you know, you are probably not gonna love this. So, you know, could I have an extra twenty four hours to revise this or, you know, things like that, which created a lot of really cool conversations with our students. So they were seeing what you were about to see first, and that led to even more revision before they submitted their true final draft. And then we saw an increase, so when we were comparing data from fall to fall, so we were trying to get as close, you know, to the same population as possible, we saw an increase of eight point five three percent in success rates, in the same developmental course. Now I did mention that we're in a new developmental course now, so I did look at our our rates over the past four years. So since bringing in Grammarly, we have increased, fourteen point four two percent. But again, I would like to mention that it is a different developmental course, similar in nature, but it went from six contact hours to three contact hours. And then it also we removed the faculty withdrawal. So we thought, you know, the grades of f would go up because we were no longer withdrawing students to save them from a failing grade, but that has not been the case either. So in our mind, you know, it's a success all around because the success rates have gone up. The numbers of d's and, excuse me, f's and w's have decreased, and then we don't even look at these, because we don't count those. So see or better is truly what we're looking for. So we've had a lot of gains, in a lot of different ways, and we think that Grammarly has made them, you know, bringing in authorship as part of the conversation. Grammarly in general and just the partnership in general, I think has made my students more confident in the tools that they're gonna need to know in the future. Yeah. I mean, that's that's we obviously love hearing that, but it's also, a a great case study of how you know, it it's not about a single tool. It's about a lot of these things coming together and, policy and, pedagogy aligning with the tools that you have at your disposal to really get at the crux of, what your institution, what your individual instructor, and what individual students are struggling with. Mhmm. As we bring as we head to Q and A and we bring Brian back into the fold here, I wanna tackle one question that came up in the chat that I think is relevant to both of you because, in using authorship, part of, authorship is process tracking. And there are instructors that I have talked to that we've talked to as a team over the years that are inherently uncomfortable with that idea. And so Mike asks, you know, how do you toe a line between surveillance and trust when, Yeah. you're employing something like authorship? And Brian, in your case, when you're explicitly stating that you're gonna use this for research purposes, like, how how what is your advice to people who might might wanna consider a tool like authorship to, improve the transparency of what students are using in their assignments while ensuring that it doesn't become, like, big brother surveillance. So, Jenny, you can you can go first. I'm gonna punt to you. Okay. That's fine. So something that we have talked a lot about and looked a lot about, you know, I I work with multiple institutions. So one, for example, uses copy leaks that has an AI detection. It usually always says that students are using AI a 100%, and my students will panic. They'll email me. I swear to you, I, you know, I wrote this. I don't know why the score is this way. You know? And then, you know, SafeAssign is looking for plagiarism, but not AI detection. Then you have Turnitin that had you know, there's all of these different tools that are hunting for AI. Right? And so I think no matter what, even if authorship wasn't part of the conversation, there's still a lack of trust with all of these reporting features that we're attaching to assignments that are built into these, you know, LMSs and things like that. So I think our students are already used to having to scan for plagiarism and things like that. But, you know, what has been cool about shifting this, conversation to authorship and grammarly is that now the student doesn't have to wait for the report and panic and email me, and are you gonna grade this and the score is showing this, and I don't know why the score shows this. You know, now it's part it's forward thinking. It's it's preactive. It's part of the conversation before they submit their work. And quite frankly, I'd I'd rather have the honest conversation before I'm ready to grade, you know, and understand, you know, why the report is showing that, why they're fearful of submitting their final draft, what they need at this point, why did you resort to use it? And so I've been really thankful for the shift of when the conversation is happening. So the conversation is happening before midnight rather than after. And I am seeing, I'm definitely seeing more, thoughtfulness in the revision process, whether it be to get themselves out of a, you know, out of a hole or, you know, just to make sure that their score is better or that what I'm about to see in the authorship video or report is better than what it was. So, you know, it has solved the need for my students having opportunity to revise. Even if they weren't revising on their own, they're revising after they see their report. Yeah. So, I mean, very similar experience. So I I'm very upfront with the students and tell them exactly why they're using it and how to use it. And I will say I do have a backup assignment. So if students do say I do not wanna use the authorship product, which is, quote, required, there is a backup, which is the color code the text they change. So, of all, I've had, I think, two students that have requested that. And I should also mention that that I do have some students that, and this is not you know, this is a Grammarly thing, but it's gonna sound bad. So they can't actually install the software in their program because their computer is owned by x entity, and they don't, have access to install new programs. So in those cases, those students have to do a backup, to the assignment. So what I'm basically saying, it's always good to have a backup, to the technology for students that are uncomfortable with it, but that's been very few, in my experience. I think another question that came up that I've heard often and is worth addressing is the, friction that tools like authorship, but it doesn't have to be exclusive to authorship, adds to an instructor's plate. And, Jenny, you you did mention, that North Carolina shifting to Canvas. One of the ways. that we are trying to make that easier for instructors is we have a, LTI connection with Canvas, where it's easier to, ensure that students, when they submit, the report automatically flows through the Canvas assignment process. You see the full. authorship data there, but it is still admittedly additional work that an instructor has to do. So, Mhmm. Brian, you labored over this data for reasons that we do not expect any of the attendees really to mirror, but, I am curious both of your perspectives on, what is the right amount of instructional cognitive load that is on the plate of instructors in the era we're living in and, like, what is the right level of responsibility balanced with, unreasonable time expectations? Yeah. I'll tackle that. I mean, I so I I love love teaching. I love developing classes. I love developing assignments, and I cannot stand grading. It's like grading and, looking over the assignments. For some reason, it's just like that's something I just don't like doing. So anything that can decrease the amount of not effort because it's always gonna take the same amount of effort, but it is gonna decrease the time of the instructor to look over the assignments is great, especially if and I know a lot of people listen to this, like me, are doing really large courses. You just can't spend five minutes on an assignment times 600. It just it just doesn't work in in when you have to grade, like, the final exam within three days because they need the grades so they can graduate, from the college or university. So in my experience, and I'm not talking about the data analysis, which is a whole different aspect, which you do not have to do to assign the grades. But just having the numbers there, which is human generated, AI generated into some of the subcategories, which Cliff can tell you about, and also being able to visually see very quickly by just scanning the assignment is incredibly useful for an instructor, at least in how I design the assignment. Jenny, good thoughts. So my faculty take a lot of time to grade already. You know, these are between two and fifteen page papers. We have four essays that are in curriculum English, final drafts. We have six, that are in my developmental English. So, you know, we are, you know, not strangers to spending a lot of time on feedback to our babies by any means. But what I quickly realized probably in '23, yeah, so probably 2023, that my faculty were spending way more time being investigators than they were educators. And that killed me because they were going through, they were finding the, you know, the links. They were going on the website. Nope. That's not legit. You know? AI made up that. You know? They made up that source. You know? And then they were having a document. Well, this in text citation is not going with this full reference. That one must have been made up. You know? And they were just exhausting me by how exhausted they were, and it killed me. It really hurt my feelings that that's what it had turned into, you know, that they couldn't just grade what they wanted to. You know? They felt like they had to go line by line. And so our partnership, bringing authorship in, you know, has put more of the, load on the student upfront. You know, if if you are going to use these tools and your instructor allows you to do so, show us how you used it, show us why it was helpful where you, you know, where you used it, and ultimately how it helped you. But if it didn't help you, why is that? You know, did you not prompt it correctly? Did you use it in a place that you weren't supposed to? Did you not learn from this? You know? And so that has put more load on the student, less load on educators being investigators, which I'm very thankful for, and has opened up much more meaningful up front conversations than the panic at the end that would often lead to either academic integrity violations or grade appeals. I I really appreciate both of those answers. I I wanted to have more time for questions, but we are nearing the top of the hour. So I am gonna close this out. Thank you everybody for engaging in the chat, asking some difficult questions, but very relevant questions. Absolutely. And as we think about just how to summarize on this, like, and you're in a situation where you're evaluating, you know, how much do we wanna lean into these tools? How much do we want to look for alternatives to just purely AI detection? Some advice is just start small. Like, you heard Brian tackled his own locus of control. It was his course. He knew that he needed to shift some certain things, and he did so accordingly. He has been fighting the good fight, in our opinion, at the University of Florida since trying to evangelize this to other departments, but that has not precluded him from making the gains that he wants to at his institution. I think the same is true for Jenny and across the entire state of North Carolina using Roe and Cabarrus. as a, really launch point for all of these things. You don't have to do this alone. Like, you heard Jenny talk about addressing the primary problems that she was hearing from her instructors. You know, there there are plenty of people who are seeking alternative solutions, and we just hope to be one of those solutions. But, there's several out there, and it is not a a silver bullet solution in a vacuum if you don't, have your faculty on board and you don't have students in the loop. And one of the questions that did come up was, is authorship free? It is free to students. There are components that are, up for the paywall, but anybody can enable authorship tracking, can share a report. And if you are a Grammarly for Education customer, you get all of that included in addition to the Canvas LTI integration that Jenny mentioned. We are about to launch one for Blackboard as well. So if you're a Blackboard user, if you're a Canvas user, you'll have tools that make that automation a little bit easier. So feel free to fill out the poll if you wanna learn more about the institutional solutions that we offer, including authorship. But let me just say a huge thank you to both Jenny and Brian for sharing their insights. Read Brian's report if you're interested. That will be included as part of the follow-up email as well. And we wish everybody the best of luck as you enter fully into the summer season. Thanks, everybody.