Coraggio Consulting
Loudoun County Public Schools, Performance Monitoring Committee Meeting, Tuesday, August 19, 2025

Date: Tuesday, August 19, 2025
Place: LCPS Administration Building / School Board Room, Ashburn, Virginia
Length: 1h 13m (video)
Official meeting video (Vimeo / LCPS-TV): https://player.vimeo.com/video/1111447257?rel=0

Machine transcript (speech-to-text) produced by Coraggio Consulting. Not an official county record. Names and numbers may be misspelled; check the video at the timestamp before quoting.
Speech-to-text: whisper.cpp, model large-v3-turbo. Produced 2026-10-13. Timestamps are [HH:MM:SS] into the meeting video.

----------------------------------------------------------------------

[00:00:58] - Good evening, everyone. I'd like to call to order this meeting of the Performance Monitoring Committee, this August 19th meeting of the Performance Monitoring Committee. My name is Arbin Estreffi, I'm the chair of this committee. I'm joined by Chair Mansfield as well as Ms. Chernoff. We'll start off with public comment.

[00:01:16] Is there any public comment for today? - There is not. - Okay, so seeing no public comment, we'll move on to the approval of the meeting minutes. Do I have a motion to approve the draft meeting minutes from the May 29th meeting? - Yes, Chair Mansfield.

[00:01:33] - I make a motion to approve the minutes from the May 29th, 2025 meeting. - I have a motion, do I have a second? - Second. - So motion by Chair Mansfield to approve the meeting minutes seconded by Ms. Chernoff. Is there any discussion?

[00:01:49] Seeing none, we'll move on to a vote. All those in favor, please say aye. - Aye. - That motion will pass unanimously. The meeting minutes have been approved and we'll move on to our information items this evening, starting with 4.01 Department of Academics, Performance Monitoring Committee Reflections and Goal Setting.

[00:02:09] Thank you, Mr. Estreffi. It's good to be back with you all this afternoon. Hope you all had a great summer. So for our first information item, it's really more of a discussion. I'm going to turn it over to Dr. Single in just a moment to lead you through a discussion on our year one reflections, what worked well, what maybe could we improve upon, and then And as we look forward to year two, what will success look like for you all for this committee?

[00:02:40] What topics should we consider? So really we just want your input and then we will take all that information back and bring a plan forward similar to how we did in last year. So with that, I'll turn it over to Dr. Singel.

[00:02:55] Thank you, Dr. Ellis. Good evening or good afternoon, board members and staff. We intentionally wanted to make sure that we reflected and set some goals at the beginning of this meeting because our June meeting got canceled, and so we had planned to have a discussion with you at the June PMC meeting to talk about the first year of the PMC meeting, to talk about successes, some room for growth, and then to set goals for this current school year, but because the meeting was canceled, we are hoping to engage in dialogue this evening.

[00:03:35] So if we go to the next slide, because it has been a couple of months, if you will, it's important to just remind ourselves of the meeting structure that we had decided on, as well as the roadmap. So at the first meeting during the previous school year, we had you engage in dialogue with us around some goals, what presentations you wanted to see, what data we should bring forward. Each meeting, we talked about goal one data, which is student centered. And then we had a school-based presentation following that goal discussion,

[00:04:09] goal one discussion. We rotated between goals two, three, and four throughout the year. And then if there was any special topics that the board that you wanted us to discuss, we brought that forward as well. So in thinking about, um, reflecting on year one, what, or are there any structural or procedural changes that could improve effectiveness? Um, what worked well? And then what might be some areas for improvement? Um, those are some wonderings that we have, um, just so that we can make sure we take that into account this year as we, um, put

[00:04:44] together a plan for our work together. Do we continue and come back to this or do we stop. I think we can stop. Thank you very much. Board members, any discussion? Well, I do say that I did like to have the principals and schools here to talk about it, what they're doing and what is making a difference in their schools. The one thing that is hard for me is that we're doing

[00:05:20] it in all these different little pockets and where can we find the consistency um so that's something that i would like to look at how we can make things more consistent from school to school um and i have some more thoughts but i'll let other people speak real quick thank you um i appreciate this conversation i've really enjoyed this committee i don't think that that's a secret to anyone. I enjoy our conversations. I think this is the bread and butter of what we do

[00:06:00] in terms of the data. We've had a lot of exciting developments in terms of the platform, and you know, I've certainly learned a lot. One of my biggest takeaways was learning about the program review process, just what programs we do review and how often. I mean, that was really eye-opening to me that we have that rolling. I echo what Chair Mansfield said. I love hearing from our schools especially having the students here that one time is always just brings the work to life I would like to this isn't really an improvement because we've been we're so young I feel like still but something I'd like to see us get in a rhythm of is revisiting some of the topics and

[00:06:39] a lot of the times we've said like oh well the schools are trying this and then revisiting back and seeing were there good outcomes there and are those things that we want to continue to support I'm thinking specifically, and not to call out Mr. Schaefer, but I'm thinking specifically about Riverbend and how they use flexible staffing and how that was supporting their data, and they'd seen such significant shifts.

[00:07:00] So this year, if there's been any shifts in their staffing, for better or for worse, let's look at those numbers and see how did that impact. And then that can help drive the conversation of where we need to put our resources. And so that's something I think we're still going to have to kind of hammer out because I think we are dabbling in a lot of different topics, which has been a necessity of getting started.

[00:07:20] But I would really like to see a cycle of sort of review in certain areas. And I'm also looking forward to us sort of revamping our strategic plan. I believe we will have a much more streamlined strategic plan.

[00:07:36] I think that that seems to be the goal from this school board and from staff and the superintendent. So maybe streamlining that will help us streamline our work here, because we seem to be aligned with that work. So thank you. I love that point, Ms. Chernoff, and I agree with all of the comments.

[00:07:57] I think hearing the stories directly from the schools has been super helpful. But I think that the combination of the data or the quantitative side of the presentations with the examples from the schools, that combination has been really powerful.

[00:08:15] And the original intent of this committee was to be able to focus and hone in on student achievement and all of the things that impact student achievement that we don't necessarily cover as directly in some of the other committees. And I think where it's been most impactful for me is that I think so much of what we've discussed in this committee so far has, one, touched on our individual and collective priorities as a school board in a more focused way than other committees get to do.

[00:08:50] And in doing that, I think we've been able to inform the entire board as a collective in our school board meetings as we discuss those priorities and be able to inform some of those decisions in a way that's been more focused.

[00:09:07] So that's my impression. I think that has been one of the successes. I'm curious from the team and from staff if you all see it as a benefit as well from that perspective that we get to dig into some of these topics and how this committee and the topics we've touched on may have impacted our approach operationally outside of just school board decisions, if I may.

[00:09:38] Sure, I appreciate that question. And I will start and ask if any of the team has anything to add. I agree with what all of you have said. I do think that for a baby committee, we've done a great job getting it off the ground.

[00:09:54] I do think that it would make sense to, I hear Chair Mansfield's point about it's great to hear from one school having success and what are we doing to support the schools that maybe aren't having a success in that area.

[00:10:11] So looking at it more broadly, I think that that was a really good place for us to start. I think it's important to have that principal voice, to celebrate the successes, and then also dig into where there are areas of improvement. I think that we can work together as a staff and committee team to tighten up the topics.

[00:10:32] And I think that will also happen naturally as we tighten up in our new strategic plan. And I think when we do tighten up, we will then be able to really hit on what Ms. Chernoff was saying in terms of cycles of review, right? So are there things, now that we're going into the second year, and our first full year, really, are there things that we should be reviewing on a regular cadence?

[00:10:54] Are there things that are important enough to you to look at quarterly, for example, whereas there's certainly not time in the full board meetings to look at, for example, student achievement data. Every single meeting, is there a more frequent cadence that you all want to look at certain things?

[00:11:12] And I think I touched on all of that, but all that to say, I think we've gotten a great start and we can certainly tighten up some things. And if I could just add to what Dr. Ellis shared, what I really appreciated was the strategic plan coming to life through the storytelling.

[00:11:32] I think that was very powerful. And the way that we engaged in talking about the data, focusing always on goal one, students, And then looking at different topics in Goals 2, 3, and 4 that supported students, didn't necessarily have a direct correlation, but not necessarily causation, to looking at, for example, the staffing, as well as student performance data.

[00:12:00] I really think that was powerful. Something that was really, something to also dig down is how to present the data. And that's something that you guys, like, nailed on for the MTSS, right?

[00:12:19] I mean, I think that was really powerful. And to do that kind of work here in this particular, you know, something that makes sense in this committee. The other thing that I was, so we're really focusing on belonging, right?

[00:12:36] And I know we're coming up with some, probably a multidimensional kind of look at it and how that affects achievement, how that affects relationships, you know, those kinds of things. I think that would be an interesting thing to look at more holistically.

[00:12:58] And absenteeism, I think that you could look at, you know, how does belonging look at, affect absenteeism? How does belonging look into discipline or hate speech? Or how does belonging affect our achievement?

[00:13:15] And I think we're going to see a lot, Because everything that I heard here was all about relationships. It was such a big part of how kids felt in their schools and how it made them be successful and have gains, that relationship.

[00:13:35] I think the other thing that I was, absenteeism. You know, part of me wonders how bad of a problem we really have with absenteeism. Is it really making a difference?

[00:13:50] And is it really a population of kids that's making a difference? So I guess that I'm going to quit babbling. I'm not very good at 6 o'clock in the afternoon at night, but I do enjoy this.

[00:14:08] And I think the presentations have been great. And the way to present the data more efficiently has been really helpful for me. And I think we're going to have to continue with VALS. And I think we're going to have to continue with the MAP data. And we're going to have to be able to show why the SOLs might not be showing, or the accountability measures might not be showing what we know is happening through a more normed measure and something that's been more consistent, and especially for our reporting groups.

[00:14:44] So, that's me. Are we moving on to the next set of questions? Okay. Yes, please. I'll let you. I think we've naturally done that. Melinda kind of went there. We have talked about some of these here.

[00:15:04] For example, I would love to hear more about some topics that we should prioritize, and then maybe we can come back to some of these questions just because we started going down that route. All right. With that, I'm sort of, after having this time under our belt, I sort of see things in buckets.

[00:15:24] I think that the achievement data is always going to be a bucket for us. I think there's a lot of overlapping factors. I think about the presentation with the Lego pieces, and you're telling the story with the data, and I think this division as a whole is headed in that direction, and I'm thrilled about it. But achievement for me just is always, I think, you know, especially at those three times a year, we're going to do that deep dive.

[00:15:47] I think MTSS really tells the whole story, and we'll see where SOLs take us. But another bucket I'm wondering about because of some of the stuff going on with collective bargaining and with HRTD and policy is if we should be absorbing some of that data about staffing and retention and having those conversations here i'm not sure if that makes total sense but when i think about buckets of achievement i also think about our staff and i think that that's something that we have really focused on sort of inadvertently we haven't named it as that but i think a lot of the things that we heard from the schools and staff are about why they love what they do and why they

[00:16:23] stay teaching and why they stay at the schools that they're at and then another bucket that i don't think we've really done a deep dive on but i think really speaks to what miss mansfield just said about belonging, which is a focus and is an umbrella that covers so much, and I agree. I think we have to look at all of that multifaceted data. Just thinking about our recent school board meeting, we had a very high-level discipline data overview, but there were lots of questions that were driven out of that presentation. I think a lot of us want more specifics. We want to do a

[00:16:56] deeper dive, and then we also want to look at the flip side of when there's referrals, when there's hate speech when there's anti-Semitic comments made or images that are drawn on desks or in bathrooms or things like that what is the follow-up what are the processes that we follow up with and then what is the data result from that as well I would really like to to have some better understanding of how we respond and is that response effective especially I know we're putting in some equity walks at schools and we're doing some new work around SEL and guidance

[00:17:33] and some of the work that we're doing with counselors and with, you know, teachers. And so I would love to see all of that be another bucket that really is the belonging bucket but kind of unpacks all of those different things. So those are some things I'm thinking about in terms of topics.

[00:17:49] You know, what's really hard about that is, like, some of it can be informal in the classroom. Some of it can be, like, reporting, formal. And then people conflate that with the emails that are sent from the principals.

[00:18:06] But so I don't know how to. Yeah, but I think having some concrete numbers on numbers of referrals and what type and where they're occurring could help us kind of get away from those generalizations maybe. And help us to assess our practices in terms of what we're doing to address, which I think goes back to your point about relationships.

[00:18:26] I think a lot of it is going to lead to intangible things, unfortunately, but it is really a people issue, right? But it's a school culture issue. I think it's layered. And so I would like those things to be cleared up by having actual numbers instead of saying, like, we're not going to report that, but you do get these e-mails every day that say we have all these things.

[00:18:46] The only time the e-mails come out is if other kids have heard it. Right. So that's the difference there. I think this bucket around school climate in general is an important one for me as well, so I appreciate the point. So school climate, belonging, and then layering in all of the other factors, such as discipline, I think are going to be a really good topic area for this committee.

[00:19:15] I agree. All right. Thank you for that. So that gives us a lot of really good information, and it does.

[00:19:31] And we will work with Mr. Streffi and put together a draft to address all of your suggestions. So thank you for that feedback. And then I'm going to turn it over to Dr. Tyler very briefly to talk about program evaluations from this past year and give you all an opportunity to discuss that.

[00:19:52] So the famous Dr. Blakelock is here from the research office. So these are the – no, it's good. I just want to make sure I introduce them as well. Thank you, Dr. Ellis. So these are the programs that are on the schedule for program evaluation this year. And I think it might have been mentioned by Dr. Ellis, Dr. Singel.

[00:20:10] We will be having a follow-up conversation about the programs on the schedule for this year, 25-26. So the previous year is wrapping up, and Dr. Blakelock is here. If you have any questions about these items that were on the schedule for last year, reports are being developed.

[00:20:26] They will be published on the LCPS research website. But the idea here is do you have any questions about these programs or follow up? Are there any particular subjects you might be interested in digging a little deeper once the reports are available and Dr. Blakelock is here, if you have any clarifying questions at the moment.

[00:20:48] - So did you make any connections between something like BEAM and EDGE and Achievement later, any correlations of any sort? - So that's a great question, thank you for that question. With both BEAM and EDGE and GIFTED, One of the things we look at is enrollment in advanced course pathways later on in the educational career.

[00:21:09] So as you're working with students to prep them to help them advance their pathways, we are actually looking at, you know, are there BEAM students enrolling in pre-algebra in the seventh grade or where they would go into algebra and geometry? Same thing with EDGE.

[00:21:24] You know, what kind of middle school experience are they looking for? Especially since EDGE is, you know, it's a program for younger students. Are they moving into gifted? So we are looking at some of the progression of what students are doing later on in their educational career after they've participated in that program.

[00:21:40] And then what about the gifted education, the, what is the program that's for, like, talent development, gifted education? No, not search.

[00:21:59] No, it's spectrum, because that's gifted. It's the one for, it's typically for a lot of kids who have disability, who are underrepresented, yes. i can't remember is that edge maybe it is maybe it is edge anyway are we seeing um them being able

[00:22:30] to be filtered in later um or just stagnant in that one uh program that talent development are we seeing later maybe it is edge it's edge so okay then never mind you've already answered that question the other thing is gifted education as far as um ao um academies of loudon and what's the difference between Academies of Loudoun and TJ?

[00:23:02] And are we seeing an increase in TJ or a decrease in TJ, increase, I mean, I guess my point is, would we have enough students if there was funding somewhere down the road in five or 10 years to actually have our own TJ?

[00:23:20] - That makes sense. TJ is not something we've looked at specifically with the gifted program. I would have to look at how we could get our hand, like access to those data and kind of walk back the gifted program a couple years to do that. Looking at both application two and enrollment in the academies is on the evaluation design though as well. Can you share a little bit about how we're evaluating the Welcome Center? Certainly. We're looking at a variety of different factors with that. From a process perspective, we're looking at the amount of training that's occurring with school-based staff as well as Welcome Center staff. We're looking at the number of people,

[00:24:00] of a number of students who have been served by the Welcome Center. We are looking at some of the supports that are available inside the Welcome Center itself. From an outcomes perspective, we are looking at parental experiences as far as the ease of enrollment in the regular school. We're looking at how well the recommendations from the Welcome Center were followed in terms of enrolling in grades in elementary and middle school. Those are sort of the big ones that we're looking at inside of that. There's a few other pieces in that

[00:24:30] evaluation design as well all right so we will let you know when these reports are available on the website and of course if you want to do a deeper dive we are more than happy to add one of them to the agenda should we move on to

[00:25:07] 4.02 Department of Academics goal 4.1 student voice we are very delighted to have miss Lynn Piscus from the student activities and athletics office joining this evening to talk about student voice her office does a lot of engagement with students and as part of our metrics one of the things that we are tracking is student voice making sure that we're amplifying it and providing for opportunities for students to share their concerns to have a voice in the

[00:25:43] functioning of the school division and with that I'm going to turn it over to - Good evening, thanks for having me here today. We've highlighted the part of the strategic plan that this student voice piece goes with, and I thought I'd start off with a definition just so we're all clear what student voice is, basically.

[00:26:09] Empowering students to have a say in their learning and be heard by educators and administrators and be engaged in creating their schools. We have lots of different ways that students have opportunities to express their voice in our schools, and I've highlighted a few of them tonight, and I'm gonna start with some numbers, going along with the data theme of the evening.

[00:26:39] This shows the overall participation in student activities in our school district at the high school level. VHSL athletics, we have 52% students VHSL activities, So that would be debate and drama and things like that.

[00:26:59] 6.7% and then extracurricular, we have almost 60% of students participating. Go ahead. And then this slide has a breakdown by schools divided by sports and by club participants.

[00:27:16] And I will tell you that we had a meeting last week with our student activities and engagement coordinators, which is who I work with primarily, and I stress to them that the data we have is only as good as the data they put in.

[00:27:33] So at the moment, we have the student activities and engagement coordinator is the only person in the high school who can enter student participation in clubs, and we're tracking a number of different clubs' participation, but we're not tracking all of them, or we haven't been, because it's a heavy lift for one person to enter all of the clubs.

[00:27:53] We have hundreds of them in the district. So we have the ones that they've been entering into Phoenix are ones that are in the majority of schools. We did come up with a solution for something at our meeting last week because the clubs that are not being tracked might some of the student population groups that have seemingly low numbers of participants, it could just be because we're not tracking the clubs that they're involved in.

[00:28:26] So I'll be creating for this year three sort of catch-all categories. One will be affinity groups, one will be interest groups, and one will be service. So some of the smaller clubs that maybe only two schools have will be caught.

[00:28:47] The data will be captured by being included in these catch-all groups, if that makes any sense. So I believe our participation is higher than what these numbers are showing. So, which, I mean, they're respectable as it is, but I think it's actually higher.

[00:29:06] These are some quotes from some student leaders in the district I asked last spring for, you know, their opinions on how student voice is expressed. And you can see they believe that they have opportunities in clubs and organizations to, you know, raise awareness about things, share ideas, influence school culture and participation gives them a place to feel like they belong.

[00:29:36] And I was really thrilled to hear you guys talking about belonging because I've been an advocate for student activity participation for over 40 years. And I can tell you that there's all kinds of research at the national level that shows that kids who are involved in student activities do better on every measure you wanna measure.

[00:29:54] Attendance, grades, pro-social activities, everything. and our friends in the research office have come up with a really nifty new app that we can now get that data for our students and it correlates with the national data.

[00:30:12] The year before last, two of our student activities and engagement coordinators drilled down into the Phoenix data and looked at the students who were involved in student activities versus students who were not. And they found that students who are not engaged in school activities have on average more than four additional days of school absences than kids who are involved.

[00:30:36] And nearly 20% of the students who are not engaged have been flagged as chronically absent. So that, I believe, was sort of the precursor to the app that our office and the research office developed, which was just unveiled last week to the Student Activities and Engagement Coordinators.

[00:30:55] And it's really exciting because you can see by school what the participation levels are for the different demographic groups in the school. Okay, so, and then they can connect that with the engagement app and look at, see who's not engaged and try to work to get them involved.

[00:31:14] And a lot of people think that the, you know, so-called better students are involved in activities, but the research shows that if you get a student who's not involved engaged, their performance improves. So, it's a powerful tool.

[00:31:33] Next slide. This one shows you as far as directly related to student voice. This is the student initiated new club request approvals. We have a process at every school. There's a Google form that any student who wants to start a club, if there's not one at their school for an interest that they have, they can fill out the form.

[00:31:52] They need to find a faculty sponsor and five students wanting to be involved in it to be eligible to be approved. It goes through a review process. I review them, the risk management folks review them, their principal reviews them, and then they're either approved or not approved.

[00:32:10] And we had 362 new clubs this year across the high schools. So you can see the breakdown by school up there, how many were at each school. It's a really powerful way for a student who wants to make a difference and get something done to show their voice.

[00:32:37] As a side note, I would like to point out that that's 362 faculty members who took on extra work for no pay. This is a sampling of the interests of the different groups that were started this year.

[00:32:58] The wide variety of interest areas at the top is linked to the list of all the new clubs, if you're curious. But I just sort of broke down some of the common ones. There's everything from academic interests to affinity groups to careers to fitness to games.

[00:33:15] Dungeons and Dragons clubs were popular this year. It's coming back. It never was. Competitive Pokemon, which you wouldn't think at the high school, but it's there. And then lots of other interests as well.

[00:33:31] So, you know, kids have lots of interests, and I'm happy to say they're being able to express them in their schools. And then this next part is just a few highlights to give you some anecdotal information as well.

[00:33:49] The WILD Student Leadership Conference, we had our third one this year. That was, I'm starting my fourth year in this position, and that was one of my first charges was to do this conference. So we've had it for three years now, and I'm happy to say it's flourishing.

[00:34:06] We were actually a little bit smaller this year because we held it at Watson Mountain on a school day, and we were limited to the size of the auditorium, so we only had 350 student leaders from around the district, but we had 34 breakout sessions that were student-led, student-initiated.

[00:34:27] They did their own proposals for what breakout sessions they wanted to do and then presented them. So we had 76 student leaders presenting, 34 breakout sessions that all the students went to. We had a guest speaker.

[00:34:44] And the way I've structured this is Dr. Spence, when he spoke at the conference, said, Wilde could also be we're investing in leadership development. And it's true. And I believe we should be investing in the younger students. So only students in grades 9 through 11 can attend the conference.

[00:35:03] The seniors, if they want to attend, they have to present. so it's it's kind of a dual leadership development thing because they can attend and learn stuff and become you know better leaders and then when they get to be seniors they get to give back and develop their leadership skills in a different way by by sharing what they've learned and presenting to to the younger kids there's a few comments from students who attended and I think they all really appreciated it I get a lot of positive feedback from it and then the next thing is

[00:35:36] Another committee that our office, the Athletics and Activities Office, does, Derek Ferry, our director of athletics, who's here tonight, meets with student athletes, two student athletes from each school.

[00:35:52] They meet to talk about athletic topics and such, and they've developed a new sportsmanship creed that they liked because it wasn't just a laundry list of, like, sportsmanship topics.

[00:36:07] which the next slide has the the new creed they developed this new creed last year and then they've just recently finished filming um a couple of videos one for parents on sportsmanship and one for student athletes on on sportsmanship with basically with giving examples of um what good sportsmanship is so um that's another way for students to you know have a have a an influence in their schools and then um i put some sca student council association examples up here from

[00:36:43] various schools i think our student councils do a really good job of of reaching out and trying to engage all the different um demographics in their school different groups they highlight i i always like to go around and visit when they have pep rallies just because they're fun and i like to get out into the schools and i have seen so many different kinds of groups being highlighted in the pep rallies, not just the typical athletes. You can see there's a photo up there of an Indian dance club performing at one of the pep rallies. And that's common throughout the district for

[00:37:19] different groups to be highlighted. And a few quotes for you guys to see from students of what they believe their SCA is doing with student voice. And then the last one, it's a lot of words, but I thought you might like to hear from some of the students of what they believe the impact of their involvement in student activities is.

[00:37:48] So I think that's it. Thank you very much. You're welcome. So I'm wondering, do we even look at elementary and middle school for any of this? Because I know that my kids, they were in elementary school SCA at one point, and they had a lot of activities.

[00:38:08] And I know that we're starting a pilot program for after-school activities, and we have the middle school athletic, the sports thing that we're doing. And I know that the kids also do, like in middle school, they often have a drama musical of some sort, like all of those kinds of things.

[00:38:34] Are we tracking any of those, like in elementary and middle school? The participation we have not been tracking. It's certainly something that we could look at doing. The difficulty for me is there's not a designated person in each middle school that I can easily get to to coordinate things with.

[00:38:57] Every school does their activities differently at the middle level. High schools all have the student activities and engagement coordinators, so it's easier. I have done some things with middle schools, more often the student councils than other groups.

[00:39:14] We did a program last year for what was called Digital for Good, where we had half of the middle schools opted in to come here for a training. And they brought teams of eight student leaders to get training on how to intervene in their schools to squelch digital drama, social media drama and things like that.

[00:39:35] So we have done some things, but there's a lot of room for more. And one quick thing I'll add, so we can certainly get the information on the middle school intramurals. That's not a problem at all. One other additional challenge I think that Ms. Fiscus has at the elementary level is that, and it's just a function of the way things happen.

[00:39:57] It's not, challenge is too hard of a word. Many of the elementary school activities are run through PTAs. They're after school. They're not considered school activities. And that's a significant difference from high schools that have club time during school, are run by sponsor.

[00:40:19] They have to be run by sponsors that are staff members. So it's just a little more difficult to collect the data. And even when we do, it wouldn't be consistent. The middle school intramural program would be a good place to start for looking at the data on participation.

[00:40:38] we'll just need to sort out how we can get that information into Phoenix because as I said the way it is at the high schools the student activities and engagement coordinator is the one who enters they get the lists from the club advisors of who's participating in their clubs and then they enter it into Phoenix I mean I know we have equity ambassadors as well I mean that's a huge student voice yeah and um so I just and I know that they're that we wanted to start that earlier in middle school i know that we have middle schools so well i'm always for i mean schools are our communities right so get the kids involved in their community it is one of my goals

[00:41:18] to be more involved with the middle level programs this year thank you so much um this is really great i um my kids are at harper park both of them and they're making a switch this year uh they are adding clubs into part of their advisory and so because they were having a ton of transportation issues. One, they were seeing a huge drop-off because of the intramurals, and people were competing, and then when kids were in the drama production, they were missing their clubs, and so

[00:41:55] they're trying to give more equitable access to clubs, which I'm really excited to see how that plays out, and I love that they're thinking about that. And so I'm just wondering, what do the challenges look like in how you discuss that at the high school level as well? Do you guys get together and talk about what are some of the things that are hindering kids from clubs or some different factors because if it if it's done differently at every middle school I'm wondering at the high school level is it more standardized and are you building off of those barriers or how does that work because when I see the school listing and I see you know some

[00:42:32] schools have a ton and some don't and I know the data probably isn't fulsome how does that process work for your office and working with them in terms of troubleshooting I guess is my big question um we do have monthly meetings with the student activities and engagement coordinators and and those meetings are a great opportunity for them to share ideas and best practices so we talk about different issues and what they're doing to get more people involved and and how they work with their club advisors and so forth so that's a good it's a good problem solving group because they

[00:43:11] often will share ideas of like what works for them yeah i guess this kind of goes back to your point earlier chair mansfield about the consistency not only with data practices but with these kinds of things like what's working for one maybe could work for another um and then i is there some way that we can get a program or something for ddi to help us enter these numbers i mean it seems like if one person is sitting there entering all these numbers but they can't keep up with it is that a is that a good use of their time? I'm just wondering. And B is, we're like, you have an app

[00:43:47] now. I'm like, can we make this happen? Okay. So we started getting them to put this data in when I started this position and, and now we finally showed them last week, okay, this is why you've been doing all this because now we, we can do something with the data that you've been entering.

[00:44:05] Right. So I think they have a better grasp on why they should be better about, you know, like some of those schools that only had 300 and some people you know there's more kids in that participating that it's just they didn't put the data in so i i firmly believe that it will be better this year and our numbers are probably going to skyrocket because they're because now they know why they should be putting it in there right um they know better why they should be putting it in there um and it's honestly it's not that difficult to do because if they get the list from the advisor of the names and the student IDs, they can just upload the list.

[00:44:43] It's not like they have to do it one person at a time. But it is only one person that does it all. And I ask them to get the data in by November 1st, and then we do it again second semester right at the beginning of third quarter just to make sure we...

[00:44:59] So it's not like they have to do it every single week all the time. No, and we don't... Okay, that's a good clarification. Yeah, yeah. I have two more questions. Should I go ahead? Please. Okay. One other question I had is, actually, maybe it's only one. I had a high school actually reach out to me.

[00:45:16] They saw this was going to be on our agenda, and they mentioned that this year that clubs need to host at least one event a semester or their club will be terminated. I would just love to hear if that's the case. I mean, sometimes stuff gets translated to us.

[00:45:33] The terminated part, I think, was a locally stated thing. Okay. Okay. So, yes, I worked with the SAACs last spring to come up with a list of advisor responsibilities for an activity advisor.

[00:45:52] And it was a list of these are the things we expect of you because there wasn't anything. You know, people could just do whatever. And these were basic things like follow the fundraising guidelines and make sure you use the proper forms to request field trips and fundraisers and things like that.

[00:46:10] But we did have things, and this partly goes to some of those 362 clubs that you saw up there were kids who wanted to be able to put on their college application that they started a club. So they start them and then they just fizzle out and don't do anything.

[00:46:30] So these guidelines, and I can share them with you so you can see what they are, none of them is onerous, are basically to have legitimate clubs, right? And to participate in an activity each semester, it could be a school-wide thing.

[00:46:51] So, you know, if there's an event, one club could sell the concessions and another club could take tickets at the door, you know, whatever, but just to make sure that they're actually active and doing things. i appreciate the clarification and your willingness to answer um i figured there was more more to that i love the service component of that i think that that could be something that could be really powerful i love that that could be something they could serve their community together i love that i appreciate your work so much this is such a critical part of what we do in our schools and

[00:47:27] you know kids the academics we we hear all the time is so much pressure and you know we know that the research tells us that these communities and these outlets are crucial to not only mental health but to their success beyond the classroom so i just really appreciate the work that you do it's a big task so thank you thank you thank you uh so i just have one or two comments first um i i can't get over how amazing some of um our internal in-house conferences are uh that are kind of leadership focused so wild being one of them but but and i'd love to come to that as well

[00:48:02] um january 23rd at independence high school wonderful um so i think so i i grew up in a school system that was localized right didn't could not take advantage of these kind of countywide opportunities and so um just being able to see how much we actually do that where we bring students from the same class or from different grades together across the division um to interact and to extend that voice that they have has been wonderful for me to see 'cause I didn't get to experience that.

[00:48:37] I also just wanna kinda broaden the conversation around student voice. When I, one of the ways I think about student voice is around students' ability to influence and advocate for themselves around instruction and classroom settings.

[00:48:53] And as I've visited schools and seen the different classrooms and seeing the setups even, like some of them have couches or open space and circle desks, depending on what the topic is and the discretion of the teacher. And there's that component.

[00:49:10] So there's instruction, classroom, and potentially policies as well that we've seen around cell phone usage. And then the other side of that is, and when I think about that, I also think about students that maybe don't belong to affinity groups, but maybe specific students that are EL students and their experience and their voice or students with special needs or abilities and seeing how they might kind of, their voice might come out in a school setting or in a school community.

[00:49:47] So just wanted to kind of throw that topic out there. We don't have to address it at this moment, but that'd be a really good topic. And it would be great to hear maybe from students or specific school communities on those types of things. The two groups you mentioned, English learners and special needs, those were the ones when last week I mentioned that we, one of the SAACs pointed out, she said, because when you look at the new Click app, it shows that Hispanic participation is lower than it should be, and also students with disabilities and English learners.

[00:50:25] And she said, but the clubs that my Hispanic kids are involved in are not ones we track. And so that's why we came up with this affinity group thing so that we can tag students as participating in an affinity group, not necessarily the specific one.

[00:50:45] So those numbers should show some improvement, I think, just because we haven't been tracking the right thing. That makes sense. And then I lost my train of thought. It was about middle school participation. No worries.

[00:51:03] Oh, well, they'll come back to me. And then the last curiosity was maybe around counseling. So I've attended some of the, I'm trying to remember, quarterly counseling, council, yes. So, and I've lost kind of the name of it, but the student participation in that council at the school level has been amazing to see.

[00:51:32] So just wondering about that as well. School advisory council. School advisory council. Yes, sorry. I know what I was going to say is that it's not student activities per se, but Dr. Singel and I worked last year with the Student Advisory Committee, the Superintendent Student Advisory Committee, and that is an amazing student voice vehicle because I don't know if you guys came to the presentations in May where the students gave their solutions to the problems that they had talked about all year long, but that gives them a real opportunity to make a difference in the district because as we told them you're going to have the ear of people

[00:52:12] who can make the changes that you want made so here's your chance and they really took it seriously um so you were talking about the kids who wanted to start a committee wanted to start an activity or a club and you know some of them might not have the skills is there like a video or something that people have made this is how you start a club and this is what you're responsible for and this will You know, that kind of thing.

[00:52:41] There is not, but that's a great idea. Something like that. Something, you know, fun. Or that if you want to start a club, come to this conference or whatever. Come to this, you know, seminar that we're providing here at the admin building.

[00:52:58] And this is how you do it. That's a great idea. Break out. We do have a group that's district-wide called Pando, which is presidents and officers. It's the student council presidents and the class council officers of all the high schools.

[00:53:16] And I think that would be a great job for them to put a video together to share how to start a club. And then, so what's a grandma club? Grandma? Is that one of the ones on there? Yeah, that was listed. Whoever, if anybody's listening, I have to know what a grandma club is.

[00:53:33] My favorite is the Trader Joe's Club, where they try new items each month from Trader Joe's. And Dr. Spence and I have asked for an invite, and we have not been invited yet. So if you're listening, we're still waiting. I do not know what the Grandma Club is.

[00:53:49] I can find out. Thank you so much. You're very welcome. With that, we'll move on to 4.03, Department of Academic Surveys.

[00:54:07] Dr. Ellis? All right. I am going to turn it over to the amazing Dr. Tyler and Dr. Blake Locke. Hello again. A topic of interest amongst many school boards has been looking at our surveys and maximizing our survey responses.

[00:54:27] So Dr. Blakelock is here to provide an overview of the current practices and another, an alternative, a methodology for gathering feedback from our invested parties. Thank you, Dr. Tyler. Going back to the beginning of the conversation we had today, when we're looking at things like measuring school climate and sense of belonging, those are things that are very much measured through surveys.

[00:54:51] So what we want to talk about today is what our current distribution practices are, some response rate concepts, and then as Dr. Tyler just teased a little bit, perhaps a different way to look at things as well. So if we could go to the next slide. So right now, what the process looks like when we're distributing a survey is several months in advance, we'll actually develop a communications plan with DCE, and then we'll distribute that.

[00:55:16] We translate each of our surveys into five different languages, and when it's time to actually email it out, we send it to the first two contacts that are in parent view. So a typical family survey distribution is about 120,000 emails that are going out at once. And one of the things to consider with that is sometimes one parent will take it and not another, which will have a little bit of an impact on the response rate overall, and just something to consider. We watch participation very closely, actually, if not daily, at least twice a day, and we'll create dynamic links to share with other division leaders so that they can see what participation looks like at a school

[00:55:56] level or in different areas as well every time they click it they get current live data we work very hard to troubleshoot any survey access issues so we have the research at lcps.org website we get a lot of emails from folks we can't find the survey link the survey link's not working a lot of cases we'll have to troubleshoot to say okay is this actually an lcps parent and then kind of walk it through try to find their link if not go ahead and email them a link and then throughout the process as we're watching participation when we see participation start to hit a lull we will send a reminder out to all non-participants that you know here's a survey the link is right there

[00:56:35] to try to boost that participation as well we tend to send those emails at times that people are most likely to see them so you know sometimes five o'clock in the morning sometimes 4 30 just to kind of make sure that when people are naturally looking at their email it's towards the top of the list. So looking at response rates, we've got the positive and negative impact as well. And I think one of the things we want to think about is that you can probably agree that we are an over-surveyed society. You know, you go out, you know, you get a Starbucks, you get your haircut, and you pump gas. And by the time you get home, you've got three surveys sitting in your inbox waiting for you. And it's something that we're trying to kind of think about as we do this.

[00:57:15] We've actually recently come up with an SOP that's been very helpful as far as what survey practices look like to try to keep an eye on how many surveys the division's putting out in a given period of time. So looking at that positive impact piece, we're looking at communication. So we've talked about the communication plan, the easy access so that it actually comes to the person rather than having to send the person to it itself. And then the trust and benefits are kind of tied together and they go with what's known as cost, which goes over in the negative impact, which is kind of maximizing trust, sharing the benefits. So confidentiality, anonymity, you know, it's a

[00:57:51] safe space to share. Here's what's in it for you in taking the survey. This is how it will help you. And then the big piece is going to be closing the loop as well. Once the survey is closed, results are reviewed, what action steps are going to be taken as a result of that data and communicating that back out into the community as well. And you start building the cyclical process of people like, oh, you know, the voice does matter. I'm going to weigh in on that.

[00:58:14] On the negative impact side of it is going to be survey fatigue, which I just alluded to because we get many, many surveys in our lives. And then the other two are what speak to, I mentioned the cost, is the length of the survey. And we tend to think of length as not necessarily the number of questions, but the amount of time it takes somebody to complete the survey.

[00:58:35] And then the other aspect of that is going to be cognitive load, which is basically how hard people have to think when they take the survey. So when you have a lot of open-ended questions, that increases cognitive load. When you have people ranking, you know, seven or eight things in relation to yourself, those are things that increase cognitive load that may have people say, all right, I don't want to take this anymore, and just stop taking it altogether.

[00:59:00] So as we move to the next slide, and we talk about sampling, on the left-hand side is our current practice, which is known as census or non-probability sampling. The pro of that is everyone gets a voice.

[00:59:16] Everyone has a chance to do it. In our current practices, when I talk about 120,000 emails going out, it takes 1,200 responses to get one percentage point of participation. So when we think about getting it up to a higher rate, you're really looking at a lot of participation.

[00:59:36] When the survey is all said and done, it is not necessarily data that is generalizable. What we want to think about is, and I think it's been mentioned from the days before, what's known as non-response bias, which is like, what are people who didn't take the survey thinking? How representative of this is? One of the ways that I've encouraged other folks to think about it and work I've done across the country is if you've got 10,000 responses and 25% of them are disagreeing with something, then you know you have at least 2,500 people who are saying something's not happening. So it's a way to think about as far as breaking it down. But the reality

[01:00:12] is that there's no industry standard for response rate. You ultimately get what you get, and you have that data to work with. In a previous life, did this work across the country quite a bit. 10 to 15 percent for families was probably about the standard. Maybe in a smaller district, they got a little bit higher, but it's really hard to get them to engage in the survey in a way that they'll complete it as well, which leads nicely to the next side of the other column over there, which is random sampling and what this speaks to is a much more rigorous research methodology where you're

[01:00:50] actually taking a random sample of people for a district our size roughly three to four thousand would probably would cover what we need to do to have the statistical power we would need because it's random it is representative of the whole when you have the response rate if you have low response rate you're actually taking response rate into account and calculating a margin of error. So you'll know that you'll have a range in which the responses fall based on how many people responded. So minimizing that non-response bias and you're able to account for that. The other piece of it as well is that you could do some weighting to it so that you take the people who

[01:01:28] did respond and weight different based on different reporting groups so that it does look more representative of the overall population. Now, if we were to do that, we would report it as both weighted and unweighted so that we have that transparency there. But those are some of the benefits of doing the random sampling. The other piece is that people are not getting a lot of surveys because it's the random sample. They may get called up a few times with it, but ultimately it's fewer surveys going out from the end user from them receiving it. Of course, the con of that is the perception of not being included. You lose that idea of everyone has a voice that you would

[01:02:06] see in a census sample, but from a back-end perspective, from a methodological perspective, you can glean a lot more from the data and look at it in a more generalizable fashion than you can from the census sample itself. So I'll now pause and see if there's any questions.

[01:02:22] Thank you very much. So if I could add to that. So there's options, meaning we're not necessarily expecting a complete pivot from census sampling to random sampling, but it leaves the option of Dr. Blakelock and his team having the ability to maybe do both in the spring and that we could actually have a random sampling scenario where we could get a response rate and get a better sense of how representative it actually is in comparison to the census sampling because there have been concerns from you and previous boards about is a 15% response rate, what is that really telling us about what the community feels?

[01:03:00] So if we could have a scenario where we have a random sampling implementation in the spring along with the census sampling, we could at least test the waters of considering this because we don't expect a complete switch from census to random sampling. So I hand it back over to the board members for a discussion.

[01:03:17] I think something that really bothered me with the calendar survey is that we had teachers, we had teachers as parents, we had teachers as teachers, and then we had teachers with the superintendent advisory committee.

[01:03:37] So talk about weighted, that was pretty high. And that was more of a census sampling, right? This sounds like my research and design class from like 30 years ago.

[01:03:54] So my thought is that there's probably some surveys that are more appropriate for census and some surveys that are more appropriate for random sampling. And while I totally appreciate what you're talking about, like comparing the two to begin, I still think, even so, we're still, I think that there's still some questions and surveys that are better done randomly than other words.

[01:04:29] It just means you guys get to do more math. So I'll also just add, so from my perspective, I like the idea of experimenting. Just one of the hesitations that I have is not just that a potential respondent's voice may not be included.

[01:04:52] but that there is a reason why this communication is going out, and so as a school board, we also think about how do we get people to be aware of these discussions that are taking place, and the surveys also hold, though they might not get a response, they hold, I think, an important, they have a, it's a way to communicate to the community, right?

[01:05:17] And so the fact that something is being considered and we're asking for that input, even if we don't get a response, it's important that the community hears that we are seeking input on this issue. And sometimes those types of things, more and more I think each year they're becoming, it's something that we need input sooner rather than later if it's not a consistent survey.

[01:05:45] I think the other thing I wanted to add around timing of surveys is just wondering, I probably need to see a visual at some point of what are all of the surveys that go out at different points in the year maybe that might help me give better feedback on this.

[01:06:01] But I think one of the ideas that immediately comes up and you've probably all thought about this already is maybe we can consolidate some surveys altogether. And even though there might be more cognitive load or time, you can tie them to a back to school night or in a way that you know you're gonna get the responses because you tie them to something they have to do already, so maybe some conferences or expos.

[01:06:31] If I want that treat that's sitting at the table in an exhibitor, whether it's a USB drive or a hat, usually it's going to come at some type of a cost. So I have to fill out a survey or do something else. But yeah, so that's just the only other thought I had around ability to consolidate them using some type of way to tie it.

[01:06:55] So that's a great point, And that's actually a practice that we're engaging in at this point now. So actually part of that SOP is coming up with a survey calendar for the year, which is on the research website. There's not a lot on there right now.

[01:07:11] But as surveys come up, people can grab real estate on that so that they have that time. And we did actually have two surveys coming up in two weeks. We're going to be doing a survey around special programs and the capital improvement plan. Both came separately, and we have combined them together, and we've sewn them together in more of a cohesive way.

[01:07:30] We're doing a slightly different way of, we're not doing the emails, we're actually doing a link where people can put in their email address, and it'll give them that single-use access to it, which will be happening at events at schools. There will be a QR code with that as well, so we are looking to market those high-volume areas as well as combining those surveys to reduce the number of surveys.

[01:07:49] We actually did that last year as well with the bell schedule and school calendar survey, I think it was at the beginning of the school year. So we are trying to combine surveys as much as possible right now. Thank you. I will try and streamline my comments just for time's sake, but I, and maybe we'll just have another time where we get together and chat because I have so many thoughts about this and I'm just so many mixed emotions because feedback is so critical to our role, but it's so hard to get.

[01:08:21] And then it's so hard to say, like this feedback is the feedback that I can rest my case on, I've struggled with that so much because, and sometimes it is, it's like, do the teachers matter more here or do the families matter? Well, they both do, you know, and you don't want to, and the calendar is the perfect example, right? And I, you know, sometimes joke that it's still like I'm recovering from the calendar conversation. And I feel like I am because we were put in that difficult situation.

[01:08:44] And on top of that, you had the focus groups that were also a part of that. So it was another layer of like tangible feedback that we knew we got I don't know anyway like I said I have I need to sort through my thoughts on all of this I I think a comparison of random and census is going to be necessary in order to even just bridge the trust gap I don't think the trust is there with the full public yet I think we're still working towards that and so if you go to random sampling I think that that could threaten to diminish trust further because people are going to say well how'd you

[01:09:20] account for bias and well who did you sample and how did you pick them and you're not picking my people you know so I just I we get a lot of that so I'm just projecting that out I do wonder about our survey sometimes being hard to understand being accessible to everyone having more of a leading question versus something that's really black and white like a yes no you know like do you like the LCPS calendar? Yes or no? I would like to see that data. And then I am also really

[01:09:53] wondering and marinating on AI because something that I do on my social media is I'll blast out a topic for feedback. Really the only ones that get a lot are calendars. So calendars and early start times, bus schedules, big, big time get response. You know, I'm talking hundreds of people commenting and liking and angry faces and all the things.

[01:10:15] And I'll just put that into AI and say, give me the high-level summary and what were some solutions people gave and why are people mad and doing some analysis piece. So I'm wondering how our surveys will evolve as AI becomes a tool that we can use to generate that.

[01:10:32] And I would also like to have a solution-driven side of our surveys. Like if you're mad, tell me how we can fix it. What's your idea? Maybe you don't have the full context of what LCPS can accomplish or what the school board is, what's even in our lane, but at least tell me your ideas.

[01:10:50] Because just answering a ranking, I feel like there's so much more there that we hear later that we don't hear as part of the feedback process. So, again, I have tons of thoughts on this. I think surveys are important. I think we need to do them. I agree with Arben, Mr. Streffy, sorry, wholeheartedly that it does at least put awareness.

[01:11:09] You get a, what would you like in a new superintendent survey? You wake up and say, oh, are they getting a new superintendent? Some people are not plugged into all of that. And so I do, you know, oh, what do you think about the grading policy? Oh, they're updating that. It is a great way for us to share information with the community, but it also comes with so many nuances that sometimes are we using it to its full advantage?

[01:11:36] So that's my soapbox. Thank you. If I may engage just the AI part of the conversation for just a moment, So there's two things to share on that is, one, so we have used versions of AI before. So when it came to the feedback around when the PD days were going to be a year and a half ago or so, when we put out a Let's Talk form and got feedback, we used some unsupervised topic modeling to be able to pull out what are the solutions in this, you know, 5,000 responses that we got to Let's Talk.

[01:12:02] So we're able to do it that way. I've recently been through a training through the American Evaluation Association about how to best use AI with analyzing qualitative data and sort of that it's not just one prompt, it's multiple prompts and being able to give it the local context so it can kind of get really, really good quality, deep analysis of what it is you're looking for.

[01:12:20] So it is a direction that we are definitely moving in and we'll continue to pursue that path. And just to piggyback on what Arvin asked, Mr. Streffy, I'm sorry, I'm being so informal, about would like to see all the surveys that we give.

[01:12:35] I would like to see that and also the response rate just to kind of pull out what matters to most people. Like, what are people responding to? What piques their interest? Is it, I mean, I know calendars is going to be probably number one, is my guess, but I'm curious about what themes people are most interested in.

[01:12:53] I'm venturing it's what impacts their daily life, but you never know. All great comments. Any other discussion? Okay, well, seeing none, Dr. Ellis, if we don't have anything else, then we'll move on to new business.

[01:13:14] Is there any new business? Seeing no new business, we'll now adjourn. August 19th meeting of the Performance Monitoring Committee. Thank you all very much.

----------------------------------------------------------------------
Coraggio Consulting · aguyonx.com