What is the best way to build a district-wide edtech adoption scorecard that tracks usage across all schools in 2027?

The renewal meeting that started this

A curriculum director walks into a spring budget review with eleven edtech renewals stacked in front of her totaling a meaningful share of the department's discretionary spend. The superintendent asks a reasonable question: which of these are actually being used? She has a vendor-supplied PDF for four of the products, a spreadsheet a principal built for one building, and nothing at all for the rest. The vendor reports each define "active user" differently — one counts any login in the last 90 days, another counts a session over five minutes, a third counts any account provisioned at the start of the year whether or not it was ever touched. None of them break out results by school in a way that lines up with how the district organizes its own buildings. The conversation ends with a decision to renew everything, because nobody can defend cutting a tool without evidence, and the evidence does not exist in comparable form.

That is the problem a district-wide adoption scorecard solves, and it is worth being precise about what "solves" means. The scorecard is not a dashboard for its own sake. It exists to answer four questions with the same numbers every time they are asked: What did we buy? Who has access? Who is actually using it? Is that usage concentrated in a few classrooms or spread across the building? Everything else — the charts, the color coding, the tier labels — is presentation on top of those four answers.

The scale matters for design. A district of twenty schools with fifteen contracted products is tracking roughly three hundred school-product pairs. At forty schools and thirty products it is twelve hundred pairs, which is past the point where anyone maintains it by hand. The build has to assume automated data collection from the start, because a scorecard that depends on someone emailing principals for numbers each month will produce two good months and then quietly stop. The single most common failure mode is not bad data — it is a scorecard that decays because its refresh depends on human effort that nobody has budgeted.

The other thing the scenario reveals: the hardest part is not technical. It is agreeing, across the district, on what a single number means. If one school's instructional coach believes "adoption" means teachers have accounts and another believes it means students are producing work in the tool, the scorecard will be argued about rather than used. Settle the definitions before you write a line of pipeline code, and write them down somewhere principals can read them.

How the data actually gets from vendors into one table

The scorecard has a plumbing layer and a presentation layer, and the plumbing is where the work is. There are four realistic sources of usage data, and most districts end up using three of them at once.

Vendor APIs and reporting endpoints. Larger platforms expose administrator reporting — some through documented REST APIs, some through scheduled CSV exports to SFTP or cloud storage, some only through a UI export a human clicks. Ask every vendor during procurement whether they offer programmatic access to usage data at the school level. If the answer is no, that is a fact worth recording in the scorecard itself, because "we cannot measure this product" is a legitimate input to a renewal decision.

Single sign-on logs. If your district routes app access through Google Workspace, Microsoft Entra, Clever, ClassLink, or a similar broker, the broker's own logs are frequently the best cross-vendor dataset you have. They are consistent by construction — one definition of a login event, one timestamp format, one user identifier — and they cover every app behind the broker regardless of whether that vendor cooperates. The limitation is real and should be stated plainly on the scorecard: an SSO log records that someone opened the tool, not that they did anything meaningful inside it. Treat SSO data as a reach metric and vendor data as a depth metric.

Rostering and SIS data. This is what turns raw events into a rate. A login count is meaningless without a denominator, and the denominator comes from your student information system and rostering layer: how many teachers are assigned to this school, how many students, which grade bands, which course sections. Rostering data also carries the school identifier that lets you join everything together.

Manual attestation. For products with no API and no SSO integration, a short quarterly form from the building leader is acceptable as long as it is clearly labeled as self-reported and never mixed into the same column as measured data.

The joining logic is where districts get stuck, and it comes down to identity. Vendor A knows a teacher as their district email. Vendor B knows them as a numeric ID assigned at provisioning. Your SIS knows them as a staff number. The scorecard needs a crosswalk table mapping every external identifier to one internal person record and one school code. Build that crosswalk once, store it as its own table, and have every ingestion job write through it. School codes deserve the same treatment: keep one canonical list with a stable code per building, and never let a vendor's freeform "site name" field become the join key, because it will arrive as "Lincoln HS" one month and "Lincoln High School" the next and silently split one school into two rows.

Store the result as a single fact table with one row per school, per product, per week: school code, product, week ending date, seats provisioned, distinct active users, sessions, and whatever depth metric that vendor supports. Everything else on the scorecard is a query against that table. Weekly granularity is the right default — daily is noisy because of school calendars, monthly hides the pattern where a tool is used hard for two weeks and abandoned.

Refresh cadence: pull weekly, publish monthly. Weekly pulls keep the gap small if a job breaks; monthly publication matches the rhythm at which anyone can actually act. Add a freshness stamp on every view showing the last successful pull per product, so a stale number is visibly stale rather than quietly wrong.

Choosing metrics, denominators, and tier cutoffs

The metric that carries the most weight is active users as a percentage of seats provisioned, computed per school, per product, over a rolling 30-day window. Two design choices inside that sentence do most of the work.

First, the denominator is seats provisioned, not seats purchased district-wide. If a district buys 4,000 licenses and a school provisioned 300 of them to its staff and students, that school's rate should be measured against 300 — otherwise every small school looks like a failure. But the district rollup should also carry a separate line for seats purchased versus seats provisioned, because unprovisioned seats are pure waste and that gap is often the single largest recoverable dollar figure the scorecard surfaces.

Second, "active" needs one definition applied to every product, even when vendors define it differently. The practical approach is to define your own threshold — for instance, a user counted as active if they had at least one session in the trailing 30 days that exceeded a minimum duration — and compute it yourself from event-level data wherever you have it. Where you only have a vendor's aggregate, mark the cell as vendor-defined and do not compare it head-to-head with a self-computed number. A scorecard that quietly mixes definitions loses credibility the first time a principal notices.

Beyond the headline rate, four supporting metrics earn their place:

• Breadth: distinct teachers with any activity, as a share of teachers rostered to the relevant grade bands. This separates "the whole building uses it" from "three enthusiasts use it a lot."
• Depth: a product-appropriate output measure — assignments created, lessons completed, minutes per active user per week. Depth is what distinguishes a tool that was opened from a tool that was used.
• Consistency: number of weeks out of the last twelve with any activity. A product used in three concentrated bursts scores differently from one used steadily, and consistency exposes the "we log in during walkthroughs" pattern.
• Cost per active user: annual contract cost divided by 30-day distinct active users. This is the number that ends renewal debates, and it should be shown at both school and district level.

For tiers, keep it to four bands and apply the same cutoffs everywhere. A workable starting frame: Embedded for schools above roughly 70% active-to-provisioned with activity in ten or more of the last twelve weeks; Established for roughly 40–70% with regular activity; Emerging for roughly 15–40% or high rates concentrated in a small share of teachers; Dormant below roughly 15% or fewer than four active weeks in the last twelve. Treat those numbers as a starting point to calibrate, not a standard — run them against a full year of your own historical data first and adjust so the bands actually separate schools you already know to be different. If 80% of your school-product pairs land in one tier, the cutoffs are wrong.

Two calibration rules that prevent embarrassment. Set a minimum denominator — do not assign a tier to any school-product pair with fewer than a small handful of provisioned seats, because a school with 4 licenses swings between 25% and 100% on a single person's behavior. And segment expectations by product type: a daily-practice math platform and an annual assessment tool cannot share a benchmark. Group products into categories — core instructional, supplemental practice, assessment, productivity, administrative — and only compare within a category. Cross-category comparison is the fastest way to produce a chart that is technically accurate and completely misleading.

What you gain and give up with each build approach

There are four realistic ways to build this, and the right choice depends far more on who will maintain it in year three than on what looks best in a demo.

Spreadsheet with manual or semi-automated imports. Fastest to stand up, essentially free, and genuinely appropriate for a small district or a first pilot covering five or six products. It fails predictably at scale: version proliferation, broken formulas after someone inserts a column, and total dependence on one person. If you choose this, choose it explicitly as a time-boxed pilot with a decision date, not as the destination.

Business intelligence tool over a district data warehouse. The most common durable answer for mid-size and large districts. Load the fact table into whatever warehouse the district already runs, build the views in the BI tool the finance or research office already licenses, and reuse existing access controls. The advantages are real: row-level security so a principal sees their building, scheduled refresh, and version history. The cost is that someone must own the pipelines, and BI development is a skill that may not sit in the technology department. Budget for the ongoing maintenance, not just the build.

A vendor-supplied adoption or app-management platform. Several vendors in the rostering and app-management space offer usage analytics as part of their product, and if you already run one as your SSO broker, its built-in reporting may cover most of what you need with no build at all. The trade-off is definitional control: you inherit the vendor's definition of active, their school hierarchy, and their release schedule. You also inherit a coverage gap for any product not behind that broker. This is often the right call for districts without data engineering capacity, and it pairs well with a small supplemental build for the products it misses.

Custom application. Warranted only when the district has genuine software capacity in-house and specific requirements nothing else meets. Otherwise the maintenance burden outlives the person who built it.

One trade-off cuts across all four options: how much you invest in normalizing vendor definitions versus accepting them as-is. Full normalization — computing every metric yourself from event-level exports — gives you clean cross-product comparison but requires event data many vendors will not provide and multiplies the engineering work. Accepting vendor aggregates is fast but means your comparisons are approximate. The pragmatic middle: normalize within product categories where you can get event data, and clearly label the rest as vendor-reported. Two visibly different classes of number beat one falsely uniform one.

Also weigh who sees what. Fully open scorecards where every principal sees every school's tiers create peer pressure that sometimes drives improvement and sometimes drives gaming and resentment. A common compromise is that each principal sees their own school's detail plus the district distribution without other schools named, while cabinet-level users see everything. Decide this before launch and say it out loud, because changing visibility after the fact reads as a shift in how the data will be used.

Where these scorecards go wrong

Treating logins as adoption. The most frequent and most damaging error. A login count rewards tools that sit behind a portal students pass through anyway and punishes tools used deeply by a smaller group. Always pair reach with a depth metric, and if a product cannot supply any depth signal, say so on the scorecard rather than letting the reach number stand in for the whole picture.

Letting the scorecard become an evaluation instrument by accident. The moment a building leader believes tier placement affects their evaluation, behavior changes — mass logins during a reporting window, provisioning inflation, pressure on teachers to open a tool they do not use. Guard against it structurally: state the scorecard's purpose in writing at the top of every view, keep it out of formal evaluation processes, and pair every low tier with a support conversation rather than a consequence. If the district cannot commit to that, expect the numbers to degrade within two cycles.

Ignoring the denominator drift. Enrollment shifts, staff turnover, and mid-year provisioning changes all move the denominator. If you compute rates against a September roster all year, a school that grew 12% looks like it declined. Recompute denominators from current rostering data on every refresh and store the denominator alongside the numerator in the fact table so historical rows stay reproducible.

Building for the cabinet and forgetting the principal. A district rollup is useful once a quarter to about six people. The school view is what determines whether the scorecard changes anything, and it needs to answer a building leader's actual questions: which of my products are dormant, which grade levels or departments are driving the gap, and what would move the number. If a principal cannot get to a useful answer within roughly two clicks, the scorecard will be opened during the rollout meeting and never again.

Silent pipeline failure. A vendor rotates an API key, changes a CSV column name, or moves an endpoint, and the job fails quietly. The scorecard keeps rendering last month's numbers with no visible difference. Every product needs a last-successful-refresh timestamp visible on the view, an alert when a pull is more than a set number of days stale, and a row-count sanity check that flags when a product's weekly volume moves by more than a large threshold against its trailing average. The failure that costs the most is not a wrong number — it is a stale number nobody knew was stale.

Over-engineering the first version. Districts routinely spend two quarters designing a comprehensive scorecard covering every product and never ship. Start with the five to eight highest-cost contracts, get the pipeline and the school view working end to end, publish it, and let real use tell you what the next fields should be. A narrow scorecard that gets opened beats a complete one that gets built next year.

Skipping the procurement loop. The scorecard's data availability is decided at contract signing, not at build time. Add usage-data access to your standard procurement requirements: school-level breakdowns, a programmatic export, a documented definition of active user, and retention of at least a full academic year of history. Districts that add these clauses find the scorecard gets easier every renewal cycle, and the ones that do not find themselves re-solving the same integration problem for every new product.

Publishing without a narrative. A tier grid with no interpretation invites everyone to read their own story into it. Each monthly publication should carry three or four sentences: what moved, what is being investigated, and what decision is coming. That short framing is the difference between a report people act on and a file people archive.

Related questions

How often should the scorecard be refreshed?

Pull data weekly so a broken job is caught within days, but publish to school and district audiences monthly. Weekly publication generates noise from calendar effects — testing windows, breaks, snow days — that leaders will misread as trends.

Should individual teachers appear on the scorecard?

No. Report at school, grade band, or department level. Teacher-level detail turns an adoption instrument into a surveillance one, which changes behavior and destroys the data's usefulness. Building leaders can drill into their own staff privately if the district's policy permits.

What if a vendor will not provide school-level data?

Record it as a coverage gap on the scorecard rather than leaving the cell blank. Use SSO logs as a reach proxy where possible, and add school-level reporting to the requirements for that contract's next renewal.

How do we handle products used by only one grade band?

Segment the denominator. A tool licensed for grades 3–5 should be measured against grade 3–5 rostering, not building-wide staff and student counts. Store the applicable population with each product so the rate is computed correctly everywhere.

Does the scorecard replace classroom observation?

No. It measures whether tools are opened and used, not whether they are used well. Pair adoption tiers with instructional walkthrough data when deciding whether a low tier reflects a product problem or an implementation one.

FAQ

What is the minimum data needed to start a district-wide adoption scorecard?

Three things: a canonical list of schools with stable codes, a list of contracted products with cost and seat counts, and at least one usage signal per product — vendor export, SSO log, or documented attestation. With those you can compute an active-to-provisioned rate and a cost-per-active-user figure, which is already enough to change a renewal conversation. Everything else is refinement.

How long does a first version typically take to build?

For a district with an existing data warehouse and BI tooling, a scoped first version covering the highest-cost products is usually a matter of weeks of part-time work, with most of that spent on identity crosswalks and definition agreement rather than on charts. Districts starting without a warehouse should expect longer and should consider a broker-based or spreadsheet pilot first.

Who should own the scorecard?

Ownership works best when it is shared between the technology department, which owns the pipelines and identity data, and the academic or curriculum office, which owns the definitions and the interpretation. A scorecard owned solely by technology tends to be technically sound and instructionally ignored; one owned solely by curriculum tends to break silently. Name both owners in writing.

Should the scorecard include free tools teachers adopt on their own?

Include them where you can see them through SSO or app-inventory data, but keep them in a separate section from contracted products. Unmanaged tools matter for privacy review and for spotting genuine demand, but mixing them into a spend-accountability view muddies the renewal decision the scorecard exists to support.

How do we prevent schools from gaming the numbers?

Structurally rather than punitively. Use depth and consistency metrics alongside reach so a burst of logins does not move a tier much, keep the scorecard out of formal evaluation, and monitor for anomalous patterns like a school's activity spiking sharply in the days before a publication date. Gaming is usually a symptom that the scorecard is being used for something it was not designed for.

What should happen when a school lands in the lowest tier?

A conversation, not a consequence. Dormant usage has several very different causes — the product does not fit the school's instructional model, provisioning failed, training never happened, or a competing tool covers the same need. The scorecard's job is to trigger the diagnosis, and the district's response should be to determine which cause applies before deciding whether to support the school differently or drop the product.

Sources

• https://www.ed.gov/
• https://nces.ed.gov/
• https://www.cosn.org/
• https://www.iste.org/
• https://www.classlink.com/
• https://www.clever.com/
• https://www.imsglobal.org/
• https://studentprivacycompass.org/
• https://developers.google.com/workspace/admin/reports/reference/rest
• https://learn.microsoft.com/en-us/entra/identity/monitoring-health/

Related on PULSE

• How to negotiate edtech renewals using usage data
• Building a district data warehouse for instructional reporting
• Rostering and identity management across school systems
• What "active user" means across software vendor contracts
• Cost-per-active-user as a software renewal metric