Illustrative program assessment
/html
Regional AI Capacity Cohort, Round 1
Renew, with two changes.
Six of eight organizations still use what the program gave them, and in five of those six, one person is holding it together, mostly on unfunded time. Round 2 should pay for that person and stop paying for the parts nobody uses.
The six questions
Each finding shows how confident it is and what it rests on.
| Question | Finding | Confidence | Basis |
|---|---|---|---|
| What is still in ordinary use, and by whom? | 3 of 8 use it weekly, 3 use parts of it, 2 stopped by spring. | High | Observed at 6 sites |
| What does it cost to keep, in money and in staff time? | Median 7 staff hours a month per site. About $1,200 a year in software once the subsidy ends. | Medium | Told, checked against invoices at 4 sites |
| Who is carrying it, and would it continue without them? | One person at 5 of the 6 active sites, at 4 of them as an unfunded addition to their job. 4 of those 5 said it would stop if they left. | High | Told, confirmed by managers |
| Did the people left responsible gain any authority to change it? | 2 of 6 carriers can change the workflow or move budget. The rest need sign-off for any change. | High | Told by carriers and managers |
| What was dropped, and was dropping it a sound decision? | The second tool was dropped at 7 of 8 sites. Staff found a simpler way to do the same task. Sound. | High | Observed |
| What would a next round need to pay for that this one did not? | The owner's time, and a short setup sprint for new sites. Not more tools. | Medium | Inferred from the above |
Patterns across the eight organizations
Reported in aggregate. Detail about any one organization stays with that organization unless it agrees to share it.
Use
Who carries it (6 active sites)
- One person carries it
- 5 of 6
- Their time is funded
- 2 of 6
- They can change the workflow or budget
- 2 of 6
Cost to keep (6 active sites)
- Median staff time
- 7 hrs / month
- Range
- 4–12 hrs / month
- Software after subsidy
- ~$1,200 / year
Why sites stopped (2)
- The person carrying it left
- 1
- Poor fit with existing records
- 1
- Would restart with paid staff time
- 1 of 2
What a next round would need to pay for
What the findings point to. Not yet a budget.
Active sites rely on 4 to 12 hours a month of one person's time, mostly unfunded.
Owners described solving problems alone once program support stopped.
One site stopped because the tool could not work with its existing records.
Turning these into budget lines, selection criteria and exit conditions is the Assessment and Next-Round Design engagement.
What to stop
With the reason you can give the board.
Used at one site of eight. Staff replaced it with a simpler method.
No grantee or program staff member reported reading it. It cost each site about 2 hours a month to fill in.
Questions for the docket
- Will Round 2 fund the owner's time directly, or require grantees to match it?
- Can a site leave the program mid-round without losing the rest of its support?
- Who decides when a site may change the tool or the workflow: the grantee, the intermediary, or the foundation?
Method
- Program files reviewed before interviews
- 19 interviews: 8 leads, 7 day-to-day users, 4 budget managers
- 11 sessions watching the work in use
- Findings marked as observed, told or inferred
Limits
- No comparison group; this is not an impact evaluation
- Client outcomes were not measured
- The two sites that stopped were interviewed by phone only
- Cost figures at 4 sites rest on staff estimates
What each organization receives
Not shared with the commissionerEvery participating organization receives its own one-page record. It belongs to them, and the commissioner does not see it unless the organization chooses to share it. One is shown here only to illustrate the format.
"It works because I do it on Thursdays. If I left, it would stop the next week."