Program evaluation

Goals from the program's own documents, each effect estimated against places it did not reach, revenue against projection, every goal rated. Every figure traced to its source

Customize this Agent

DOCXXLSX16 steps

The problem

Goal met on paper: 11 percent fewer vehicles against a 10 percent target

The MTA's 73,000 fewer vehicles a day, 11 percent, sets zone entries against traffic before the toll. That count also carries everything else that moved traffic since January 2025: return to office, construction, weather, the economy. Grade the 10 percent objective on it and the program passes or fails on the wrong number

The request

Evaluate New York's congestion pricing at twenty months, January 2025 through August 2026, against the objectives in its environmental assessment: at least 10 percent fewer vehicles entering the zone daily and 5 percent fewer vehicle miles inside it, and net revenue to fund $15 billion of transit capital. Add speeds, transit ridership, safety, air quality and traffic diverted around the zone. Estimate each effect against comparison areas and corridors the toll could not reach, with 2022 to 2024 as the before period, never from a before and after count. Take air quality from the health department's August study. Rate each goal met, partly met or not met, with recommendations. Report and workbook, every figure sourced.

How Axe works

ApprovedChanges asked

Your reviewApprove, or say what to change

Approved or changes

Report and workbookReport, workbook and fact check

Final reviewApprove the revised files

Revised report and workbookRevised files and fact check

Step

Build the comparison groups

Prompt

Sent to the model at this step

The zone has no twin, so choose each outcome's comparison before estimating anything and justify every unit in it. Candidates for traffic and speeds: crossings that carry no trips bound for or through the zone, and DOT count locations and speed segments in the business districts of the other boroughs and in Upper Manhattan away from the zone's approaches. Never use a road a driver could switch to in order to avoid the toll: diverted trips raise its counts and inflate the effect. The FDR Drive, the West Side Highway, the approaches to the zone, the crossings that offer a way around it and the expressways the health department monitored form the spillover set, measured as an outcome in its own right. Test Manhattan just north of 60th Street the same way, since drivers can park there and finish by transit. For ridership and safety, take stations, routes and police precincts outside the zone on the same test. Then set each candidate against the zone over 2022 to 2024, monthly trends side by side and an event study on the before months, and reject any group whose trend departs from the zone's. Where no single group passes, weight the candidates into a composite that tracks the zone over the before months, and report the weights and the fit. Record each group, why it was chosen, what was rejected and why, and the pre trend results.

Uses

Design matrix
Monthly series
Published evidence

Produces

Comparison groups

What you get back

DOCXEvaluation reportGoal ratings first, then design, findings by goal, revenue, spillover, limits and recommendations

XLSXAnalysis workbookSeries from 2022, comparison groups, pre trend tests, effect estimates, revenue, ratings, sources

Fact checkOpen items first, then each figure and claim marked checked or corrected

Make it yours

Point it at another program, set your own goals and rating thresholds, or change the comparison groups and the period. Every step and every prompt can be edited

Customize this Agent

Opens a private copy of this Agent in your workspace, ready to adapt to your own sources

More agent examples

Sovereign debt economistsDebt sustainability analysisDOCXXLSX16 steps

Policy analystsPolicy briefDOCXXLSX12 steps

Capital markets analystsBroker opinion of valuePPTXXLSX16 steps

Financial planning analystsBudget variance analysisXLSXDOCX13 steps

Credit research analystsCapital structure analysisXLSX12 steps