Case study · shipped ops product · 2026-08-04 lift

ManyChat affiliate partner research & scoring

A human approval workspace on top of an n8n research agent that scores PartnerStack applicants with an explicit four-factor rubric and writes dossiers back for ops review.

Live interface Scoring rubric n8n workflow Prompt anatomy Applications list Provenance

The problem

Partner applications arrived faster than humans could research them. Ops needed a gate that was faster than manual Google tabs, but not a black-box auto-approve — every AI dossier still lands in a review queue with a decision enum.

Pipeline

01 IntakePartnerStack applications land in Mongo / CSV exports

02 Researchn8n “3. Research [final]” agent + Firecrawl / Bright Data tools

03 Score0–100 rubric: Reach 40 · Fit 35 · Engagement 15 · Identity 10

04 DecideAPPROVED_EXCELLENT / MONITOR / MANUAL_REVIEW_*

05 Ops gateops.html — search, filter, open dossier, human decides

Live interface (lifted)

Real ops.html + ops.js from aleksei-manychat-gh, pointed at anonymised local fixtures. Layout, filters, and scoring display are unchanged.

app/ops.html · Partner approvals workspace Open full screen

app/applications.html · Applications list (53 pending shape / 2,671 intake rows) Open full screen

Scoring rubric — actual weights

Production research agent (manychat/prompts/V3.2-workflows/Node_1_Research_Agent.md) scores out of 100:

Factor	Weight	Source
Audience Reach	40 / 100	Node_1_Research_Agent.md scoring framework
Profile Fit	35 / 100	Node_1_Research_Agent.md — independent of audience size
Engagement Quality	15 / 100	Node_1_Research_Agent.md
Identity Verification	10 / 100	Node_1_Research_Agent.md

Decision thresholds from the same prompt:

Outcome	When	Notes
APPROVED_EXCELLENT	score ≥ 80 and fraud < 0.35	Approve + dual-language welcome
APPROVED_MONITOR	score 60–79 (or quality override), fraud < 0.45	Approve + monitoring
MANUAL_REVIEW_QUESTIONS	missing proof / fraud 0.35–0.6	Leave pending + ≤2 questions
MANUAL_REVIEW_DECLINE	fraud ≥ 0.6 or identity failure	No direct DECLINE from agent

A separate pilot schema also exists at manychat/scoring/schema.json (historical / engagement blend). Shown for completeness — not substituted for the production Node_1 table above:

schema.json key	Weight	Source
historical	0.4	manychat/scoring/schema.json (pilot/historical layer)
profileFit	0.35	manychat/scoring/schema.json (pilot/historical layer)
engagement	0.15	manychat/scoring/schema.json (pilot/historical layer)
tenureAndTags	0.1	manychat/scoring/schema.json (pilot/historical layer)

Ops UI decision chips collapse research outcomes into APPROVE / MANUAL_REVIEW / DECLINE for the human gate — that mapping is in the unified dataset’s summary.decision, not invented here.

n8n research workflow (artifact)

Rendered from the exported workflow JSON — nodes, types, and connections only. No screenshot, no invented studio UI, no execution payloads.

When Executed by Another WorexecuteWorkflowTrigger

Structured Output Parser1outputParserStructured

OpenRouter Chat ModellmChatOpenRouter

#agent

BrightData - Custom API callbrightDataTool

OpenRouter Chat Model4lmChatOpenRouter

Loop Over Items1splitInBatches

Json parser1agent

Get applicants from Mongo1mongoDb

Set new status1set

Update status in Mongo1mongoDb

Set result data1set

Update result in Mongo1mongoDb

MCP Firecrawl1mcpClientTool

Access and extract data frombrightDataTool

Extract structured data frombrightDataTool

List available datasets in BbrightDataTool

Call next workflow1executeWorkflow

Aggregate1aggregate

Webhook1webhook

Source: manychat-dashboard/docs/n8n/3. Research [final]/workflow-configs/3. Research [final] (2).json. Structure also saved at data/n8n_research_workflow.json.

Get applicants → Loop — batch pull and per-applicant iteration.
# agent + Firecrawl/Bright Data tools — research loop with tool budget.
Json parser1 — coerce output into the dossier schema.
Update result in Mongo → Aggregate → Call next workflow — persist and hand off.

What it produced (counted volumes)

91unified applicants in ops workspacemeta.totalApplicants · applicants_unified.json

46matched research (application + dossier)meta.matchedResearch

38research-only dossiersmeta.researchOnly

53pending PartnerStack queuemeta.totalPartnerstack

2,671applications export rowsapplications.json meta.total

85research dossier files on disk (source tree)len(data/research) in source; 83–84 attached in unified lift

228top partners analytics JSON (not mounted — needs Mongo/CSV)manychat/dashboard/top_228_partners.json

200top partners by revenue itemstop_partners_by_revenue.json

These denominators are not merged. 91 ≠ 2,671 ≠ 228.

What was hard / what it got wrong

Bright Data endpoints were flaky in production prompts — the agent prompt itself documents retries and Firecrawl-first fallbacks.
Direct DECLINE was forbidden at the agent layer; declines became MANUAL_REVIEW_DECLINE for a human (named in the original prompt — redacted here).
Streamlit / Go+React analytics exist in-repo but are not shown as interactive here: they need Mongo/OAuth/secrets. Claiming they “ran” in this portfolio session would be false.
Applications export is flat CSV-shaped; the table renderer expects a richer row model — the lift reshapes anonymised rows to that schema and says so in provenance.

What is real

ops.html / ops.js / applications.html / applications.js / applications.css; scoring weights from Node_1 + schema.json; n8n node graph from the exported workflow JSON; volume counts from meta fields; decision/score distributions preserved from the unified dataset.

What is reconstructed / transformed

All names, emails, employers, URLs, handles, and dossier prose are anonymised. Applications rows were reshaped to the UI schema the table already expected. Partner analytics Streamlit UI is omitted (not runnable without secrets). Prompt anatomy page is the real production prompt with secrets/PII scrubbed and section annotations added.

Also read: Anatomy of a production prompt · PROVENANCE.md