AI Agents
WAT Agent — Job-Market Scraper
An agentic workflow built on my WAT framework (Workflows, Agents, Tools): Claude follows a plain-language SOP and runs Firecrawl-powered Python tools to turn 6,200+ Naukri.com listings into a clean Excel dataset.
Challenge
I wanted a dependable picture of the job market for operations and accounting roles in India: every listing, with full job details, in one spreadsheet. Asking an AI to “just scrape it” fails at this scale, because small error rates compound over thousands of steps.
Process
I designed the WAT framework to separate reasoning from execution:
- Workflows. Markdown SOPs, written the way you’d brief a team member: objective, inputs, tools, outputs, edge cases.
- Agent. Claude reads the workflow, runs the tools in order, handles failures, and updates the SOP with what it learned (rate limits, timing quirks).
- Tools. Deterministic Python scripts that do the actual work: scrape listing pages, fetch each job’s details via Firecrawl, and export to Excel.
The run has three phases (listings, then details, then export), with checkpoint files after every page and batch. A multi-hour run can stop and resume without losing progress.
Outcome
6,200+ job listings with full details, exported to a formatted Excel file. The framework itself is reusable: new jobs need only a new workflow and a few small tools.
This is how I build with AI: the AI makes decisions and plain code does the execution, which keeps long runs reliable.
Cover: illustrative diagram of the WAT architecture.