Dev builds 12-tool MCP server in 4 hours, shares hard lessons
Perufitlife bundled 12 Apify web scrapers into a single MCP server in about 4 hours, documenting critical lessons on dual-transport support, silent scraper failures, Cloudflare evasion, and a BYOK billing pattern.
Score breakdown
Developers building MCP servers need to validate both SSE and Streamable HTTP transports from day one and add explicit zero-result guards to scrapers — skipping either step risks silently broken tools that pass local tests but fail in real agent clients.
- 01Built 12 Apify web scrapers into one MCP server in approximately 4 hours.
- 02Modern MCP clients (Claude Desktop, Cursor, MCP Inspector) require Streamable HTTP (`POST /mcp`); SSE alone is insufficient in 2026.
- 03The fix mounts both SSE and Streamable HTTP transports on the same server using session ID management via a `Map`.
Perufitlife describes bundling 12 web scrapers from Apify Store into a single MCP server in roughly 4 hours, motivated by the observation that MCP-exposed tools are callable directly from inside Claude, ChatGPT, Cursor, and other agent loops — shifting the "buyer" from a human developer browsing a marketplace to an LLM selecting tools from a registry based on description quality and output format.
First, implementing only the legacy SSE transport (`GET /sse` + `POST /messages`) passed `curl` tests but failed end-to-end with Claude Desktop, which expects the modern Streamable HTTP transport (`POST /mcp`).
The post details several concrete failure modes encountered. First, implementing only the legacy SSE transport (`GET /sse` + `POST /messages`) passed `curl` tests but failed end-to-end with Claude Desktop, which expects the modern Streamable HTTP transport (`POST /mcp`). The fix mounts both transports on the same Express server, using a `Map` to manage sessions via `mcp-session-id` headers and the `StreamableHTTPServerTransport` from `@modelcontextprotocol/sdk`. Second, eBay's search results DOM silently migrated from `.s-item` to `.s-card` selectors, causing scrapers to return 0 items without throwing errors for weeks. The recommended fix is an explicit guard — throwing an error when 0 items are extracted — combined with Apify's `Actor.fail()` on smoke runs at build time.
On the anti-bot front, Yelp's Cloudflare Turnstile defeated every stealth technique tried (Camoufox, residential proxies, mouse jitter, logged-in cookies), leading to a pivot to the Yelp Fusion API's free tier of 5,000 calls per day using a BYOK (Bring Your Own Key) pattern where users supply their own API key as a tool parameter. For cases requiring a real browser, Camoufox running in containers below 4GB silently OOM-crashed, with Playwright reporting the page as perpetually loading; bumping memory to 4GB resolved it, as the browser process alone consumes 1.5–2GB at steady state. The billing architecture uses Apify's Standby mode so users authenticate with their own Apify token, meaning the author pays nothing for compute or distribution while any MCP client gains access to all 12 scrapers instantly via Smithery.ai as a registry gateway.
Key facts
- 01Built 12 Apify web scrapers into one MCP server in approximately 4 hours.
- 02Modern MCP clients (Claude Desktop, Cursor, MCP Inspector) require Streamable HTTP (`POST /mcp`); SSE alone is insufficient in 2026.
- 03The fix mounts both SSE and Streamable HTTP transports on the same server using session ID management via a `Map`.
- 04eBay's DOM silently changed from `.s-item` to `.s-card` selectors, causing scrapers to return 0 items undetected for weeks.