Weekly Updated Menu Scraper

I’m building an online directory similar to menuswithprice.com and need an automated pipeline that gathers data from roughly 6–10 external sites. The directory must hold three kinds of records at launch—restaurant menus, product prices, and service listings—and refresh the full data set on a weekly schedule.

Here’s the flow I have in mind: scraper scripts (Python with Scrapy, BeautifulSoup, or a comparable framework) pull structured information from each source, normalise it, and load it into a database that powers the public-facing site. A cron job or cloud function should trigger the weekly update, logging what changed so I can spot anomalies quickly. Duplicate detection, polite rate-limiting, and respect for robots.txt are essential.

Deliverables
• Clean, well-commented scraping code for every target site
• Normalisation routine that maps each source’s fields to a unified schema
• Database (MySQL, Postgres, or NoSQL—recommend what fits best) seeded with the first full import
• Automated weekly update job with simple reporting (e-mail or dashboard)
• Brief setup guide so I can deploy or extend the scraper myself

Acceptance criteria
The directory pages must load complete, de-duplicated records for all three content types; a manual run of the job must finish without fatal errors; and the first scheduled weekly run should execute automatically and push its summary report.

If you’ve built data-driven sites or price-tracking tools before, this should feel familiar. Please outline your proposed stack, how you’ll monitor for site-layout changes, and any questions you have about the sources.