Knowledge sources
Web pages
Add public web pages to a Web Crawler connection so agents answer from their content, refreshed on your sync interval.
A Web Crawler connection fetches public web pages and indexes their content, so agents can answer from your website, help center or product pages.
Add pages
- 1Go to Studio → Connections, click Create Connection and choose Web Crawler.
- 2Enter a Connection Name, for example "Company Website", and a Description.
- 3Under Website URLs, enter the first address. Click Add URL for each additional page.
- 4Click Create Connection. Harmona starts fetching and indexing right away.
What gets indexed
- Each URL you add is fetched as one page. Links on that page are not followed, so add every page you want indexed.
- Harmona keeps the main content of the page as text for search.
- A URL that points straight to a document, such as a PDF or DOCX file, is downloaded as a file and processed like an uploaded document.
Warning
Only add pages that are publicly accessible, and only from sites you own or have permission to index.
Keep pages fresh
Harmona fetches all URLs again on your organization's sync interval, once a day by default. To refresh now, open the connection and click Sync Connection. Do this after you publish an important change.
Keep URLs stable. If a page moves, edit the connection with Edit Connection, replace the old address and sync again.
When to use something else
- Docs hosted on ReadMe.com: use a ReadMe Docs source. It syncs every guide, API reference page and changelog post. See Documentation connections.
- Files you already have: upload them to an Internal Files connection. See Files and images.
Updated 2026-09-24