Auto Refresh for Knowledge Base Trained Links in Sparkforge

Auto Refresh schedules automatic re-training for the web URLs in your Sparkforge Knowledge Base. Instead of clicking Refresh on each link, Sparkforge re-crawls the pages on the cadence you choose so your Conversation AI bot keeps answering with current content.

What It Does

It periodically re-crawls your Knowledge Base URLs, updates the table data, and re-indexes the content the bot relies on. Product pages, help docs, and blog articles stay in sync without ongoing manual work.

Choose a Refresh Frequency

Click the Auto Refresh icon (circular arrows) and pick:

• Daily — re-crawls roughly every 24 hours.
• Weekly — refreshes on the same weekday each week.
• Monthly — refreshes on the same calendar day each month.

Monitor Status and Logs

The Status column shows the current state of each URL:

• Getting Data — Sparkforge is actively re-crawling.
• Trained — the latest data is saved. Check Last data refreshed at for the timestamp.
• Failed — the crawl hit an error. Retry or check page accessibility.

Set Up Auto Refresh

1. From the sub-account, go to AI Agents → Knowledge Base.
2. Open the knowledge base you want to configure and click Web Crawler.
3. Click the Auto Refresh icon and pick Daily, Weekly, or Monthly. Click Save.

Best Practices

• Use stable URLs — avoid pages that frequently redirect or require login.
• Keep pages lightweight — long or script-heavy pages can crawl slowly and time out.
• Audit failures weekly — investigate any rows stuck in Failed and fix broken links.
• Stagger heavy sites — for large libraries, prefer Weekly or Monthly to spread load.
• Prioritize critical pages — set Daily for high-change pages that affect customer answers directly.