Monitoring Sources Over Time: Alerts, Change Tracking, and Archives

BY DR. LELAND W. SMITH

A lot of investigative value comes not from what a source says today but from how it changes. A company quietly removes a director from its team page. A product listing changes its country of origin. A forum thread is deleted the day after a news story. A social account changes its name. If you only ever look once, you miss all of it.

Monitoring is the practice of watching defined sources over time, catching meaningful changes, and preserving evidence of what was there before. Done well, it is cheap, quiet and steady. Done badly, it buries you in noise, misses the one change that mattered, and may breach a site's terms or the law. This guide covers the main monitoring methods, how to keep a monitoring log, how to manage alert fatigue, and the legal limits on automated collection.

Decide what you are watching for

Monitoring without a question is just collecting. Before you set up a single alert, write down what changes would matter and why. That usually comes straight from your intelligence requirements.

• New mentions: a name, company, product or address appearing somewhere new.
• Content changes: edits to specific pages, such as leadership lists, terms of service, pricing or contact details.
• Removals: pages, posts or documents taken down.
• New filings or records: registry updates, court dockets, regulatory notices or trademark applications.
• Status changes: a domain's DNS, a website going offline, or an account becoming private.

For each item, note how quickly you need to know. Some changes need same-day attention. Most can wait for a weekly review.

Search alerts

Search alerts notify you when a search engine indexes new results for a query. Google Alerts is the best-known free example: you enter a query, choose how often to receive results and which sources to include, and receive an email or RSS feed when new matches appear.

Alerts are easy to set up and useful as a broad net, but they have real limits:

• They only report what the search engine indexes, and indexing is incomplete and delayed.
• They tend to miss content behind logins, in apps, in many social platforms and in documents that are not crawled.
• They are not a reliable record that something did not appear. Silence is not evidence of absence.

Write precise queries. Use quotation marks for exact names, add a distinguishing term to separate namesakes, and exclude obvious noise with the minus operator. A query such as "Northfield Supplies" -football is better than Northfield. Review and adjust queries after the first week, because the first version almost always needs tuning.

Many news databases, registries and court systems offer their own alert or saved-search features. Where they exist, they are usually more reliable for their own content than a general search alert.

RSS and feeds

RSS is an old technology that remains one of the best monitoring tools. Many news sites, blogs, government agencies, forums and podcasts still publish feeds, and some platforms generate them for search results or user pages. A feed reader such as Feedly, Inoreader or a self-hosted alternative collects them into one place.

• Organize feeds into folders by case or topic so you can review them quickly.
• Use reader rules or filters, where available, to highlight entries that match key terms.
• Check that a feed is still updating. Feeds break silently when sites redesign.
• Remember that a feed shows what the publisher chooses to include, which may be a summary rather than full text.

Page-change monitoring

Page-change monitoring tools fetch a web page on a schedule, compare it with the previous version, and notify you when something changes. There are hosted services and open-source tools you can run yourself. Most let you choose the check frequency, restrict monitoring to part of the page, and view a side-by-side or highlighted comparison.

Some practical guidance:

• Target the element that matters. Whole-page monitoring triggers on rotating adverts, timestamps and cookie banners. Restrict the check to the relevant section, such as the team list or the price table.
• Choose a sensible frequency. Daily is enough for most corporate pages. Checking every few minutes is rarely justified and puts unnecessary load on the site.
• Keep the history. A change notification is only useful if you can see the before and after. Make sure the tool stores versions, or pair it with an archive capture.
• Expect dynamic pages to be harder. Pages that load content with JavaScript may need a tool that renders them, and may still produce false changes.

Scheduled archive captures

Monitoring tells you something changed. Archiving proves what was there. The two work best together.

The Internet Archive's Wayback Machine lets anyone request a capture of a public page through its Save Page Now feature, and the capture receives a timestamped, publicly accessible URL. That makes it useful as independent corroboration, because the record is held by a third party rather than only on your machine. Archive.today is another public archive that some investigators use alongside it. Neither captures everything: some sites block archiving, dynamic content may not render, and logged-in content is not captured.

For evidential work, keep your own captures as well:

• Save the page as a full-page screenshot and, where possible, as a web archive file or PDF with the URL and time visible.
• Record the date, time and time zone, the URL, and the tool used.
• Compute a cryptographic hash, such as SHA-256, of each saved file and record it in your log, so you can later show the file has not changed.
• Store originals read-only and work from copies.

Schedule captures for pages that matter most, for example weekly for a subject's corporate site, and immediately whenever a change alert fires.

Keeping a monitoring log

A monitoring log is the record that turns scattered alerts into usable intelligence. It also shows a client, a lawyer or a court what you watched and when. A spreadsheet is fine. Useful columns include:

• Monitor ID and case reference.
• Source: URL, feed, query or registry.
• Method: alert, feed, page-change tool or manual check.
• Frequency and date started.
• Requirement it supports.
• Date and time of each change detected.
• Description of the change, in neutral language.
• Capture reference and file hash.
• Assessment: relevant, not relevant or needs review, and why.
• Date stopped, and reason.

Also log checks that found no change, at least at the review level. "Weekly review, 14 sources, no relevant changes" is a useful entry. It shows the monitoring was running.

Alert fatigue

Alert fatigue is what happens when you receive so many notifications that you stop reading them carefully. It is the most common way monitoring fails in practice. The important alert arrives in a pile of irrelevant ones and gets archived unread.

Ways to keep it under control:

• Start narrow. Add monitors when a requirement justifies them, not because a source might be interesting.
• Tune in the first week. Review every alert from a new monitor and adjust the query or page selection to cut false positives.
• Route by priority. Send urgent monitors to a channel you check often, and low-priority ones to a digest you review weekly.
• Batch reviews. Set a fixed time to review feeds and digests rather than reacting to every notification.
• Retire monitors. Set an end date or review date for every monitor. Stop the ones that never produce anything useful.
• Measure. Note in the log how many alerts each monitor produced and how many were relevant. A monitor with a very low hit rate needs fixing or removing.

A worked example

Suppose a client is in a commercial dispute with a former distributor and asks you to watch for signs that the distributor is selling the client's products through a new company. The engagement runs for three months.

1. Define what matters. New companies registered by the distributor's known directors; changes to the distributor's website product pages; new online listings using the client's product names.
2. Set up monitors. You save searches for each director's name in the relevant company registry's alerting service, where available, and otherwise schedule a manual weekly registry check. You set a page-change monitor on the distributor's product page, restricted to the product list, checking daily. You create search alerts for the client's three product names with the distributor's town and with marketplace terms.
3. Archive a baseline. On day one you capture the distributor's site in the Wayback Machine and locally, and record hashes.
4. Tune. In week one the search alerts generate dozens of hits from legitimate retailers. You add exclusions for the client's authorized sellers and switch the alerts to a weekly digest.
5. Detect a change. In week five the page monitor reports that four of the client's products have been removed from the distributor's page. You capture the page immediately. The same week, the registry check shows a new company registered with one of the distributor's directors.
6. Report. You log both changes, note that the timing is consistent with a move to a new entity, and set new monitors on the new company's name. You do not conclude wrongdoing. You give the client's lawyer a dated, hashed record of exactly what changed and when.

Legal and terms-of-service considerations

Automated collection raises issues that a one-off manual visit does not. Take them seriously and get advice when in doubt.

• Terms of service. Many websites prohibit automated access or scraping in their terms. Breaching terms can expose you or your client to contract claims and account bans, and can undermine the credibility of your evidence.
• Access controls. Never automate access to content behind a login you are not authorized to use, and never circumvent technical barriers such as rate limits, CAPTCHAs or blocks. Depending on the jurisdiction, doing so may engage computer-misuse laws.
• robots.txt and load. Respect a site's robots directives where your tool uses them, keep request frequency low, and identify your tool honestly where possible. Monitoring should never degrade a site's service.
• Personal data. Ongoing monitoring of an individual is more intrusive than a single check. Data-protection rules such as the GDPR require a lawful basis, minimization and defined retention. Monitor the specific sources your requirement needs, not a person's whole online life.
• Harassment and stalking laws. Persistent monitoring of a private individual without a legitimate purpose can cross into conduct that harassment and stalking laws prohibit. Work only for clients with a legitimate, documented purpose, and decline requests that look like personal surveillance.
• Retention. Agree how long monitoring data will be kept and delete it when the engagement ends, unless legal hold or regulation requires otherwise.

Limits and common mistakes

• Trusting silence. No alerts can mean no change, or a broken monitor. Spot-check monitors periodically.
• Alerting without archiving. A notification that a page changed is useless if nobody captured the earlier version.
• Monitoring everything. More monitors do not mean better coverage. They usually mean worse attention.
• No end date. Monitors left running after an engagement ends create privacy risk and noise.
• Over-interpreting changes. Websites change for many ordinary reasons. Treat a change as a prompt for analysis, not a conclusion.

Checklist

• Tie every monitor to a written intelligence requirement.
• Write precise search alert queries and tune them in the first week.
• Use RSS feeds where available and check they still update.
• Restrict page-change monitors to the relevant section and a modest frequency.
• Archive a baseline before monitoring starts, and capture again whenever a change fires.
• Record URLs, timestamps, tools and SHA-256 hashes for captures.
• Keep a monitoring log, including reviews that found nothing.
• Route alerts by priority and batch low-priority reviews.
• Respect terms of service, access controls and site load; never bypass technical barriers.
• Set an end date for every monitor and delete data according to the agreed retention.

An HTTP date is a monitoring clue, not a change finding

A page monitor may expose a Last-Modified response header. This is the origin server's stated modification time; it is not an independent record of which words changed. MDN describes it as a conditional-request validator and notes that an ETag is more accurate for identifying whether file contents match.

Keep the HTTP value separate from your own access time and comparison result. For a fictional public policy page at example.com/policy, a practical review entry could use these fields:

• Observed at: the UTC time you retrieved the page.
• Server metadata: Last-Modified and ETag, if supplied; retain the exact values.
• Comparison: the specific passage that differs from your saved baseline, or “no relevant difference found.”
• Evidence: links to both captures and their recorded hashes.
• Assessment: what the passage supports, what remains uncertain, and whether the intelligence requirement needs a follow-up.

If the header changes but the relevant passage does not, record that distinction. If the passage changes but the header does not, preserve the passage comparison rather than discarding the observation. This keeps an automated alert separate from the analyst's finding.

Technical source: MDN: Last-Modified response header. For the wider record, use the practical evidence-log guide.