Site Pulse — Website & Domain Health Monitor

The Challenge

Websites with many localized pages and multiple domain extensions need reliable uptime monitoring — both for desktop and mobile visitors. Manual checks are impractical when the number of URLs grows dynamically from the sitemap. Additionally, domain registrations across multiple TLDs require proactive expiration tracking to prevent service disruptions from missed renewals or unexpected nameserver changes.

The Solution

I built an automated monitoring system with three core components:

• Dynamic URL discovery — parses sitemap.xml with automatic XML namespace detection, so monitoring always reflects the current site structure
• Dual-mode health checks — validates every URL in both desktop and mobile modes using realistic User-Agent headers, with retry logic and response size thresholds
• Domain watchdog — monitors WHOIS records for expiration dates and nameserver changes across all TLDs, alerting 90 days before expiry
• Telegram alerts — instant notifications with automatic message chunking and granular controls (success/warning/error toggles)
• Log analyzer — post-check analysis that groups pages by base path and surfaces the slowest, most unstable, and largest responses

Concurrent Page Health Checks

Parallel URL validation using ThreadPoolExecutor with session reuse and conditional Telegram notifications:

# website_monitor.py
def check_pages() -> None:
    """Check all websites from the list"""
    timestamp = datetime.now().strftime("%d.%m.%Y %H:%M:%S")
    messages_to_send = []

    with requests.Session() as session:
        def check_with_session(url):
            return check_single_page(url, session)

        with ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
            future_to_url = {
                executor.submit(check_with_session, url): url
                for url in URLS_TO_CHECK
            }

            for future in concurrent.futures.as_completed(future_to_url):
                message, is_success, is_warning, is_error = future.result()

                if (is_success and NOTIFY_SUCCESS) or \
                   (is_warning and NOTIFY_WARNING) or \
                   (is_error and NOTIFY_ERROR):
                    messages_to_send.append(message)

    if messages_to_send:
        message = f"📅 {timestamp}\n\n" + "\n\n".join(messages_to_send)
        send_telegram_message(message)

Result: 10 parallel workers with session reuse, configurable notification granularity, automatic message aggregation.

Desktop & Mobile Validation with Retry

Each URL is checked in both device modes with automatic retry on failure and response size validation:

# website_monitor.py
def check_single_page(url: str, session: requests.Session) -> tuple:
    def try_request(headers, device_type):
        start_time = time.time()
        response = session.get(
            url,
            timeout=(CONNECT_TIMEOUT, READ_TIMEOUT),
            headers=headers
        )
        content_length = len(response.content)
        elapsed_time = time.time() - start_time

        if response.status_code == 200 and content_length >= MIN_RESPONSE_SIZE:
            return response, content_length, elapsed_time, None
        else:
            if response.status_code != 200:
                raise Exception(f"Response code: {response.status_code}")
            else:
                raise Exception(f"Response size too small: {content_length} bytes")

    # Desktop check with retry
    try:
        try:
            response, content_length, elapsed_time, _ = try_request(
                DESKTOP_HEADERS, "Desktop"
            )
        except Exception:
            time.sleep(5)
            response, content_length, elapsed_time, _ = try_request(
                DESKTOP_HEADERS, "Desktop"
            )

Result: Catches both HTTP errors and suspiciously small responses (e.g. empty pages returning 200), with a 5-second retry delay.

Domain Expiration & Nameserver Monitoring

WHOIS-based domain check that validates expiration dates and verifies nameservers match expected configuration:

# website_monitor.py
def check_domain(domain: str) -> tuple[str, bool]:
    """Check domain registration expiry and nameservers"""
    w = whois.whois(domain)

    if isinstance(w.expiration_date, list):
        expiry_date = min(w.expiration_date)
    else:
        expiry_date = w.expiration_date

    days_until_expiry = (expiry_date - datetime.now()).days

    domain_ns = [ns.lower() for ns in w.name_servers]
    our_ns = [ns.lower() for ns in OUR_NAMESERVERS]
    using_our_ns = any(ns in domain_ns for ns in our_ns)

    messages = []
    has_warning = False

    if days_until_expiry <= EXPIRY_WARNING_DAYS:
        messages.append(f"⚠️ Domain will expire in {days_until_expiry} days")
        has_warning = True
    if not using_our_ns:
        messages.append(f"⚠️ Domain is using external nameservers")
        has_warning = True

Result: Handles WHOIS quirks (some registrars return lists instead of single dates), monitors both expiration and DNS configuration drift.

Results

Metric	Value
Codebase	~757 lines Python
Parallel workers	10 (ThreadPoolExecutor)
Device modes	Desktop + Mobile per URL
Domain monitoring	13 TLDs with WHOIS
Expiry warning	90 days before expiration
Scheduling	Cron-based, fully automated

The system runs unattended via cron, dynamically adapting to sitemap changes and delivering Telegram alerts within seconds of detecting an issue — whether it's a down page, a slow response, or an expiring domain.