Discourse exposes application errors and warnings at /logs through Logster, its Redis-backed exception reporter. This tutorial shows how to export the currently retained Logster records using a Discourse admin API key.

A complete export requires more than one request. The list endpoint is paginated and normally omits environment data, while grouped rows use a different cursor format. The exporter below handles those details, saves the raw API responses, fetches every available detail record, and reports whether its downloaded count matches Logster’s reported count.

The export is still subject to Logster’s retention and truncation settings. Those limits are covered after the instructions.

Scope: /logs and /admin/logs are different

Discourse has several unrelated logging surfaces:

The procedure below covers precisely the first item: the data represented at /logs.

Create a dedicated API key

In the Discourse admin interface:

  1. Open Admin → API Keys.
  2. Create a dedicated granular key.
  3. Add the logs:messages scope.
  4. Use system as the API username, or an administrator associated with the key.
  5. Revoke the key after the export.

Current Discourse exposes two relevant scoped endpoints:

POST /logs/messages.json
GET  /logs/show/:id.json

The endpoint is deliberately POST, despite being a read operation. Support for the API-key scope landed in Discourse PR #24956.

Set the connection details:

export DISCOURSE_URL='https://forum.example.com'
export DISCOURSE_API_KEY='paste-key-here'
export DISCOURSE_API_USERNAME='system'

Then verify access:

curl --fail-with-body -sS \
  -X POST "$DISCOURSE_URL/logs/messages.json" \
  -H "Api-Key: $DISCOURSE_API_KEY" \
  -H "Api-Username: $DISCOURSE_API_USERNAME" \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  --data '' |
  jq .

A successful response looks roughly like this:

{
  "messages": [],
  "total": 123,
  "search": "",
  "filter": ""
}

Fetch full message details

POST /logs/messages.json returns up to 50 rows. It does not normally load each message’s env, which is where Logster keeps request paths, parameters, hostnames, process IDs, application versions, timestamps, and other debugging context.

The complete record comes from:

GET /logs/show/<message-key>.json

So a faithful export has two phases:

  1. Walk the list with its before cursor and collect every exposed key.
  2. Fetch the detail endpoint for every key before it is evicted.

Grouped rows complicate both steps. A group has a row_id cursor and a messages array rather than an ordinary top-level message key. The exporter has to preserve the group row, collect its visible member keys, and paginate using row_id.

The exporter

Save this as export_logster.py:

#!/usr/bin/env python3

import json
import os
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import quote

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry


BASE_URL = os.environ["DISCOURSE_URL"].rstrip("/")
API_KEY = os.environ["DISCOURSE_API_KEY"]
API_USERNAME = os.environ.get("DISCOURSE_API_USERNAME", "system")

timestamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
output_dir = Path(os.environ.get("OUTPUT_DIR", f"logster-export-{timestamp}"))
pages_dir = output_dir / "pages"
messages_dir = output_dir / "messages"

pages_dir.mkdir(parents=True, exist_ok=False)
messages_dir.mkdir()
os.chmod(output_dir, 0o700)

session = requests.Session()
session.headers.update(
    {
        "Api-Key": API_KEY,
        "Api-Username": API_USERNAME,
        "Accept": "application/json",
        "User-Agent": "discourse-logster-export/1.0",
    }
)

retries = Retry(
    total=6,
    connect=6,
    read=6,
    status=6,
    backoff_factor=1,
    status_forcelist=(429, 500, 502, 503, 504),
    allowed_methods=frozenset(("GET", "POST")),
    respect_retry_after_header=True,
)
session.mount("https://", HTTPAdapter(max_retries=retries))
session.mount("http://", HTTPAdapter(max_retries=retries))


def save_json(path, value):
    temporary = path.with_suffix(path.suffix + ".tmp")
    temporary.write_text(
        json.dumps(value, ensure_ascii=False, indent=2) + "\n",
        encoding="utf-8",
    )
    os.chmod(temporary, 0o600)
    temporary.replace(path)


def list_page(before=None):
    form = {}
    if before:
        form["before"] = before

    response = session.post(
        f"{BASE_URL}/logs/messages.json",
        data=form,
        timeout=60,
    )
    response.raise_for_status()
    return response.json()


def get_detail(key):
    response = session.get(
        f"{BASE_URL}/logs/show/{quote(key, safe='')}.json",
        timeout=60,
    )

    if response.status_code == 404:
        # The record may have been evicted while the export was running.
        return None

    response.raise_for_status()
    return response.json()


def keys_from_row(row):
    """Return all ordinary message keys exposed by one list row."""
    keys = []

    key = row.get("key")
    if isinstance(key, str):
        keys.append(key)

    for message in row.get("messages") or []:
        if isinstance(message, dict):
            key = message.get("key")
            if isinstance(key, str):
                keys.append(key)

    # A grouped row's cursor is also the key of an underlying message.
    row_id = row.get("row_id")
    if isinstance(row_id, str):
        keys.append(row_id)

    return keys


def cursor_from_row(row):
    if row.get("group") and isinstance(row.get("row_id"), str):
        return row["row_id"]

    if isinstance(row.get("key"), str):
        return row["key"]

    return None


started_at = datetime.now(timezone.utc).isoformat()
before = None
previous_cursors = set()
seen_keys = set()
unavailable_keys = []
group_rows = []
page_number = 0
initial_total = None
last_reported_total = None

rows_file = (output_dir / "list-rows.jsonl").open("w", encoding="utf-8")

try:
    while True:
        page = list_page(before)

        if initial_total is None:
            initial_total = page.get("total")

        last_reported_total = page.get("total")
        rows = page.get("messages") or []

        save_json(pages_dir / f"{page_number:05d}.json", page)

        if not rows:
            break

        for row in rows:
            rows_file.write(json.dumps(row, ensure_ascii=False) + "\n")

            if row.get("group"):
                group_rows.append(row)

            for key in keys_from_row(row):
                if key in seen_keys:
                    continue

                seen_keys.add(key)
                detail = get_detail(key)

                if detail is None:
                    unavailable_keys.append(key)
                    continue

                save_json(messages_dir / f"{key}.json", detail)

        # Rows are oldest-to-newest within the returned window. The first
        # row is therefore the cursor for the next older page.
        next_before = cursor_from_row(rows[0])

        print(
            f"page={page_number} "
            f"rows={len(rows)} "
            f"keys={len(seen_keys)} "
            f"reported_total={last_reported_total}",
            file=sys.stderr,
        )

        if not next_before:
            print("Stopped: no usable pagination cursor", file=sys.stderr)
            break

        if next_before == before or next_before in previous_cursors:
            print(f"Stopped: repeated cursor {next_before}", file=sys.stderr)
            break

        previous_cursors.add(next_before)
        before = next_before
        page_number += 1

        # Be polite to the production site.
        time.sleep(0.05)

finally:
    rows_file.close()
    os.chmod(output_dir / "list-rows.jsonl", 0o600)

save_json(output_dir / "group-rows.json", group_rows)

summary = {
    "site": BASE_URL,
    "api_username": API_USERNAME,
    "started_at": started_at,
    "finished_at": datetime.now(timezone.utc).isoformat(),
    "pages_downloaded": page_number + 1,
    "initial_reported_total": initial_total,
    "last_reported_total": last_reported_total,
    "unique_keys_discovered": len(seen_keys),
    "details_downloaded": len(seen_keys) - len(unavailable_keys),
    "unavailable_keys": unavailable_keys,
    "group_rows_seen": len(group_rows),
    "stable_total_during_export": initial_total == last_reported_total,
    "count_matches_initial_total": (
        isinstance(initial_total, int)
        and len(seen_keys) - len(unavailable_keys) == initial_total
    ),
    "warning": (
        "A count mismatch means records changed during export or Logster "
        "grouping prevented the API from exposing every retained key."
    ),
}

save_json(output_dir / "summary.json", summary)

print(json.dumps(summary, indent=2), file=sys.stderr)
print(output_dir)

Run it with uv:

uv run --with requests export_logster.py

Or use a conventional virtual environment:

python3 -m venv .venv
. .venv/bin/activate
pip install requests
python export_logster.py

What it produces

logster-export-20260730T010203Z/
├── summary.json
├── list-rows.jsonl
├── group-rows.json
├── pages/
│   ├── 00000.json
│   ├── 00001.json
│   └── ...
└── messages/
    ├── <logster-key>.json
    └── ...

The raw pages are retained so the export can be audited later. list-rows.jsonl provides a convenient streaming index, group-rows.json preserves group metadata, and each file under messages/ contains the full detail response available for that key.

A typical detail record includes:

{
  "message": "Example failure",
  "progname": "app",
  "severity": 3,
  "timestamp": 1785370000000,
  "first_timestamp": 1785360000000,
  "key": "0123456789abcdef",
  "backtrace": ["..."],
  "count": 17,
  "protected": false,
  "env": [
    {
      "hostname": "web-only.example.internal",
      "process_id": 123,
      "application_version": "abcdef",
      "REQUEST_URI": "/example",
      "HTTP_USER_AGENT": "...",
      "params": {},
      "time": 1785370000000
    }
  ]
}

Deciding whether the export is complete

Inspect the summary:

jq . logster-export-*/summary.json

The clean result is something like:

{
  "initial_reported_total": 742,
  "last_reported_total": 742,
  "unique_keys_discovered": 742,
  "details_downloaded": 742,
  "unavailable_keys": [],
  "stable_total_during_export": true,
  "count_matches_initial_total": true
}

That establishes a strong practical result: the store’s reported count remained stable and every reported record produced a detail payload.

A mismatch means one of four things happened:

  1. New records arrived or old records were evicted during the export.
  2. A custom grouping pattern hid retained message keys from the list API.
  3. A record disappeared between the list request and its detail request.
  4. Protected records or concurrent administrative changes changed the available record set.

Run the export again during a period with little log activity. If the reported total remains stable and the mismatch persists, a grouping rule may be preventing the API from exposing every retained key.

Retention and completeness limits

Discourse configures Logster’s backlog with:

Logster.store.max_backlog = GlobalSetting.max_logster_logs

The default is:

# maximum number of log messages in /logs
max_logster_logs = 1000

Logster also limits retained detail within merged and custom-grouped records:

LimitDefault
Retained Logster message records1,000
Environments retained per merged message50
Members retained in a custom group100
Individual environment size1,000 bytes

There are more truncation limits around message and serialized-record size. The important consequence is simpler: count: 500 does not imply that 500 complete occurrence environments remain available. Logster is an operational exception viewer, not an immutable audit archive.

Suppression creates a harder boundary. A hard-suppressed event was never stored. Retention creates another: once an unprotected record is trimmed from Redis, the admin API has nothing left to return. Neither case is an exporter bug.

If complete prospective preservation matters, continuously ship logs to durable external storage rather than treating /logs as the archive. If exact access to every currently retained Redis message is required and custom grouping prevents API enumeration, use a server-side export or add a purpose-built admin endpoint. The stock API can fetch a retained message only when its key is available.

Security

Treat the output as sensitive. Logster environments may include:

The exporter creates its output directory with mode 0700 and files with mode 0600. Store the archive securely, do not paste it into a public gist, and revoke the temporary API key when the export is complete.

Source references

The relevant implementation is compact enough to verify directly:

The script preserves raw responses, fetches details immediately, deduplicates keys, retries transient failures, and records count mismatches in summary.json. This makes the result auditable while keeping the export process straightforward.