ChristianityAtScaleBot

User agent

Every request sends this exact user agent. It is never rotated, randomized or disguised.

ChristianityAtScaleBot/1.0 (+https://christianityatscale.com/bot; bot@christianityatscale.com)

Filter it out of download statistics by matching ^ChristianityAtScaleBot/. Our downloads are not listens and should never count toward a show's numbers or an ad.

What it fetches

• RSS feeds of public Christian podcasts.
• Publisher transcripts (<podcast:transcript>). We use these whenever a feed has them, instead of audio.
• Episode audio only when no transcript is published: a few minutes by byte range to decide whether a show fits the project, then the whole episode once if it does.
• A few pages of a podcast's own website (home and about or beliefs pages), obeying robots.txt.

How it behaves

• Feeds are re-fetched with conditional requests (ETag, If-Modified-Since) and gzip, and respect RSS skipHours and skipDays.
• A 429 or 503 is retried only after its Retry-After, and no other download goes to that host in the meantime.
• A host that answers 401, 403 or 451 repeatedly is not asked again. Refusals are respected, never routed around.
• At most 2 downloads at a time per host from each worker, hosts are spread out over time, and new runs start slowly.
• Feeds marked <podcast:block>yes or <itunes:block>Yes are not transcribed.

What we never do

• Host, republish or redistribute audio.
• Publish transcripts or quote speakers. Reports contain aggregate findings and, at most, paraphrases.
• Use audio or transcripts to train AI models.

When a report points to a specific episode, it links to the episode where its publisher hosts it, so listens count for the creator.

Opt out or get in touch

Email bot@christianityatscale.com with the feed or host you want excluded, and the crawler will stop fetching it. Hosting providers are welcome to ask for a rate limit or a different arrangement.