v1.0.0 available soon

Meet Trawlr.

A self-hosted, real time data collection platform for Telegram. Built for researchers, analysts and security professionals who require data collection and analysis at scale.

Task History
12:45:02MEDIADownloaded video_1847.mp4 (14.7MB) from group trawlr
12:44:59ENTITYExtracted 1 URL, 1 Domain and 1 Mention
12:44:58PROCESSEDMessage sent to group trawlr
12:44:51USERUser @dylan joined group trawlr
12:44:47PROCESSED3 messages from Channel #1
12:44:39ENTITYExtracted 2 URLs, 1 mention
12:44:31MEDIADownloaded photo_928.jpg (2.4MB) from channel source_channel
12:44:25FORWARDTraced to @source_channel
12:44:18SCANHistory scan completed for group trawlr
12:44:12PROCESSEDMessage from osint_feed
12:44:05USERUser @analyst_7 joined group source_channel
12:45:02MEDIADownloaded video_1847.mp4 (14.7MB) from group trawlr
12:44:59ENTITYExtracted 1 URL, 1 Domain and 1 Mention
12:44:58PROCESSEDMessage sent to group trawlr
12:44:51USERUser @dylan joined group trawlr
12:44:47PROCESSED3 messages from Channel #1
12:44:39ENTITYExtracted 2 URLs, 1 mention
12:44:31MEDIADownloaded photo_928.jpg (2.4MB) from channel source_channel
12:44:25FORWARDTraced to @source_channel
12:44:18SCANHistory scan completed for group trawlr
12:44:12PROCESSEDMessage from osint_feed
12:44:05USERUser @analyst_7 joined group source_channel

Platform Features.

Everything you need to collect, process and analyse Telegram data.

sensors

Real-Time Monitoring

Automatically capture events including new messages, edits, deletions, joins and user profile updates in real time.

archive

Message & Media Archiving

Archive message content with edit history, sender info, view counts, reactions, reply chains and forward sources. Track deletions while preserving content.

link

Automatic Entity Extraction

URLs, Domains, @mentions, #hashtags, phone numbers, emails and code blocks are automatically parsed and indexed from every received message.

person_search

User Tracking

Trawlr automatically tracks user posts, username changes, profile updates and cross-channel membership. Full change history is available.

inventory_2

Media Downloads

Configure automatic media downloads with per-channel settings. Tralwr supports message deduplication through SHA256 and Telegram file ID, concurrent downloads, automatic retry and thumbnail generation.

search

Full-Text Search

Full text search powered by PostgreSQL Gin. Search across messages, media and entities by source, sender, media type, date range, and extracted data.

forum

Forum/Topic Support

Automatic detection of forum-enabled supergroups with topic syncing and message attribution.

device_hub

Multi-Account Architecture

Parallel listeners support multiple accounts with flood wait (rate limit) detection. These work in tandem to scale data collection.

api

REST API

The REST API supports token authentication and contains OpenAPI/Swagger documentation, advanced filtering and pagination for data stored by Trawlr.

Telegram MTProto API
hearing listener Listens for and pushes new events into the processing queue
RabbitMQ
engineering Concierge
download Downloader
history Processor
storage Database
dashboard Web
schedule Scheduler

Built for Scale.

Trawlr uses a container based architecture to allow for the scaling of it's supporting services.

  • check_circle Containerised deployment with Docker Compose
  • check_circle Distributed task processing using Dramatiq Tasks and RabbitMQ
  • check_circle Download pictures, videos and files using a priority based queue
  • check_circle Real-time download manager using WebSocket updates
  • check_circle Periodic task orchestration using APScheduler
  • check_circle Django and HTMX powered web front end
  • check_circle PostgreSQL database backend
  • check_circle Support for archiving with Apache Solr & Elastic coming soon

How It Works.

From deployment to intelligence in four steps.

Connect

Add your Telegram account/s. Trawlr supports multiple accounts.

Monitor

Telegram events are captured in real time. Join new sources from Trawlr or use channel auto-discover to keep data updated.

Archive and Download

Download historical messages, media and metadata from Telegram sources. View downloaded content in the file manager.

Analyse

Perform full-text search across messages and users, or visualise entity data using built-in vis.js node/edge maps