AI Digest Daily
Published:
Duration: Jan 2025 – Present
Affiliation: Personal
Overview
Automated daily curation pipeline that fetches ArXiv papers, RSS tech news, and GitHub trending repositories each morning, scores them for relevance with an LLM, and publishes a digest to GitHub Pages with a Telegram push notification. Runs entirely on a self-hosted GitHub Actions runner — SQLite persists between runs with no cloud database required. See demo: https://hiimmuc.github.io/Personal-AI-Digest/
Key Features
- ArXiv pipeline — RSS per category + keyword search; Semantic Scholar h-index prefilter before any LLM call; two LLM passes for relevance scoring and structured insight generation; Papers With Code code-repo lookup
- News pipeline — RSS/Atom polling with URL deduplication against SQLite; LLM summarisation with topic categorisation and relevance filtering
- GitHub Trending pipeline — Keyword pre-filter before LLM; GitHub API enrichment (README, topics, push date); relevance scoring and persistence
- Weekly rollup — LLM-generated narrative paragraph from the week’s DB entries, published every Sunday
- Delivery — Daily and weekly digests committed to GitHub Pages; Telegram push for papers and news headlines
- LLM routing — Sequential fallback: self-hosted vLLM → Ollama → Azure OpenAI
Technologies
Python SQLite Jekyll GitHub Actions vLLM Ollama Azure OpenAI ArXiv API Semantic Scholar API Papers With Code API Telegram Bot API feedparser gtrending
Gallery
