--- name: Fetch Tweets slug: fetch-tweets category: Automation description: Fetch Tweets searches X/Twitter by keyword, topic, account, list, or AI-agent preset and turns results into a clustered digest. Use it when you need a curated roundup with signal scoring instead of a chronological feed. github: "https://github.com/aeonfun/aeon/tree/main/skills/fetch-tweets" language: TypeScript stars: 679 forks: 242 install: "npx degit https://github.com/aeonfun/aeon/tree/main/skills/fetch-tweets ~/.claude/skills/fetch-tweets" installs_to: ~/.claude/skills/fetch-tweets source_path: skills/fetch-tweets/SKILL.md collection_size: 25 category_size: 1523 collection_url: "https://dirskills.com/collections/aeonfun/aeon" added: 2026-08-24T05:16:31.890Z last_synced: 2026-08-24T05:16:31.890Z canonical_url: "https://dirskills.com/skills/fetch-tweets" --- # Fetch Tweets Fetch Tweets searches X/Twitter by keyword, topic, account, list, or AI-agent preset and turns results into a clustered digest. Use it when you need a curated roundup with signal scoring instead of a chronological feed. **Install:** ```bash npx degit https://github.com/aeonfun/aeon/tree/main/skills/fetch-tweets ~/.claude/skills/fetch-tweets ``` ## README > **${var}** — `:` where `` ∈ `keyword | topic | account | list | agent-buzz`. The `` is source-specific (a query, a topic, a handle, comma-separated list IDs, or an optional focus). If no `source:` prefix is given, the source is inferred from the shape of `` (see **Source selector**). **Required** for `keyword` and `list`; optional for `topic`, `account`, and `agent-buzz`. Today is ${today}. This skill fetches X/Twitter content along one of five **source axes** and produces a *curated* digest — clustered by sub-narrative, ranked by signal, one insight per item — never a flat chronological dump. ## Source selector Parse `${var}` into `SOURCE` and `ARG` before doing anything else. **Explicit form (recommended):** `:` - `keyword:$SOL OR solana OR "solana network"` — raw X search query, passed to Grok **verbatim** (OR/AND honored). - `topic:brain-computer interfaces` — a single topic roundup. `topic:` (empty arg) → resolve a topic **list** from MEMORY.md, then built-in defaults. - `account:vitalikbuterin` — one account's recent tweets. `account:` (empty arg) → digest **every** handle in `memory/topics/tracked-accounts.yml`. - `list:1953536336675365173,1937207796270829766` — one or more numeric X list IDs. Append `|` for a topic booster: `list:195...,193...|AI agents`. - `agent-buzz` — the curated AI-agent-ecosystem preset. `agent-buzz:MCP protocol` prioritizes a project/topic within the preset. **Implicit form (back-compat with migrated bare-var configs):** when `${var}` has **no** recognized `source:` prefix, infer `SOURCE` in this order: 1. `${var}` is empty → `topic` (default multi-topic roundup). 2. `${var}` is all-digits, or comma-separated all-digits (optionally with a `|` suffix) → `list`. 3. `${var}` is `@handle` or matches `^[A-Za-z0-9_]{1,15}$` (a bare handle) → `account`. 4. Anything else → `keyword`. Note: `agent-buzz` has **no** distinct implicit shape (its arg looks like a keyword/topic), so it is **only** selectable via the explicit `agent-buzz` / `agent-buzz:...` prefix. Once `SOURCE` and `ARG` are set, jump to the matching branch below. Only one branch runs per invocation. ## Shared preamble (all branches) 1. Read `memory/MEMORY.md` for context and the recent `memory/logs/` (each branch specifies its lookback window — 2 or 3 days) to dedup already-reported tweets. 2. **Load the dedup set `SEEN_TWEETS`** by unioning two sources: - The branch's **persistent seen-file** (per-mode path below), if it exists — read all URLs. - The branch's **log lookback window** — grep each `memory/logs/*.md` file in range for lines matching `https://x.com/`. Per-mode seen-files (kept at their legacy paths so dedup history survives the merge): | mode | seen-file | log lookback | |---|---|---| | keyword | `memory/fetch-tweets-seen.txt` | 3 days | | topic | `memory/tweet-roundup-seen.txt` | 3 days | | account | *(logs only — see branch)* | 2 days | | list | `memory/list-digest-seen.txt` | 2 days | | agent-buzz | *(logs only — 3-day `status/` set)* | 3 days | 3. Formatting invariants shared by **every** branch's notification: - Use `x.com/handle` (**never** `@handle`) so Telegram doesn't ping/tag users. *(Exception: the account-digest and agent-buzz formats below historically use `@handle` in-body; keep their documented format but prefer `x.com/handle` when practical.)* - Every surviving tweet gets a tappable Markdown link — `[View](url)` / `[View tweet](url)`. If a URL is unavailable, drop the link and say "(link unavailable)". - Never fabricate engagement counts. Missing → `0`, not a guess. - **Notify only on signal.** A legitimately empty or all-duplicate run logs its status and sends **nothing**. ## Voice Used by the `account` and `agent-buzz` branches for one-line takes/insights. If `soul/SOUL.md` and `soul/STYLE.md` are populated, read both and match the operator's voice. If they are empty templates or absent, write in a clear, direct, neutral tone — state what the tweet says, no hedging or editorializing beyond the tweet itself. --- ## Branch: keyword (`source:keyword`) Search X for tweets matching `ARG` and produce a curated digest grouped by sub-narrative. **Seen set:** `memory/fetch-tweets-seen.txt` + last 3 days of logs (loaded in preamble). 1. **Build the search prompt.** Pass `ARG` to Grok **verbatim** as the query — do NOT narrow it to a single angle; broad coverage is the goal. Ask for **at least 15–20 candidate tweets** (you'll cull to ~7–10). Always require explicit engagement counts (likes, retweets, replies) so ranking is data-driven. 2. **Fetch tweets.** Record `SOURCE_PATH=api|websearch` for the log. **Path A — X.AI API** (primary; see the **Fetching (all branches)** contract — attempt this, set the Bash tool `timeout` ≥180000, capture the HTTP status): ```bash FROM_DATE=$(date -u -d "yesterday" +%Y-%m-%d 2>/dev/null || date -u -v-1d +%Y-%m-%d) TO_DATE=$(date -u +%Y-%m-%d) PROMPT="Search X for tweets about: ${ARG}. Date range: ${FROM_DATE} to ${TO_DATE}. Return at least 15-20 candidate tweets — mix of high-engagement posts and smaller accounts that add a distinct angle. For each tweet include: @handle, the full text, date posted, exact engagement counts (likes, retweets, replies — never N/A; if unknown, say 0), and the direct link (https://x.com/handle/status/ID). Return as a numbered list." jq -n --arg p "$PROMPT" '{model:"grok-4.6", input:[{role:"user",content:$p}], tools:[{type:"x_search"}]}' > /tmp/xai-ft-keyword.json HTTP=$(./secretcurl -s -o /tmp/xai.json -w '%{http_code}' --max-time 150 -X POST "https://api.x.ai/v1/responses" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer {XAI_API_KEY}" \ -d @/tmp/xai-ft-keyword.json) echo "xai http=$HTTP bytes=$(wc -c " after:${FROM_DATE}`. Note at the top of the log the **true reason** (`http-` / `timeout` / `empty`, never "unavailable" when the key was set) and "results compiled via WebSearch — quality lower than usual". WebSearch favours high-engagement older tweets — **prioritise results dated within the last 48 hours**. Mark `SOURCE_PATH=websearch`. 3. **Empty vs. error handling** (distinguish): - **Legitimate empty** (0 tweets): log `FETCH_TWEETS_EMPTY (source=${SOURCE_PATH})` and **stop — no notification**. - **API/cache error** (HTTP error, malformed JSON, all paths failed): log `FETCH_TWEETS_ERROR (last_path=${SOURCE_PATH}, reason=...)` and **stop — no notification**. 4. **Deduplicate** each candidate URL against `SEEN_TWEETS`. If ALL are dupes: log `FETCH_TWEETS_NO_NEW: all results already reported` and **stop — no notification**. 5. **Curate** (the core step): a. **Cluster** survivors into 2–4 sub-narratives by what they're claiming/discussing (e.g. for a token: "price action", "team announcement", "criticism/FUD", "ecosystem integration"). Name the *angle*, not the topic. b. **Rank within each cluster by signal** (not raw engagement): `signal = likes + 2×retweets + replies`, but **demote** pure replies, generic shilling, and near-duplicate paraphrases. Drop tweets with <5 total engagement unless they add a unique angle. c. **Cap each cluster at 2–3 tweets, total 7–10.** Quality over quantity — if only 5 pass, send 5. Don't pad. d. **Extract the claim/signal** per tweet — *what's new or interesting*, not a literal paraphrase. Bad: "User says token is going up." Good: "Calls out the team's silence on the postponed unlock — first major holder to do so publicly." e. **Compute a one-line signal** for the top of the notification — one observation about the *shape* of the conversation (e.g. "Sentiment split — 4 bullish on the launch, 3 critical of the unlock terms."). 6. **Save + update seen-file** (see Log). Append each kept tweet URL (one per line) to `memory/fetch-tweets-seen.txt` (create if missing). 7. **Notify via `./notify`** with the clustered output: ``` *Top Tweets — ${ARG} (${today})* _${signal_one_liner}_ *${cluster_1_name}* 1. x.com/handle — [insight summary] Likes: X | RTs: Y | Replies: Z [View tweet](https://x.com/handle/status/ID) 2. x.com/handle — [insight summary] Likes: X | RTs: Y | Replies: Z [View tweet](https://x.com/handle/status/ID) *${cluster_2_name}* 3. x.com/handle — [insight summary] ... ``` The signal one-liner is italic (`_..._`) directly under the title; cluster headers are `*bold*`. **Status codes:** `FETCH_TWEETS_OK` (notified) | `FETCH_TWEETS_EMPTY` | `FETCH_TWEETS_ERROR` | `FETCH_TWEETS_NO_NEW`. --- ## Branch: topic (`source:topic`) Gist of the latest X chatter on one or more configurable topics. **Seen set:** `memory/tweet-roundup-seen.txt` + last 3 days of logs. 1. **Resolve the topic list** (priority order): 1. `ARG` set → `TOPICS=("$ARG")` (single-topic mode). 2. Else if MEMORY.md has a `## Tweet Roundup Topics` section → use its bulleted lines, one query per line. 3. Else built-in defaults: - `artificial intelligence OR AI agents OR LLM` - `crypto OR bitcoin OR DeFi` - `technology OR startups OR open source` 2. **Fetch per topic** — track `SOURCE ∈ {api, websearch, failed}` per topic. **Path A — direct X.AI curl** (primary): for each topic, call Grok's `x_search`. ```bash FROM_DATE=$(date -u -d "yesterday" +%Y-%m-%d 2>/dev/null || date -u -v-1d +%Y-%m-%d) TO_DATE=$(date -u +%Y-%m-%d) PROMPT="Search X for recent tweets about: ${TOPIC}. Date range: ${FROM_DATE} to ${TO_DATE}. Return up to 8 substantive tweets. For each: @handle, full text, date, exact engagement counts (likes, retweets, replies; 0 if unknown), and the direct link https://x.com/handle/status/ID." jq -n --arg p "$PROMPT" '{model:"grok-4.6", input:[{role:"user",content:$p}], tools:[{type:"x_search"}]}' > /tmp/xai-ft-topic.json ./secretcurl -s -o /tmp/xai-topic-out.json -X POST "https://api.x.ai/v1/responses" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer {XAI_API_KEY}" \ -d @/tmp/xai-ft-topic.json ``` Parse with the standard `jq` extractor. If it yields text, `SOURCE=api`. Extract each tweet's `@handle`, text, engagement counts, and permalink. **Path B — WebSearch fallback** (only if `XAI_API_KEY` unset, or Path A errors/empty): `site:x.com "" after:`. Always include the word "today" and `${today}` to force fresh results. Discard any result whose visible date is older than 48h. Collect up to 5 candidates per topic. Mark `SOURCE=websearch`. If both paths return nothing, mark `SOURCE=failed`. 3. **Score and filter.** Require: a known `@handle`; a `https://x.com//status/` URL (if missing, keep but mark "link unavailable"); posted within 48h; URL **not** in `SEEN_TWEETS`. Compute `signal_score = likes + 2×retweets + replies` (on WebSearch path with no counts, use result rank as a weak proxy). **Demote −50%**: replies to a parent tweet; near-duplicates of a higher-scoring tweet (>70% text overlap or same linked URL). 4. **Curate per topic:** - **0 survivors** → drop the topic. Do NOT pad. - **1–3 survivors** → list ranked by `signal_score`, highest first. - **4+ survivors** → group into 2–3 sub-narratives (shared keywords/entity/claim); label each, surface the top-1 tweet per narrative as exemplar. Write an **insight** per reported tweet (what it asserts/reveals, not a headline paraphrase). Write a one-line **conversation shape** per topic ("bullish momentum, dissenters quiet", "split opinion on X's launch", "single story dominating — Y"). 5. **Notify.** If every topic dropped: log `TWEET_ROUNDUP_EMPTY` and **stop — no notify**. Otherwise send via `./notify` (≤4000 chars): ``` *Tweet Roundup — ${today}* _Source: api:X websearch:Y failed:Z_ *[Topic 1]* — _conversation shape_ - x.com/handle — insight (signal: 12.3k) [View](https://x.com/handle/status/ID) - x.com/handle — insight (signal: 4.1k) [View](https://x.com/handle/status/ID) *[Topic 2]* — _conversation shape_ - x.com/handle — insight (signal: 8k) [View](https://x.com/handle/status/ID) ``` Show `signal: ` only when engagement counts were available (api path); omit silently on WebSearch. 6. **Persist + log** (see Log). Append each reported URL (one per line) to `memory/tweet-roundup-seen.txt` (create if missing). **Constraints:** never notify an empty roundup (silence beats filler); never `@handle` anyone; never report a URL already in `SEEN_TWEETS`. **Status codes:** `TWEET_ROUNDUP_OK` | `TWEET_ROUNDUP_EMPTY`. --- ## Branch: account (`source:account`) Two sub-modes: **single handle** (decision-ready gist of one account) vs. **all tracked accounts** (theme-grouped digest of a watchlist). Choose by `ARG`. **Seen set:** last 2 days of logs — extract every `https://x.com/` URL under a prior `### fetch-tweets` account entry into `SEEN_URLS`. ### account — single handle (`ARG` is one @handle) 1. **Normalize `ARG`.** Strip leading `@`, `https://x.com/`, `https://twitter.com/`, `https://nitter.net/`, trailing slash / `/status/...`. Lowercase. Reject if empty, contains whitespace, or >15 chars. On reject → `REFRESH_X_NO_VAR`: send `./notify "fetch-tweets: REFRESH_X_NO_VAR — set an X handle"` and exit 0. Store the cleaned handle as `ACCOUNT`. 2. **Load tweets:** - **Path A — X.AI API** (primary): search this account's recent tweets via Grok's `x_search`. ```bash PROMPT="Search X for the latest tweets, replies, and quote tweets from @${ACCOUNT} in the last 2 days. Return each with full text, timestamp, type (original|reply|quote), what it replies to/quotes if any, exact engagement counts (likes, retweets, replies; 0 if unknown), and the permalink https://x.com/${ACCOUNT}/status/ID. Skip retweets of others. Return chronological." jq -n --arg p "$PROMPT" '{model:"grok-4.6", input:[{role:"user",content:$p}], tools:[{type:"x_search"}]}' > /tmp/xai-ft-account.json ./secretcurl -m 30 -s -o /tmp/xai-account-out.json -X POST "https://api.x.ai/v1/responses" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer {XAI_API_KEY}" \ -d @/tmp/xai-ft-account.json ``` Parse with the standard `jq` extractor. Record `source=api`. - **Path B — WebFetch fallback** (only if `XAI_API_KEY` unset, or Path A errors / parsed text has zero x.com status URLs): WebFetch `https://x.com/${ACCOUNT}` with prompt: *"List every tweet, reply, and quote tweet visible on this profile with its full text, timestamp, engagement counts (likes/retweets/replies) if shown, and the permalink https://x.com/handle/status/ID. Return a chronological list."* Record `source=webfetch`. - **Path C — degraded**: if `XAI_API_KEY` unset and WebFetch returns nothing → skip to step 8 with status `REFRESH_X_NO_API_KEY` (key missing) or `REFRESH_X_ERROR` (key set but both paths failed). 3. **Parse into structured tweets:** `url`, `text`, `timestamp`, `type` (original/reply/quote), `reply_to`, `quoted_text`, `likes`, `retweets`, `replies`. Drop retweets of others. Missing counts → 0. Compute `signal_score = likes + 2*retweets + replies − (3 if type=reply else 0)`. 4. **Dedup and gate:** drop any tweet whose `url` is in `SEEN_URLS` (`deduped_count`). If fewer than 3 tweets survive AND no thread is detectable (step 5) → skip to step 8 with `REFRESH_X_NO_NEW` (everything deduped) or `REFRESH_X_EMPTY` (account posted nothing). 5. **Detect threads:** a thread = 2+ tweets by `ACCOUNT` within 30 minutes where later tweets reply to earlier ones OR share ≥2 meaningful keywords with the opener. Thread tweets are atomic units regardless of individual score. Record `{opener_url, tweet_count, combined_signal}`. 6. **Cluster and extract insights:** group survivors (threads = one unit) into **2–4 sub-narratives** by topic overlap; if <2 emerge, use one cluster. Per cluster: **Title** (3–8 words), **Top tweet(s)** (1–3 excerpts ≤200 chars each, with permalink + engagement), **Insight** (one sentence — what the cluster reveals about the author's stance/claim/shift; not a paraphrase — if you can't beat paraphrase, drop the cluster). Per thread: a 1–2 sentence landing summary + opener URL. 7. **Write the verdict** (pick exactly one) + a ≤20-word lede: | Verdict | When | |---|---| | `ANNOUNCEMENT` | launch, hire, policy, or product drop | | `ARGUMENT` | majority signal from contrarian takes or fights | | `BUILDING` | ships/code/tech-progress clusters dominate | | `SHITPOST` | jokes, memes, low-stakes banter dominate | | `CONTEXT` | mostly reacting to a news cycle, not driving one | | `QUIET` | <3 originals and no thread | 8. **Save gist** (see Log). On empty/no-new/error/no-var statuses, write only the account header + status footer, skip cluster sections. 9. **Update MEMORY.md (conditional):** only if a cluster carries an announcement, specific claim, named project, or stance shift — add one bullet under a `## Tracked X Accounts` section (create if missing): `- @ACCOUNT YYYY-MM-DD: [one-sentence claim] — [permalink]`. No paraphrases/memes/generic opinions. 10. **Notify via `./notify`.** On `REFRESH_X_OK`: ``` x refresh — @ACCOUNT ([VERDICT]) [lede] top cluster: [title] — "[≤80 char excerpt]" ([likes]❤) [N tweets, T threads, K deduped] ``` On `REFRESH_X_EMPTY` / `REFRESH_X_NO_NEW`: **skip notify** (write the log entry only). On `REFRESH_X_NO_API_KEY` / `REFRESH_X_ERROR` / `REFRESH_X_NO_VAR`: notify with the status code + a one-line hint (e.g. `"fetch-tweets: REFRESH_X_NO_API_KEY — set XAI_API_KEY in workflow secrets"`). **Constraints:** never fabricate engagement; never include a `SEEN_URLS` URL; an insight that only paraphrases is not an insight (drop the cluster); MEMORY.md updates are one line each. **Status codes:** `REFRESH_X_OK` | `REFRESH_X_EMPTY` | `REFRESH_X_NO_NEW` | `REFRESH_X_NO_API_KEY` | `REFRESH_X_ERROR` | `REFRESH_X_NO_VAR`. ### account — all tracked accounts (`ARG` empty) Use this to answer "what did *these specific people* post" across a watchlist. 1. **Read config** `memory/topics/tracked-accounts.yml`. If missing or `accounts: []` → log `TWEET_DIGEST_NO_CONFIG` and exit (no notification). Schema: ```yaml accounts: - handle: vitalikbuterin why: ethereum core thinking # optional — grouping/context label - handle: balajis why: macro + tech narratives ``` 2. **Fetch recent tweets per account.** For each `handle`: - **Path A — live curl** (primary, `XAI_API_KEY` is injected and set): ```bash PROMPT="Search X for the latest tweets from:${HANDLE} in the last 3 days. Return the 5 most interesting or substantive tweets. For each: full text, date, direct link (https://x.com/${HANDLE}/status/ID). Skip retweets of others." jq -n --arg p "$PROMPT" '{model:"grok-4.6", input:[{role:"user",content:$p}], tools:[{type:"x_search"}]}' > /tmp/xai-ft-acct1.json ./secretcurl -m 30 -s -o /tmp/xai-acct1-out.json -X POST "https://api.x.ai/v1/responses" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer {XAI_API_KEY}" \ -d @/tmp/xai-ft-acct1.json ``` Parse with the standard `jq` extractor. If `XAI_API_KEY` is unset, log `TWEET_DIGEST_NO_KEY: skill requires XAI_API_KEY` and exit (no notification). **Dedup:*