Scrape trending Twitter hashtags

Scraping trending Twitter hashtags can be done using either Twitter’s API (the official and ethical method) or unofficial scraping methods (not recommended due to policy violations). Here’s a practical breakdown of how to do it with the Twitter API v2, which is the most reliable and sustainable method.

How to Scrape Trending Twitter Hashtags Using Twitter API v2

Step 1: Set Up Your Developer Account

1. Go to Twitter Developer Portal.

2. Apply for a free developer account.

3. Create a Project and an App to obtain your Bearer Token.

Step 2: Get WOEID for Trend Locations

Twitter provides trends by geographic region. You’ll need the WOEID (Where On Earth ID) for your target location.

Common WOEIDs:

• Worldwide: 1

• United States: 23424977

• United Kingdom: 23424975

• India: 23424848

You can also use the endpoint to get available locations:

bash
CopyEdit
GET https://api.twitter.com/1.1/trends/available.json

Step 3: Fetch Trending Hashtags

Python Example Using Tweepy (v4)

python
CopyEdit
import tweepy

# Authenticate
bearer_token = "YOUR_BEARER_TOKEN"
client = tweepy.Client(bearer_token=bearer_token)

# Tweepy v4 does not directly support trends; use v1.1 with OAuth1 for this
import requests
from requests_oauthlib import OAuth1

consumer_key = "YOUR_CONSUMER_KEY"
consumer_secret = "YOUR_CONSUMER_SECRET"
access_token = "YOUR_ACCESS_TOKEN"
access_token_secret = "YOUR_ACCESS_TOKEN_SECRET"

auth = OAuth1(consumer_key, consumer_secret, access_token, access_token_secret)

# WOEID for Worldwide trends
woeid = 1

url = f"https://api.twitter.com/1.1/trends/place.json?id={woeid}"
response = requests.get(url, auth=auth)

if response.status_code == 200:
    trends = response.json()[0]['trends']
    for trend in trends:
        if trend['name'].startswith('#'):
            print(trend['name'])
else:
    print(f"Failed to retrieve trends: {response.status_code}")

Step 4: Automate & Store Data

You can run this script on a schedule using a tool like cron (Linux/macOS) or Task Scheduler (Windows) to monitor trends every X minutes or hours. Store them in:

• CSV or JSON file

• SQLite/MySQL database

• Google Sheets via API

Notes

• Rate limits: 75 requests per 15 minutes for /trends/place (API v1.1).

• Terms of Use: Never publish scraped data without attribution or violate Twitter’s Developer Policy.

• Upgrading: For more advanced filtering, use Twitter API v2’s search endpoints to analyze tweets with specific hashtags.

Optional: Using Third-Party APIs

If you want an even easier way (especially without OAuth1 headaches), try these services:

• Trends24 (visual trends, no API)

• GetDayTrends – Free API, limited access

• SocialSearcher – Paid plans with hashtag analytics

Let me know if you need a no-code method, dashboard integration, or webhook setup.