Complete Guide to Instagram Scraping API: Extract Posts, Stories, User Data & More

Instagram is still one of the more useful social datasets on the web, but it is also one of the more frustrating ones to collect directly.

The official API is limited, a lot of the frontend is tied to internal requests, and basic scraping setups tend to break quickly. This guide focuses on the endpoints that are most practical if you need profile data, posts, comments, hashtags, or media downloads.

Why Instagram Is Hard to Scrape Reliably

Before getting into the endpoints, it helps to understand why direct scraping breaks so easily:

What You're Up Against

❌ The official API is basically useless: You need app review, tons of permissions, and even then you can barely access anything
❌ Their bot detection is insane: Browser fingerprinting, behavior tracking—they know when you're not a real person
❌ Everything needs login: Can't see much without being authenticated, which makes automation even harder
❌ Rate limits everywhere: Make too many requests and you're blocked. Sometimes permanently.
❌ GraphQL headaches: All the content loads through complex API calls that change constantly

15 Instagram Endpoints That Cover Most Workflows

The endpoints below cover the workflows people usually need first: profile lookups, post and reel extraction, comment collection, and keyword-based discovery.

1. Get User Profile by Username

This is usually the starting point. Pass a username and you get the account profile, follower counts, bio, and other top-level fields.

# Get user profile by username
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/user/by_username",
    "params": {
      "username": "instagram"
    }
  }'

User Profile Data

Basic info: Username, full name, biography, profile picture
Stats: Follower count, following count, total posts
Verification: Verified badge, business account status
Contact: Email, website, phone number (if public)
Category: Account category/type for business accounts

2. Get User's Posts

Use this when you want the recent posts for a profile, whether that is for monitoring, analytics, or content archiving. The response includes page_info.end_cursor when there is another page.

# Get user's posts by user ID
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/medias/by_user_id",
    "params": {
      "user_id": "25025320",
      "count": 12,
      "end_cursor": ""
    }
  }'

Leave end_cursor empty for the first request, then reuse the exact page_info.end_cursor value from the previous response to grab the next batch.

3. Get User's Reels

If you specifically need reels (Instagram's TikTok competitor), this pulls just those. Same idea as regular posts, but filtered to short videos. Reels pagination uses paging_info.max_id.

# Get user's reels
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/reels/by_user_id",
    "params": {
      "user_id": "25025320",
      "count": 12,
      "max_id": ""
    }
  }'

4. Get Post by URL or Shortcode

Got a link to an Instagram post? Drop it in here and you'll get the full details—images, videos, captions, likes, comments, all of it.

# Get post by URL
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/media/by_url",
    "params": {
      "url": "https://www.instagram.com/p/ABC123xyz/"
    }
  }'

# Or get by shortcode directly
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/media/by_shortcode",
    "params": {
      "shortcode": "ABC123xyz"
    }
  }'

Post Data Structure

Content: Caption, hashtags, mentions
Media: Photo URLs, video URLs (HD)
Engagement: Like count, comment count, view count
Location: Tagged location if available
Timestamp: When the post was created
Carousel: Multiple images/videos if it's a carousel post

5. Get Post Comments

If you need engagement or discussion data, this endpoint returns the comments attached to a post. When more comments are available, use page_info.end_cursor to paginate.

# Get post comments
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/comments/media_comments_by_id",
    "params": {
      "media_id": "2834567890123456789",
      "end_cursor": ""
    }
  }'

6. Search Users

Looking for influencers or specific accounts? Search by keyword and find users who match.

# Search for users
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/search/users_by_keyword",
    "params": {
      "keyword": "photography"
    }
  }'

7. Search Hashtags

This endpoint is useful when you want to gauge interest around a hashtag or discover related topics.

# Search hashtags
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/search/hashtags_by_keyword",
    "params": {
      "keyword": "fitness"
    }
  }'

8. Search Places/Locations

Need posts from a specific location? Search for places and get all the content tagged there.

# Search places
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/search/places_by_keyword",
    "params": {
      "keyword": "New York"
    }
  }'

What Can You Actually Build With This?

Influencer Analytics Dashboard

One straightforward use case is influencer tracking. Pull the profile, fetch recent posts, and calculate an engagement rate from the returned metrics.

// JavaScript example - Track influencers
async function analyzeInfluencer(username) {
  // Get profile info
  const profileRes = await fetch('https://scrapingbot.io/api/v1/instagram', {
    method: 'POST',
    headers: {
      'x-api-key': 'YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      endpoint: '/user/by_username',
      params: { username }
    })
  });
  
  const profile = await profileRes.json();
  
  // Get their posts
  const postsRes = await fetch('https://scrapingbot.io/api/v1/instagram', {
    method: 'POST',
    headers: {
      'x-api-key': 'YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      endpoint: '/medias/by_user_id',
      params: { user_id: profile.data.id }
    })
  });
  
  const posts = await postsRes.json();
  
  // Calculate engagement rate
  const totalLikes = posts.data.items.reduce((sum, p) => sum + p.like_count, 0);
  const avgLikes = totalLikes / posts.data.items.length;
  const engagementRate = (avgLikes / profile.data.follower_count) * 100;
  
  return {
    username,
    followers: profile.data.follower_count,
    avgLikes,
    engagementRate: engagementRate.toFixed(2) + '%'
  };
}

Brand Monitoring

Brand monitoring is another common use case. One simple approach is to follow relevant hashtags and inspect the volume and recent activity.

# Python example - Monitor brand hashtags
import requests

def monitor_brand_hashtag(brand_hashtag):
    response = requests.post(
        'https://scrapingbot.io/api/v1/instagram',
        headers={'x-api-key': 'YOUR_API_KEY'},
        json={
            'endpoint': '/search/hashtags_by_keyword',
            'params': {'keyword': brand_hashtag}
        }
    )
    
    data = response.json()
    
    if data['success']:
        hashtag_data = data['data']
        print(f"#{brand_hashtag} Statistics:")
        print(f"  - Total posts: {hashtag_data['media_count']:,}")
        print(f"  - Trending: {'Yes' if hashtag_data.get('is_trending') else 'No'}")
        
        # Get recent posts with this hashtag
        # (would require additional API calls)
        
    return data

monitor_brand_hashtag('mybrand')

Instagram Downloader

Building a tool to download Instagram photos/videos? It's actually pretty simple:

// Download Instagram post
async function downloadInstagramPost(postUrl) {
  const response = await fetch('https://scrapingbot.io/api/v1/instagram', {
    method: 'POST',
    headers: {
      'x-api-key': 'YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      endpoint: '/media/by_url',
      params: { url: postUrl }
    })
  });
  
  const data = await response.json();
  
  if (data.success) {
    const media = data.data;
    
    if (media.video_url) {
      // Download video
      console.log('Video URL:', media.video_url);
      window.open(media.video_url, '_blank');
    } else if (media.image_url) {
      // Download image
      console.log('Image URL:', media.image_url);
      window.open(media.image_url, '_blank');
    }
    
    // For carousel posts, media.carousel_media contains all items
    if (media.carousel_media) {
      media.carousel_media.forEach((item, i) => {
        console.log(`Item ${i + 1}:`, item.image_url || item.video_url);
      });
    }
  }
}

Practical Tips Before You Scale This Up

✅ Smart Moves

✓ Cache profiles: People don't change their bio every hour. Cache it and save some credits.
✓ Don't grab everything: If you only need the last 10 posts, don't fetch all 1,000. Use pagination smartly.
✓ Check if accounts are private: You can't get posts from private accounts anyway, so check that first.
✓ Watch your credits: Each API call costs 5 credits. Don't go crazy with unnecessary requests.

⚠️ Mistakes I've Seen (And Made)

⚠ Adding @ to usernames: Don't. Just use "username", not "@username". The @ will break it.
⚠ Mixing up IDs and usernames: Some endpoints need the numeric user_id, not the username. Check the docs.
⚠ Forgetting about private accounts: You can't just scrape someone's private account. The API will return nothing.
⚠ Not handling errors: Posts get deleted. Accounts get banned. Always check if the response is actually successful.

How to Get Started

If you want to test the API quickly, start with a short setup:

Quick Start

1
Sign up — You get 100 free credits to test with
2
Grab your API key — It's sitting in your dashboard the moment you sign up
3
Copy one of the examples above — Start with a profile lookup or a single post URL
4
Build around one clear use case — Analytics, downloads, or monitoring

Start Scraping Instagram Now →

Closing Thoughts

Instagram scraping is still a moving target, but it gets much easier when the extraction layer is already handled for you.

If you are also collecting data from TikTok, the companion guide Complete Guide to TikTok Scraping API covers the equivalent workflows for TikTok endpoints.

Next step: Start with one profile, one post, or one hashtag query and confirm the response shape before you build the larger pipeline.