Complete Guide to Instagram Scraping API: Extract Posts, Stories, User Data & More

Instagram has over 2 billion active users and is one of the richest sources of visual content and social data on the internet. Whether you're building a social media analytics tool, tracking brand mentions, or analyzing influencer content, scraping Instagram data is incredibly valuable. But here's the challenge: Instagram's API is extremely limited for third-party developers, and their anti-bot protection is among the toughest out there.

In this guide, I'll walk you through every Instagram scraping endpoint available through ScrapingBot, show you exactly what data you get back, and give you real examples you can use right away.

Why Scraping Instagram Is So Challenging

Before we dive into solutions, let's talk about why Instagram scraping is notoriously difficult:

The Instagram Scraping Challenge

❌ Limited API access: Instagram's official API is severely restricted and requires app review
❌ Aggressive bot detection: Instagram uses sophisticated fingerprinting and behavioral analysis
❌ Login requirements: Most content requires authentication to access
❌ Rate limiting: Instagram aggressively throttles automated requests
❌ Dynamic content: Everything loads via complex GraphQL API calls

The ScrapingBot Instagram API: 15 Powerful Endpoints

ScrapingBot provides access to 15 different Instagram endpoints that cover everything from user profiles to posts, reels, comments, and search. Let's go through each one with real examples.

1. Get User Profile by Username

Get complete profile information for any Instagram user. This is the most commonly used endpoint.

# Get user profile by username
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/user/by_username",
    "params": {
      "username": "instagram"
    }
  }'

User Profile Data

Basic info: Username, full name, biography, profile picture
Stats: Follower count, following count, total posts
Verification: Verified badge, business account status
Contact: Email, website, phone number (if public)
Category: Account category/type for business accounts

2. Get User's Posts

Extract all posts from a user's profile. Perfect for content analysis and tracking posting patterns.

# Get user's posts by user ID
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/medias/by_user_id",
    "params": {
      "user_id": "25025320",
      "max_id": ""
    }
  }'

The max_id parameter is for pagination. Start with an empty string, then use the returned next_max_id to fetch the next page.

3. Get User's Reels

Extract specifically reels content from a user. Reels are Instagram's short-form video content.

# Get user's reels
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/reels/by_user_id",
    "params": {
      "user_id": "25025320"
    }
  }'

4. Get Post by URL or Shortcode

Get detailed information about a specific post including images, videos, captions, and engagement metrics.

# Get post by URL
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/media/by_url",
    "params": {
      "url": "https://www.instagram.com/p/ABC123xyz/"
    }
  }'

# Or get by shortcode directly
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/media/by_shortcode",
    "params": {
      "shortcode": "ABC123xyz"
    }
  }'

Post Data Structure

Content: Caption, hashtags, mentions
Media: Photo URLs, video URLs (HD)
Engagement: Like count, comment count, view count
Location: Tagged location if available
Timestamp: When the post was created
Carousel: Multiple images/videos if it's a carousel post

5. Get Post Comments

Extract all comments from a post. Great for sentiment analysis and engagement tracking.

# Get post comments
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/comments/media_comments_by_id",
    "params": {
      "media_id": "2834567890123456789"
    }
  }'

6. Search Users

Find Instagram users by keywords. Perfect for influencer discovery.

# Search for users
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/search/users_by_keyword",
    "params": {
      "keyword": "photography"
    }
  }'

7. Search Hashtags

Discover trending hashtags and get hashtag analytics.

# Search hashtags
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/search/hashtags_by_keyword",
    "params": {
      "keyword": "fitness"
    }
  }'

8. Search Places/Locations

Find Instagram posts by location. Useful for local business intelligence.

# Search places
curl -X POST https://scrapingbot.io/api/v1/instagram \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/search/places_by_keyword",
    "params": {
      "keyword": "New York"
    }
  }'

Real-World Use Cases

Use Case 1: Build an Influencer Analytics Dashboard

Track influencer performance by combining user profiles with their posts:

// JavaScript example - Track influencers
async function analyzeInfluencer(username) {
  // Get profile info
  const profileRes = await fetch('https://scrapingbot.io/api/v1/instagram', {
    method: 'POST',
    headers: {
      'x-api-key': 'YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      endpoint: '/user/by_username',
      params: { username }
    })
  });
  
  const profile = await profileRes.json();
  
  // Get their posts
  const postsRes = await fetch('https://scrapingbot.io/api/v1/instagram', {
    method: 'POST',
    headers: {
      'x-api-key': 'YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      endpoint: '/medias/by_user_id',
      params: { user_id: profile.data.id }
    })
  });
  
  const posts = await postsRes.json();
  
  // Calculate engagement rate
  const totalLikes = posts.data.items.reduce((sum, p) => sum + p.like_count, 0);
  const avgLikes = totalLikes / posts.data.items.length;
  const engagementRate = (avgLikes / profile.data.follower_count) * 100;
  
  return {
    username,
    followers: profile.data.follower_count,
    avgLikes,
    engagementRate: engagementRate.toFixed(2) + '%'
  };
}

Use Case 2: Monitor Brand Mentions

Track whenever your brand is mentioned on Instagram:

# Python example - Monitor brand hashtags
import requests

def monitor_brand_hashtag(brand_hashtag):
    response = requests.post(
        'https://scrapingbot.io/api/v1/instagram',
        headers={'x-api-key': 'YOUR_API_KEY'},
        json={
            'endpoint': '/search/hashtags_by_keyword',
            'params': {'keyword': brand_hashtag}
        }
    )
    
    data = response.json()
    
    if data['success']:
        hashtag_data = data['data']
        print(f"#{brand_hashtag} Statistics:")
        print(f"  - Total posts: {hashtag_data['media_count']:,}")
        print(f"  - Trending: {'Yes' if hashtag_data.get('is_trending') else 'No'}")
        
        # Get recent posts with this hashtag
        # (would require additional API calls)
        
    return data

monitor_brand_hashtag('mybrand')

Use Case 3: Download Instagram Content

Create a tool to download photos and videos from Instagram posts:

// Download Instagram post
async function downloadInstagramPost(postUrl) {
  const response = await fetch('https://scrapingbot.io/api/v1/instagram', {
    method: 'POST',
    headers: {
      'x-api-key': 'YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      endpoint: '/media/by_url',
      params: { url: postUrl }
    })
  });
  
  const data = await response.json();
  
  if (data.success) {
    const media = data.data;
    
    if (media.video_url) {
      // Download video
      console.log('Video URL:', media.video_url);
      window.open(media.video_url, '_blank');
    } else if (media.image_url) {
      // Download image
      console.log('Image URL:', media.image_url);
      window.open(media.image_url, '_blank');
    }
    
    // For carousel posts, media.carousel_media contains all items
    if (media.carousel_media) {
      media.carousel_media.forEach((item, i) => {
        console.log(`Item ${i + 1}:`, item.image_url || item.video_url);
      });
    }
  }
}

Best Practices & Tips

✅ Do's

✓ Cache profile data: User profiles change infrequently—cache them to save credits
✓ Use pagination wisely: Don't fetch all posts if you only need recent ones
✓ Handle private accounts: Check if an account is private before trying to fetch posts
✓ Respect rate limits: Each request costs 5 credits—plan your usage accordingly

⚠️ Common Mistakes

⚠ Using @ in usernames: Just use "username", not "@username"
⚠ Wrong ID type: User endpoints need user_id (numeric), not username
⚠ Ignoring private accounts: API can't access private account posts without permission
⚠ Not handling deleted content: Posts and accounts can be deleted—always check response status

Getting Started

Ready to start scraping Instagram? Here's how to get up and running:

🚀 Quick Start Guide

1
Sign up for ScrapingBot — Get 1,000 free credits (20 Instagram API calls)
2
Get your API key — Available instantly in your dashboard
3
Make your first request — Try any endpoint from this guide
4
Build something awesome — Analytics dashboard, content downloader, brand monitor

Start Scraping Instagram Now →

Wrapping Up

Instagram scraping doesn't have to be complicated. With the right API, you can extract photos, videos, user profiles, comments, and more—all without dealing with login systems, rate limits, or changing endpoints.

Whether you're building an influencer analytics platform, monitoring brand mentions, or creating a content download tool, ScrapingBot's Instagram API has you covered with 15 powerful endpoints and consistent, reliable data.

P.S. Have questions or need help with a specific use case? Check out our documentation or reach out to support. We're here to help you build amazing things with Instagram data.