linkedin-spider 0.1.9

A modern LinkedIn scraping library with CLI and MCP server

linkedin-spider

Effortless Linkedin scraping with zero detection. Extract, export, and automate your Linkedin data.

Features

• Search Linkedin profiles with advanced filters (location, connection type, current company, position)
• Search and extract LinkedIn posts by keywords with comprehensive metadata
• Extract complete profile information (experience, education, skills, contact details)
• Get company details and information
• Retrieve incoming and outgoing connection requests
• Send connection requests to profiles
• Get conversations list and detailed conversation history
• Built-in anti-detection and session management

Quick Start

Installation

Choose your preferred installation method:

Option 1: pip (Recommended for general use)

# For Python library only
pip install linkedin-spider

# For CLI usage
pip install linkedin-spider[cli]

# For MCP server usage
pip install linkedin-spider[mcp]

# For all features (CLI + MCP + library)
pip install linkedin-spider[all]

Option 2: Development setup with uv

# Clone the repo
git clone https://github.com/vertexcover-io/linkedin-spider
cd linkedin-spider
# Install with uv
uv sync

[!NOTE] > Authentication Update: Linkedin has enhanced their anti-bot mechanisms, temporarily affecting cookie-based authentication. We recommend using the email/password authentication method for reliable access. We are actively working on restoring full cookie authentication support.

Different ways to use it

1. Python Library

Perfect for integration into your existing Python applications:

from linkedin_spider import LinkedinSpider, ScraperConfig

config = ScraperConfig(headless=True, page_load_timeout=30)

# Authenticate (use either email/password or cookie).
# Authentication is mostly done once and the session is saved in the chrome profile
scraper = LinkedinSpider(
    email="[e-mail]",
    password="your_password",
    config=config
)

# Search for profiles
results = scraper.search_profiles("software engineer", max_results=10)

Output sample:

[
  {
    "name": "John Doe",
    "title": "Senior Software Engineer at Google",
    "location": "San Francisco, CA",
    "profile_url": "https://linkedin.com/in/johndoe",
    "connections": "500+"
  },
  {
    "name": "Jane Smith",
    "title": "Software Engineer at Microsoft",
    "location": "Seattle, WA",
    "profile_url": "https://linkedin.com/in/janesmith",
    "connections": "200+"
  }
]

# Search for posts by keywords
posts = scraper.search_posts("artificial intelligence", max_results=10, scroll_pause=2.0)

Output sample:

[
  {
    "author_name": "John Doe",
    "author_headline": "AI Research Scientist at OpenAI",
    "author_profile_url": "https://linkedin.com/in/johndoe",
    "connection_degree": "2nd",
    "post_time": "2024-01-15T14:30:00+00:00",
    "post_text": "Excited to share our latest research on [large language models](https://example.com/paper)...",
    "hashtags": ["#AI", "#MachineLearning", "#Research"],
    "links": ["https://example.com/paper"],
    "post_url": "https://linkedin.com/feed/update/urn:li:activity:123456789",
    "media_urls": ["https://media.licdn.com/dms/image/..."],
    "likes_count": 1247,
    "comments_count": 89,
    "reposts_count": 234,
    "comments": [
      {
        "author_name": "Jane Smith",
        "author_profile_url": "https://linkedin.com/in/janesmith",
        "comment_text": "Great insights! Looking forward to reading the full paper.",
        "comment_time": "2024-01-15T15:45:00+00:00",
        "reactions_count": 12
      }
    ]
  }
]

# Scrape individual profile
profile = scraper.scrape_profile("https://linkedin.com/in/someone")

Output sample:

{
  "name": "John Doe",
  "title": "Senior Software Engineer",
  "location": "San Francisco, CA",
  "about": "Passionate software engineer with 8+ years of experience...",
  "experience": [
    {
      "title": "Senior Software Engineer",
      "company": "Google",
      "duration": "2021 - Present",
      "description": "Leading backend development for search infrastructure..."
    }
  ],
  "education": [
    {
      "school": "Stanford University",
      "degree": "BS Computer Science",
      "years": "2013 - 2017"
    }
  ],
  "skills": ["Python", "Java", "Kubernetes", "AWS"]
}

# Scrape company information
company = scraper.scrape_company("https://linkedin.com/company/tech-corp")

Output sample:

{
  "name": "TechCorp Inc",
  "industry": "Software Development",
  "company_size": "1,001-5,000 employees",
  "headquarters": "San Francisco, CA",
  "founded": "2010",
  "specialties": ["Cloud Computing", "AI/ML", "Data Analytics"],
  "description": "Leading technology company focused on enterprise solutions...",
  "website": "https://techcorp.com",
  "follower_count": "45,230"
}

# Don't forget to clean up
scraper.close()

For more examples : examples

2. Command Line Interface

Great for quick data extraction and scripting:

# If installed via pip
# Search for profiles
linkedin-spider-cli search -q "product manager" -n 10 -o results.json --email [e-mail] --password yourpassword

# Search for posts
linkedin-spider-cli search-posts -k "artificial intelligence" -n 10 -s 2.0 -o posts.json --email [e-mail] --password yourpassword

# Scrape individual profile
linkedin-spider-cli profile -u "https://linkedin.com/in/johndoe" -o profile.json --email [e-mail] --password yourpassword

# Scrape company
linkedin-spider-cli company -u "https://linkedin.com/company/openai" -o company.json --email [e-mail] --password yourpassword

# Get connection requests
linkedin-spider-cli connections -n 20 -o connections.json --email [e-mail] --password yourpassword

# If using development setup
# Search for profiles
uv run linkedin-spider-cli search -q "product manager" -n 10 -o results.json --email [e-mail] --password yourpassword

# Search for posts
uv run linkedin-spider-cli search-posts -k "artificial intelligence" -n 10 -s 2.0 -o posts.json --email [e-mail] --password yourpassword

# Scrape individual profile
uv run linkedin-spider-cli profile -u "https://linkedin.com/in/johndoe" -o profile.json --email [e-mail] --password yourpassword

# Scrape company
uv run linkedin-spider-cli company -u "https://linkedin.com/company/openai" -o company.json --email [e-mail] --password yourpassword

# Get connection requests
uv run linkedin-spider-cli connections -n 20 -o connections.json --email [e-mail] --password yourpassword

Note: You typically only need to provide --email and --password once. The CLI saves your authentication session and will reuse it for subsequent commands until the session expires (usually after several hours or days). You can also set LINKEDIN_EMAIL and LINKEDIN_PASSWORD environment variables to avoid typing them repeatedly.

3. MCP Server

Set up environment variables in .env file:

# Authentication (choose one method)
LINKEDIN_EMAIL=[e-mail]
LINKEDIN_PASSWORD=your_password
# OR
LINKEDIN_COOKIE=your_li_at_cookie_value

# Configuration
HEADLESS=true

# Transport (optional, defaults to stdio)
TRANSPORT=sse
HOST=127.0.0.1
PORT=8000

Start the MCP server:

# If installed via pip
# Show available transport options
linkedin-spider-mcp

# Start with specific transport
linkedin-spider-mcp serve sse --email [e-mail] --password yourpassword
linkedin-spider-mcp serve http --host 0.0.0.0 --port 9000 --email [e-mail] --password yourpassword
linkedin-spider-mcp serve stdio --email [e-mail] --password yourpassword

# Or use environment variables
TRANSPORT=sse linkedin-spider-mcp serve

# If using development setup
# Show available transport options
uv run linkedin-spider-mcp

# Start with specific transport
uv run linkedin-spider-mcp serve sse --email [e-mail] --password yourpassword
uv run linkedin-spider-mcp serve http --host 0.0.0.0 --port 9000 --email [e-mail] --password yourpassword
uv run linkedin-spider-mcp serve stdio --email [e-mail] --password yourpassword

# Or use environment variables
TRANSPORT=sse uv run linkedin-spider-mcp serve

Claude Code Integration

# Add to Claude Code
claude mcp add linkedin-spider --transport sse <server-url>
# Example server URL format: http://localhost:8080/sse

Claude Desktop Integration

Add to your Claude Desktop configuration file:

Windows: %APPDATA%\Claude\claude_desktop_config.json macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Linux: ~/.config/Claude/claude_desktop_config.json

Option 1: Docker (Recommended)

The Docker approach provides reliable, isolated execution with all dependencies included.

First, build the Docker image:

# Build the stdio server image
docker build -f Dockerfile.stdio -t linkedin-mcp-stdio .

Then add this to your Claude Desktop configuration:

{
  "mcpServers": {
    "linkedin-spider": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "-e",
        "LINKEDIN_EMAIL=[e-mail]",
        "-e",
        "LINKEDIN_PASSWORD=your_password",
        "-e",
        "HEADLESS=true",
        "-e",
        "TRANSPORT=stdio",
        "linkedin-mcp-stdio"
      ]
    }
  }
}

Docker Development & Testing

For development and testing with Docker, you can use a single image with different transport configurations:

Build the Docker Image

# Build once for all transport types
docker build -t linkedin-mcp .

Run with Different Transports

SSE Server

docker run -p 8000:8000 -e TRANSPORT=sse --env-file .env linkedin-mcp

HTTP Server

docker run -p 8000:8000 -e TRANSPORT=http --env-file .env linkedin-mcp

STDIO Server

docker run --rm -i -e TRANSPORT=stdio --env-file .env linkedin-mcp

Authentication Methods

Method 1: Linkedin Cookie

1. Login to Linkedin in your browser
2. Open Developer Tools (F12)
3. Go to Application/Storage → Cookies → linkedin.com
4. Copy the li_at cookie value
5. Use it in your code:

scraper = LinkedinSpider(li_at_cookie="your_cookie_value")

Method 2: Email & Password (Recommended)

scraper = LinkedinSpider(
    email="[e-mail]",
    password="your_password"
)

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Disclaimer

This tool is for personal use only. Please:

• Respect Linkedin's Terms of Service
• Use reasonable rate limits
• Don't spam or harass users
• Be responsible with the data you collect

---

Ready to extract Linkedin data like a pro? Star this repo and start scraping!