Merge branch 'main' into chore/ssrf-config

Signed-off-by: -LAN- <laipz8200@outlook.com>

.github/workflows/api-tests.yml

@@ -67,6 +67,9 @@ jobs:
           cp docker/.env.example docker/.env
           cp docker/middleware.env.example docker/middleware.env
 
+      - name: Setup SSRF Proxy for Testing
+        run: sh docker/ssrf_proxy/setup-testing.sh
+
       - name: Expose Service Ports
         run: sh .github/workflows/expose_service_ports.sh
 

.gitignore

@@ -228,10 +228,14 @@ web/public/fallback-*.js
 api/.env.backup
 /clickzetta
 
+# SSRF Proxy - ignore the conf.d directory that's created for testing/local overrides
+docker/ssrf_proxy/conf.d/
+
 # Benchmark
 scripts/stress-test/setup/config/
 scripts/stress-test/reports/
 
 # mcp
 .playwright-mcp/
-.serena/
+.serena/
+

api/tests/integration_tests/ssrf_proxy/TEST_CASES_README.md

@@ -0,0 +1,175 @@
+# SSRF Proxy Test Cases
+
+## Overview
+
+The SSRF proxy test suite uses YAML files to define test cases, making them easier to maintain and extend without modifying code. These tests validate the SSRF proxy configuration in `docker/ssrf_proxy/`.
+
+## Location
+
+These tests are located in `api/tests/integration_tests/ssrf_proxy/` because they require the Python environment from the API project.
+
+## Usage
+
+### Basic Testing
+
+From the `api/` directory:
+
+```bash
+uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py
+```
+
+Or from the repository root:
+
+```bash
+cd api && uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py
+```
+
+### List Available Tests
+
+View all test cases without running them:
+
+```bash
+uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py --list-tests
+```
+
+### Use Custom Test File
+
+Run tests from a specific YAML file:
+
+```bash
+uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py --test-file test_cases_extended.yaml
+```
+
+### Development Mode Testing
+
+**WARNING: Development mode DISABLES all SSRF protections! Only use in development environments!**
+
+Test the development mode configuration (used by docker-compose.middleware.yaml):
+
+```bash
+uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py --dev-mode
+```
+
+Development mode:
+
+- Mounts `conf.d.dev/` configuration that allows ALL requests
+- Uses `test_cases_dev_mode.yaml` by default (all tests expect ALLOW)
+- Verifies that private networks, cloud metadata, and non-standard ports are accessible
+- Should NEVER be used in production environments
+
+### Command Line Options
+
+- `--host HOST`: Proxy host (default: localhost)
+- `--port PORT`: Proxy port (default: 3128)
+- `--no-container`: Don't start container (assume proxy is already running)
+- `--save-results`: Save test results to JSON file
+- `--test-file FILE`: Path to YAML file containing test cases
+- `--list-tests`: List all test cases without running them
+- `--dev-mode`: Run in development mode (DISABLES all SSRF protections - DO NOT use in production!)
+
+## YAML Test Case Format
+
+Test cases are organized by categories in YAML files:
+
+```yaml
+test_categories:
+  category_key:
+    name: "Category Display Name"
+    description: "Category description"
+    test_cases:
+      - name: "Test Case Name"
+        url: "http://example.com"
+        expected_blocked: false  # true if should be blocked, false if allowed
+        description: "Optional test description"
+```
+
+## Available Test Files
+
+1. **test_cases.yaml** - Standard test suite with essential test cases (default)
+1. **test_cases_extended.yaml** - Extended test suite with additional edge cases and scenarios
+1. **test_cases_dev_mode.yaml** - Development mode test suite (all requests should be allowed)
+
+All files are located in `api/tests/integration_tests/ssrf_proxy/`
+
+## Categories
+
+### Standard Categories
+
+- **Private Networks**: Tests for blocking private IP ranges and loopback addresses
+- **Cloud Metadata**: Tests for blocking cloud provider metadata endpoints
+- **Public Internet**: Tests for allowing legitimate public internet access
+- **Port Restrictions**: Tests for port-based access control
+
+### Extended Categories (in test_cases_extended.yaml)
+
+- **IPv6 Tests**: Tests for IPv6 address handling
+- **Special Cases**: Edge cases like decimal/octal/hex IP notation
+
+## Adding New Test Cases
+
+1. Edit the YAML file (or create a new one)
+1. Add test cases under appropriate categories
+1. Run with `--test-file` option if using a custom file
+
+Example:
+
+```yaml
+test_categories:
+  custom_tests:
+    name: "Custom Tests"
+    description: "My custom test cases"
+    test_cases:
+      - name: "Custom Test 1"
+        url: "http://test.example.com"
+        expected_blocked: false
+        description: "Testing custom domain"
+```
+
+## What Gets Tested
+
+The tests validate the SSRF proxy configuration files in `docker/ssrf_proxy/`:
+
+- `squid.conf.template` - Squid proxy configuration
+- `docker-entrypoint.sh` - Container initialization script
+- `conf.d/` - Additional configuration files (if present)
+- `conf.d.dev/` - Development mode configuration (when using --dev-mode)
+
+## Development Mode Configuration
+
+Development mode provides a zero-configuration environment for local development:
+
+- Mounts `conf.d.dev/` instead of `conf.d/`
+- Allows ALL requests including private networks and cloud metadata
+- Enables access to any port
+- Disables all SSRF protections
+
+### Using Development Mode with Docker Compose
+
+From the main Dify repository root:
+
+```bash
+# Use the development overlay
+docker-compose -f docker-compose.middleware.yaml -f docker/ssrf_proxy/docker-compose.dev.yaml up ssrf_proxy
+```
+
+Or manually mount the development configuration:
+
+```bash
+docker run -d \
+  --name ssrf-proxy-dev \
+  -p 3128:3128 \
+  -v ./docker/ssrf_proxy/conf.d.dev:/etc/squid/conf.d:ro \
+  # ... other volumes
+  ubuntu/squid:latest
+```
+
+**CRITICAL**: Never use this configuration in production!
+
+## Benefits
+
+- **Maintainability**: Test cases can be updated without code changes
+- **Extensibility**: Easy to add new test cases or categories
+- **Clarity**: YAML format is human-readable and self-documenting
+- **Flexibility**: Multiple test files for different scenarios
+- **Fallback**: Code includes default test cases if YAML loading fails
+- **Integration**: Properly integrated with the API project's Python environment

api/tests/integration_tests/ssrf_proxy/__init__.py

@@ -0,0 +1 @@
+"""SSRF Proxy Integration Tests"""

api/tests/integration_tests/ssrf_proxy/test_cases.yaml

@@ -0,0 +1,129 @@
+# SSRF Proxy Test Cases Configuration
+# This file defines all test cases for the SSRF proxy
+# Each test case validates whether the proxy correctly blocks or allows requests
+
+test_categories:
+  private_networks:
+    name: "Private Networks"
+    description: "Tests for blocking private IP ranges and loopback addresses"
+    test_cases:
+      - name: "Loopback (127.0.0.1)"
+        url: "http://127.0.0.1"
+        expected_blocked: true
+        description: "IPv4 loopback address"
+      
+      - name: "Localhost"
+        url: "http://localhost"
+        expected_blocked: true
+        description: "Localhost hostname"
+      
+      - name: "Private 10.x.x.x"
+        url: "http://10.0.0.1"
+        expected_blocked: true
+        description: "RFC 1918 private network"
+      
+      - name: "Private 172.16.x.x"
+        url: "http://172.16.0.1"
+        expected_blocked: true
+        description: "RFC 1918 private network"
+      
+      - name: "Private 192.168.x.x"
+        url: "http://192.168.1.1"
+        expected_blocked: true
+        description: "RFC 1918 private network"
+      
+      - name: "Link-local"
+        url: "http://169.254.1.1"
+        expected_blocked: true
+        description: "Link-local address"
+      
+      - name: "This network"
+        url: "http://0.0.0.0"
+        expected_blocked: true
+        description: "'This' network address"
+
+  cloud_metadata:
+    name: "Cloud Metadata"
+    description: "Tests for blocking cloud provider metadata endpoints"
+    test_cases:
+      - name: "AWS Metadata"
+        url: "http://169.254.169.254/latest/meta-data/"
+        expected_blocked: true
+        description: "AWS EC2 metadata endpoint"
+      
+      - name: "Azure Metadata"
+        url: "http://169.254.169.254/metadata/instance"
+        expected_blocked: true
+        description: "Azure metadata endpoint"
+      
+      # Note: metadata.google.internal is not included as it may resolve to public IPs
+
+  public_internet:
+    name: "Public Internet"
+    description: "Tests for allowing legitimate public internet access"
+    test_cases:
+      - name: "Example.com"
+        url: "http://example.com"
+        expected_blocked: false
+        description: "Public website"
+      
+      - name: "Google HTTPS"
+        url: "https://www.google.com"
+        expected_blocked: false
+        description: "HTTPS public website"
+      
+      - name: "HTTPBin API"
+        url: "http://httpbin.org/get"
+        expected_blocked: false
+        description: "Public API endpoint"
+      
+      - name: "GitHub API"
+        url: "https://api.github.com"
+        expected_blocked: false
+        description: "Public API over HTTPS"
+
+  port_restrictions:
+    name: "Port Restrictions"
+    description: "Tests for port-based access control"
+    test_cases:
+      - name: "HTTP Port 80"
+        url: "http://example.com:80"
+        expected_blocked: false
+        description: "Standard HTTP port"
+      
+      - name: "HTTPS Port 443"
+        url: "http://example.com:443"
+        expected_blocked: false
+        description: "Standard HTTPS port"
+      
+      - name: "Port 8080"
+        url: "http://example.com:8080"
+        expected_blocked: true
+        description: "Non-standard port"
+      
+      - name: "Port 3000"
+        url: "http://example.com:3000"
+        expected_blocked: true
+        description: "Development port"
+      
+      - name: "SSH Port 22"
+        url: "http://example.com:22"
+        expected_blocked: true
+        description: "SSH port"
+      
+      - name: "MySQL Port 3306"
+        url: "http://example.com:3306"
+        expected_blocked: true
+        description: "Database port"
+
+# Additional test configurations can be added here
+# For example:
+# 
+# ipv6_tests:
+#   name: "IPv6 Tests"
+#   description: "Tests for IPv6 address handling"
+#   test_cases:
+#     - name: "IPv6 Loopback"
+#       url: "http://[::1]"
+#       expected_blocked: true
+#       description: "IPv6 loopback address"

api/tests/integration_tests/ssrf_proxy/test_cases_dev_mode.yaml

@@ -0,0 +1,168 @@
+# Development Mode Test Cases for SSRF Proxy
+# These test cases verify that development mode correctly disables all SSRF protections
+# WARNING: All requests should be ALLOWED in development mode
+
+test_categories:
+  private_networks:
+    name: "Private Networks (Dev Mode)"
+    description: "In dev mode, private networks should be ALLOWED"
+    test_cases:
+      - name: "Loopback (127.0.0.1)"
+        url: "http://127.0.0.1"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "IPv4 loopback - normally blocked, allowed in dev mode"
+      
+      - name: "Localhost"
+        url: "http://localhost"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Localhost hostname - normally blocked, allowed in dev mode"
+      
+      - name: "Private 10.x.x.x"
+        url: "http://10.0.0.1"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "RFC 1918 private network - normally blocked, allowed in dev mode"
+      
+      - name: "Private 172.16.x.x"
+        url: "http://172.16.0.1"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "RFC 1918 private network - normally blocked, allowed in dev mode"
+      
+      - name: "Private 192.168.x.x"
+        url: "http://192.168.1.1"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "RFC 1918 private network - normally blocked, allowed in dev mode"
+      
+      - name: "Link-local"
+        url: "http://169.254.1.1"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Link-local address - normally blocked, allowed in dev mode"
+      
+      - name: "This network"
+        url: "http://0.0.0.0"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "'This' network address - normally blocked, allowed in dev mode"
+
+  cloud_metadata:
+    name: "Cloud Metadata (Dev Mode)"
+    description: "In dev mode, cloud metadata endpoints should be ALLOWED"
+    test_cases:
+      - name: "AWS Metadata"
+        url: "http://169.254.169.254/latest/meta-data/"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "AWS EC2 metadata - normally blocked, allowed in dev mode"
+      
+      - name: "Azure Metadata"
+        url: "http://169.254.169.254/metadata/instance"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Azure metadata - normally blocked, allowed in dev mode"
+
+  non_standard_ports:
+    name: "Non-Standard Ports (Dev Mode)"
+    description: "In dev mode, all ports should be ALLOWED"
+    test_cases:
+      - name: "Port 8080"
+        url: "http://example.com:8080"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Alternative HTTP port - normally blocked, allowed in dev mode"
+      
+      - name: "Port 3000"
+        url: "http://example.com:3000"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Node.js development port - normally blocked, allowed in dev mode"
+      
+      - name: "SSH Port 22"
+        url: "http://example.com:22"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "SSH port - normally blocked, allowed in dev mode"
+      
+      - name: "Database Port 3306"
+        url: "http://example.com:3306"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "MySQL port - normally blocked, allowed in dev mode"
+      
+      - name: "Database Port 5432"
+        url: "http://example.com:5432"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "PostgreSQL port - normally blocked, allowed in dev mode"
+      
+      - name: "Redis Port 6379"
+        url: "http://example.com:6379"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Redis port - normally blocked, allowed in dev mode"
+      
+      - name: "MongoDB Port 27017"
+        url: "http://example.com:27017"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "MongoDB port - normally blocked, allowed in dev mode"
+      
+      - name: "High Port 12345"
+        url: "http://example.com:12345"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Random high port - normally blocked, allowed in dev mode"
+
+  localhost_ports:
+    name: "Localhost with Various Ports (Dev Mode)"
+    description: "In dev mode, localhost with any port should be ALLOWED"
+    test_cases:
+      - name: "Localhost:8080"
+        url: "http://localhost:8080"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Localhost with port 8080 - normally blocked, allowed in dev mode"
+      
+      - name: "Localhost:3000"
+        url: "http://localhost:3000"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Localhost with port 3000 - normally blocked, allowed in dev mode"
+      
+      - name: "127.0.0.1:9200"
+        url: "http://127.0.0.1:9200"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Loopback with Elasticsearch port - normally blocked, allowed in dev mode"
+      
+      - name: "127.0.0.1:5001"
+        url: "http://127.0.0.1:5001"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "Loopback with API port - normally blocked, allowed in dev mode"
+
+  public_internet:
+    name: "Public Internet (Dev Mode)"
+    description: "Public internet should still work in dev mode"
+    test_cases:
+      - name: "Example.com"
+        url: "http://example.com"
+        expected_blocked: false
+        description: "Public website - always allowed"
+      
+      - name: "Google HTTPS"
+        url: "https://www.google.com"
+        expected_blocked: false
+        description: "HTTPS public website - always allowed"
+      
+      - name: "GitHub API"
+        url: "https://api.github.com"
+        expected_blocked: false
+        description: "Public API over HTTPS - always allowed"
+
+  special_cases:
+    name: "Special Cases (Dev Mode)"
+    description: "Edge cases that should all be allowed in dev mode"
+    test_cases:
+      - name: "Decimal IP notation"
+        url: "http://2130706433"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "127.0.0.1 in decimal - normally blocked, allowed in dev mode"
+      
+      - name: "Private network in subdomain"
+        url: "http://192-168-1-1.example.com"
+        expected_blocked: false
+        description: "Domain that looks like private IP - always allowed as it resolves externally"
+      
+      - name: "IPv6 Loopback"
+        url: "http://[::1]"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "IPv6 loopback - normally blocked, allowed in dev mode"
+      
+      - name: "IPv6 Link-local"
+        url: "http://[fe80::1]"
+        expected_blocked: false  # ALLOWED in dev mode
+        description: "IPv6 link-local - normally blocked, allowed in dev mode"

api/tests/integration_tests/ssrf_proxy/test_cases_extended.yaml

@@ -0,0 +1,219 @@
+# Extended SSRF Proxy Test Cases Configuration
+# This file contains additional test cases for comprehensive testing
+# Use with: python test_ssrf_proxy.py --test-file test_cases_extended.yaml
+
+test_categories:
+  # Standard test cases
+  private_networks:
+    name: "Private Networks"
+    description: "Tests for blocking private IP ranges and loopback addresses"
+    test_cases:
+      - name: "Loopback (127.0.0.1)"
+        url: "http://127.0.0.1"
+        expected_blocked: true
+        description: "IPv4 loopback address"
+      
+      - name: "Localhost"
+        url: "http://localhost"
+        expected_blocked: true
+        description: "Localhost hostname"
+      
+      - name: "Private 10.x.x.x"
+        url: "http://10.0.0.1"
+        expected_blocked: true
+        description: "RFC 1918 private network"
+      
+      - name: "Private 172.16.x.x"
+        url: "http://172.16.0.1"
+        expected_blocked: true
+        description: "RFC 1918 private network"
+      
+      - name: "Private 192.168.x.x"
+        url: "http://192.168.1.1"
+        expected_blocked: true
+        description: "RFC 1918 private network"
+      
+      - name: "Link-local"
+        url: "http://169.254.1.1"
+        expected_blocked: true
+        description: "Link-local address"
+      
+      - name: "This network"
+        url: "http://0.0.0.0"
+        expected_blocked: true
+        description: "'This' network address"
+
+  cloud_metadata:
+    name: "Cloud Metadata"
+    description: "Tests for blocking cloud provider metadata endpoints"
+    test_cases:
+      - name: "AWS Metadata"
+        url: "http://169.254.169.254/latest/meta-data/"
+        expected_blocked: true
+        description: "AWS EC2 metadata endpoint"
+      
+      - name: "Azure Metadata"
+        url: "http://169.254.169.254/metadata/instance"
+        expected_blocked: true
+        description: "Azure metadata endpoint"
+      
+      - name: "DigitalOcean Metadata"
+        url: "http://169.254.169.254/metadata/v1"
+        expected_blocked: true
+        description: "DigitalOcean metadata endpoint"
+      
+      - name: "Oracle Cloud Metadata"
+        url: "http://169.254.169.254/opc/v1"
+        expected_blocked: true
+        description: "Oracle Cloud metadata endpoint"
+
+  public_internet:
+    name: "Public Internet"
+    description: "Tests for allowing legitimate public internet access"
+    test_cases:
+      - name: "Example.com"
+        url: "http://example.com"
+        expected_blocked: false
+        description: "Public website"
+      
+      - name: "Google HTTPS"
+        url: "https://www.google.com"
+        expected_blocked: false
+        description: "HTTPS public website"
+      
+      - name: "HTTPBin API"
+        url: "http://httpbin.org/get"
+        expected_blocked: false
+        description: "Public API endpoint"
+      
+      - name: "GitHub API"
+        url: "https://api.github.com"
+        expected_blocked: false
+        description: "Public API over HTTPS"
+      
+      - name: "OpenAI API"
+        url: "https://api.openai.com"
+        expected_blocked: false
+        description: "OpenAI API endpoint"
+      
+      - name: "Anthropic API"
+        url: "https://api.anthropic.com"
+        expected_blocked: false
+        description: "Anthropic API endpoint"
+
+  port_restrictions:
+    name: "Port Restrictions"
+    description: "Tests for port-based access control"
+    test_cases:
+      - name: "HTTP Port 80"
+        url: "http://example.com:80"
+        expected_blocked: false
+        description: "Standard HTTP port"
+      
+      - name: "HTTPS Port 443"
+        url: "http://example.com:443"
+        expected_blocked: false
+        description: "Standard HTTPS port"
+      
+      - name: "Port 8080"
+        url: "http://example.com:8080"
+        expected_blocked: true
+        description: "Alternative HTTP port"
+      
+      - name: "Port 3000"
+        url: "http://example.com:3000"
+        expected_blocked: true
+        description: "Node.js development port"
+      
+      - name: "SSH Port 22"
+        url: "http://example.com:22"
+        expected_blocked: true
+        description: "SSH port"
+      
+      - name: "Telnet Port 23"
+        url: "http://example.com:23"
+        expected_blocked: true
+        description: "Telnet port"
+      
+      - name: "SMTP Port 25"
+        url: "http://example.com:25"
+        expected_blocked: true
+        description: "SMTP mail port"
+      
+      - name: "MySQL Port 3306"
+        url: "http://example.com:3306"
+        expected_blocked: true
+        description: "MySQL database port"
+      
+      - name: "PostgreSQL Port 5432"
+        url: "http://example.com:5432"
+        expected_blocked: true
+        description: "PostgreSQL database port"
+      
+      - name: "Redis Port 6379"
+        url: "http://example.com:6379"
+        expected_blocked: true
+        description: "Redis port"
+      
+      - name: "MongoDB Port 27017"
+        url: "http://example.com:27017"
+        expected_blocked: true
+        description: "MongoDB port"
+
+  ipv6_tests:
+    name: "IPv6 Tests"
+    description: "Tests for IPv6 address handling"
+    test_cases:
+      - name: "IPv6 Loopback"
+        url: "http://[::1]"
+        expected_blocked: true
+        description: "IPv6 loopback address"
+      
+      - name: "IPv6 All zeros"
+        url: "http://[::]"
+        expected_blocked: true
+        description: "IPv6 all zeros address"
+      
+      - name: "IPv6 Link-local"
+        url: "http://[fe80::1]"
+        expected_blocked: true
+        description: "IPv6 link-local address"
+      
+      - name: "IPv6 Unique local"
+        url: "http://[fc00::1]"
+        expected_blocked: true
+        description: "IPv6 unique local address"
+
+  special_cases:
+    name: "Special Cases"
+    description: "Edge cases and special scenarios"
+    test_cases:
+      - name: "Decimal IP notation"
+        url: "http://2130706433"
+        expected_blocked: true
+        description: "127.0.0.1 in decimal notation"
+      
+      - name: "Octal IP notation"
+        url: "http://0177.0.0.1"
+        expected_blocked: true
+        description: "127.0.0.1 with octal notation"
+      
+      - name: "Hex IP notation"
+        url: "http://0x7f.0.0.1"
+        expected_blocked: true
+        description: "127.0.0.1 with hex notation"
+      
+      - name: "Mixed notation"
+        url: "http://0x7f.0.0.0x1"
+        expected_blocked: true
+        description: "127.0.0.1 with mixed hex notation"
+      
+      - name: "Localhost with port"
+        url: "http://localhost:8080"
+        expected_blocked: true
+        description: "Localhost with non-standard port"
+      
+      - name: "Domain with private IP"
+        url: "http://192-168-1-1.example.com"
+        expected_blocked: false
+        description: "Domain that looks like private IP (should resolve)"

api/tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py

@@ -0,0 +1,482 @@
+#!/usr/bin/env python3
+"""
+SSRF Proxy Test Suite
+
+This script tests the SSRF proxy configuration to ensure it blocks
+private networks while allowing public internet access.
+"""
+
+import argparse
+import json
+import os
+import subprocess
+import sys
+import time
+import urllib.error
+import urllib.request
+from dataclasses import dataclass
+from enum import Enum
+from typing import final
+
+import yaml
+
+
+# Color codes for terminal output
+class Colors:
+    RED: str = "\033[0;31m"
+    GREEN: str = "\033[0;32m"
+    YELLOW: str = "\033[1;33m"
+    BLUE: str = "\033[0;34m"
+    NC: str = "\033[0m"  # No Color
+
+
+class TestResult(Enum):
+    PASSED = "passed"
+    FAILED = "failed"
+    SKIPPED = "skipped"
+
+
+@dataclass
+class TestCase:
+    name: str
+    url: str
+    expected_blocked: bool
+    category: str
+    description: str = ""
+
+
+@final
+class SSRFProxyTester:
+    def __init__(
+        self,
+        proxy_host: str = "localhost",
+        proxy_port: int = 3128,
+        test_file: str | None = None,
+        dev_mode: bool = False,
+    ):
+        self.proxy_host = proxy_host
+        self.proxy_port = proxy_port
+        self.proxy_url = f"http://{proxy_host}:{proxy_port}"
+        self.container_name = "ssrf-proxy-test-dev" if dev_mode else "ssrf-proxy-test"
+        self.image = "ubuntu/squid:latest"
+        self.results: list[dict[str, object]] = []
+        self.dev_mode = dev_mode
+        # Use dev mode test cases by default when in dev mode
+        if dev_mode and test_file is None:
+            self.test_file = "test_cases_dev_mode.yaml"
+        else:
+            self.test_file = test_file or "test_cases.yaml"
+
+    def start_proxy_container(self) -> bool:
+        """Start the SSRF proxy container"""
+        mode_str = " (DEVELOPMENT MODE)" if self.dev_mode else ""
+        print(f"{Colors.YELLOW}Starting SSRF proxy container{mode_str}...{Colors.NC}")
+        if self.dev_mode:
+            print(f"{Colors.RED}WARNING: Development mode DISABLES all SSRF protections!{Colors.NC}")
+
+        # Stop and remove existing container if exists
+        _ = subprocess.run(["docker", "stop", self.container_name], capture_output=True, text=True)
+        _ = subprocess.run(["docker", "rm", self.container_name], capture_output=True, text=True)
+
+        # Get directories for mounting config files
+        script_dir = os.path.dirname(os.path.abspath(__file__))
+        # Docker config files are in docker/ssrf_proxy relative to project root
+        project_root = os.path.abspath(os.path.join(script_dir, "..", "..", "..", ".."))
+        docker_config_dir = os.path.join(project_root, "docker", "ssrf_proxy")
+
+        # Choose configuration template based on mode
+        if self.dev_mode:
+            config_template = "squid.conf.dev.template"
+        else:
+            config_template = "squid.conf.template"
+
+        # Start container
+        cmd = [
+            "docker",
+            "run",
+            "-d",
+            "--name",
+            self.container_name,
+            "-p",
+            f"{self.proxy_port}:{self.proxy_port}",
+            "-p",
+            "8194:8194",
+            "-v",
+            f"{docker_config_dir}/{config_template}:/etc/squid/squid.conf.template:ro",
+            "-v",
+            f"{docker_config_dir}/docker-entrypoint.sh:/docker-entrypoint-mount.sh:ro",
+            "-e",
+            f"HTTP_PORT={self.proxy_port}",
+            "-e",
+            "COREDUMP_DIR=/var/spool/squid",
+            "-e",
+            "REVERSE_PROXY_PORT=8194",
+            "-e",
+            "SANDBOX_HOST=sandbox",
+            "-e",
+            "SANDBOX_PORT=8194",
+            "--entrypoint",
+            "sh",
+            self.image,
+            "-c",
+            "cp /docker-entrypoint-mount.sh /docker-entrypoint.sh && sed -i 's/\\r$//' /docker-entrypoint.sh && chmod +x /docker-entrypoint.sh && /docker-entrypoint.sh",  # noqa: E501
+        ]
+
+        # Mount configuration directory (only in normal mode)
+        # In dev mode, the dev template already allows everything
+        if not self.dev_mode:
+            # Normal mode: mount regular conf.d if it exists
+            conf_d_path = f"{docker_config_dir}/conf.d"
+            if os.path.exists(conf_d_path) and os.listdir(conf_d_path):
+                cmd.insert(-3, "-v")
+                cmd.insert(-3, f"{conf_d_path}:/etc/squid/conf.d:ro")
+        else:
+            print(f"{Colors.YELLOW}Using development mode configuration (all SSRF protections disabled){Colors.NC}")
+
+        result = subprocess.run(cmd, capture_output=True, text=True)
+
+        if result.returncode != 0:
+            print(f"{Colors.RED}Failed to start container: {result.stderr}{Colors.NC}")
+            return False
+
+        # Wait for proxy to start
+        print(f"{Colors.YELLOW}Waiting for proxy to start...{Colors.NC}")
+        time.sleep(5)
+
+        # Check if container is running
+        result = subprocess.run(
+            ["docker", "ps", "--filter", f"name={self.container_name}"],
+            capture_output=True,
+            text=True,
+        )
+
+        if self.container_name not in result.stdout:
+            print(f"{Colors.RED}Container failed to start!{Colors.NC}")
+            logs = subprocess.run(["docker", "logs", self.container_name], capture_output=True, text=True)
+            print(logs.stdout)
+            return False
+
+        print(f"{Colors.GREEN}Proxy started successfully!{Colors.NC}\n")
+        return True
+
+    def stop_proxy_container(self):
+        """Stop and remove the proxy container"""
+        _ = subprocess.run(["docker", "stop", self.container_name], capture_output=True, text=True)
+        _ = subprocess.run(["docker", "rm", self.container_name], capture_output=True, text=True)
+
+    def test_url(self, test_case: TestCase) -> TestResult:
+        """Test a single URL through the proxy"""
+        # Configure proxy for urllib
+        proxy_handler = urllib.request.ProxyHandler({"http": self.proxy_url, "https": self.proxy_url})
+        opener = urllib.request.build_opener(proxy_handler)
+
+        try:
+            # Make request through proxy
+            request = urllib.request.Request(test_case.url)
+            with opener.open(request, timeout=5):
+                # If we got a response, the request was allowed
+                is_blocked = False
+
+        except urllib.error.HTTPError as e:
+            # HTTP errors like 403 from proxy mean blocked
+            if e.code in [403, 407]:
+                is_blocked = True
+            else:
+                # Other HTTP errors mean the request went through
+                is_blocked = False
+        except (urllib.error.URLError, OSError, TimeoutError) as e:
+            # In dev mode, connection errors to 169.254.x.x addresses are expected
+            # These addresses don't exist locally, so timeout is normal
+            # The proxy allowed the request, but the destination is unreachable
+            if self.dev_mode and "169.254" in test_case.url:
+                # In dev mode, if we're testing 169.254.x.x addresses,
+                # a timeout means the proxy allowed it (not blocked)
+                is_blocked = False
+            else:
+                # In normal mode, or for other addresses, connection errors mean blocked
+                is_blocked = True
+        except Exception as e:
+            # Unexpected error
+            print(f"{Colors.YELLOW}Warning: Unexpected error testing {test_case.url}: {e}{Colors.NC}")
+            return TestResult.SKIPPED
+
+        # Check if result matches expectation
+        if is_blocked == test_case.expected_blocked:
+            return TestResult.PASSED
+        else:
+            return TestResult.FAILED
+
+    def run_test(self, test_case: TestCase):
+        """Run a single test and record result"""
+        result = self.test_url(test_case)
+
+        # Print result
+        if result == TestResult.PASSED:
+            symbol = f"{Colors.GREEN}✓{Colors.NC}"
+        elif result == TestResult.FAILED:
+            symbol = f"{Colors.RED}✗{Colors.NC}"
+        else:
+            symbol = f"{Colors.YELLOW}⊘{Colors.NC}"
+
+        status = "blocked" if test_case.expected_blocked else "allowed"
+        print(f"  {symbol} {test_case.name} (should be {status})")
+
+        # Record result
+        self.results.append(
+            {
+                "name": test_case.name,
+                "category": test_case.category,
+                "url": test_case.url,
+                "expected_blocked": test_case.expected_blocked,
+                "result": result.value,
+                "description": test_case.description,
+            }
+        )
+
+    def run_all_tests(self):
+        """Run all test cases"""
+        test_cases = self.get_test_cases()
+
+        print("=" * 50)
+        if self.dev_mode:
+            print("    SSRF Proxy Test Suite (DEV MODE)")
+            print("=" * 50)
+            print(f"{Colors.RED}WARNING: Testing with SSRF protections DISABLED!{Colors.NC}")
+            print(f"{Colors.YELLOW}All requests should be ALLOWED in dev mode.{Colors.NC}")
+        else:
+            print("         SSRF Proxy Test Suite")
+        print("=" * 50)
+
+        # Group tests by category
+        categories: dict[str, list[TestCase]] = {}
+        for test in test_cases:
+            if test.category not in categories:
+                categories[test.category] = []
+            categories[test.category].append(test)
+
+        # Run tests by category
+        for category, tests in categories.items():
+            print(f"\n{Colors.YELLOW}{category}:{Colors.NC}")
+            for test in tests:
+                self.run_test(test)
+
+    def load_test_cases_from_yaml(self, yaml_file: str = "test_cases.yaml") -> list[TestCase]:
+        """Load test cases from YAML configuration file"""
+        try:
+            # Try to load from YAML file
+            yaml_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), yaml_file)
+
+            with open(yaml_path) as f:
+                config = yaml.safe_load(f)  # pyright: ignore[reportAny]
+
+            test_cases: list[TestCase] = []
+
+            # Parse test categories and cases from YAML
+            test_categories = config.get("test_categories", {})  # pyright: ignore[reportAny]
+            for category_key, category_data in test_categories.items():  # pyright: ignore[reportAny]
+                category_name: str = str(category_data.get("name", category_key))  # pyright: ignore[reportAny]
+
+                test_cases_list = category_data.get("test_cases", [])  # pyright: ignore[reportAny]
+                for test_data in test_cases_list:  # pyright: ignore[reportAny]
+                    test_case = TestCase(
+                        name=str(test_data["name"]),  # pyright: ignore[reportAny]
+                        url=str(test_data["url"]),  # pyright: ignore[reportAny]
+                        expected_blocked=bool(test_data["expected_blocked"]),  # pyright: ignore[reportAny]
+                        category=category_name,
+                        description=str(test_data.get("description", "")),  # pyright: ignore[reportAny]
+                    )
+                    test_cases.append(test_case)
+
+            if test_cases:
+                print(f"{Colors.BLUE}Loaded {len(test_cases)} test cases from {yaml_file}{Colors.NC}")
+                return test_cases
+            else:
+                print(f"{Colors.YELLOW}No test cases found in {yaml_file}, using defaults{Colors.NC}")
+                return self.get_default_test_cases()
+
+        except FileNotFoundError:
+            print(f"{Colors.YELLOW}Test case file {yaml_file} not found, using defaults{Colors.NC}")
+            return self.get_default_test_cases()
+        except yaml.YAMLError as e:
+            print(f"{Colors.YELLOW}Error parsing {yaml_file}: {e}, using defaults{Colors.NC}")
+            return self.get_default_test_cases()
+        except Exception as e:
+            print(f"{Colors.YELLOW}Unexpected error loading {yaml_file}: {e}, using defaults{Colors.NC}")
+            return self.get_default_test_cases()
+
+    def get_default_test_cases(self) -> list[TestCase]:
+        """Fallback test cases if YAML loading fails"""
+        return [
+            # Essential test cases as fallback
+            TestCase("Loopback", "http://127.0.0.1", True, "Private Networks", "IPv4 loopback"),
+            TestCase("Private Network", "http://192.168.1.1", True, "Private Networks", "RFC 1918"),
+            TestCase("AWS Metadata", "http://169.254.169.254", True, "Cloud Metadata", "AWS metadata"),
+            TestCase("Public Site", "http://example.com", False, "Public Internet", "Public website"),
+            TestCase("Port 8080", "http://example.com:8080", True, "Port Restrictions", "Non-standard port"),
+        ]
+
+    def get_test_cases(self) -> list[TestCase]:
+        """Get all test cases from YAML or defaults"""
+        return self.load_test_cases_from_yaml(self.test_file)
+
+    def print_summary(self):
+        """Print test results summary"""
+        passed = sum(1 for r in self.results if r["result"] == "passed")
+        failed = sum(1 for r in self.results if r["result"] == "failed")
+        skipped = sum(1 for r in self.results if r["result"] == "skipped")
+
+        print("\n" + "=" * 50)
+        print("           Test Summary")
+        print("=" * 50)
+        print(f"Tests Passed: {Colors.GREEN}{passed}{Colors.NC}")
+        print(f"Tests Failed: {Colors.RED}{failed}{Colors.NC}")
+        if skipped > 0:
+            print(f"Tests Skipped: {Colors.YELLOW}{skipped}{Colors.NC}")
+
+        if failed == 0:
+            if hasattr(self, "dev_mode") and self.dev_mode:
+                print(f"\n{Colors.GREEN}✓ All tests passed! Development mode is working correctly.{Colors.NC}")
+                print(
+                    f"{Colors.YELLOW}Remember: Dev mode DISABLES all SSRF protections - "
+                    f"use only for development!{Colors.NC}"
+                )
+            else:
+                print(f"\n{Colors.GREEN}✓ All tests passed! SSRF proxy is configured correctly.{Colors.NC}")
+        else:
+            if hasattr(self, "dev_mode") and self.dev_mode:
+                print(f"\n{Colors.RED}✗ Some tests failed. Dev mode should allow ALL requests!{Colors.NC}")
+            else:
+                print(f"\n{Colors.RED}✗ Some tests failed. Please review the configuration.{Colors.NC}")
+            print("\nFailed tests:")
+            for r in self.results:
+                if r["result"] == "failed":
+                    status = "should be blocked" if r["expected_blocked"] else "should be allowed"
+                    print(f"  - {r['name']} ({status}): {r['url']}")
+
+        return failed == 0
+
+    def save_results(self, filename: str = "test_results.json"):
+        """Save test results to JSON file"""
+        with open(filename, "w") as f:
+            json.dump(
+                {
+                    "timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
+                    "proxy_url": self.proxy_url,
+                    "results": self.results,
+                },
+                f,
+                indent=2,
+            )
+        print(f"\nResults saved to {filename}")
+
+
+def main():
+    @dataclass
+    class Args:
+        host: str = "localhost"
+        port: int = 3128
+        no_container: bool = False
+        save_results: bool = False
+        test_file: str | None = None
+        list_tests: bool = False
+        dev_mode: bool = False
+
+    def parse_args() -> Args:
+        parser = argparse.ArgumentParser(description="Test SSRF Proxy Configuration")
+        _ = parser.add_argument("--host", type=str, default="localhost", help="Proxy host (default: localhost)")
+        _ = parser.add_argument("--port", type=int, default=3128, help="Proxy port (default: 3128)")
+        _ = parser.add_argument(
+            "--no-container",
+            action="store_true",
+            help="Don't start container (assume proxy is already running)",
+        )
+        _ = parser.add_argument("--save-results", action="store_true", help="Save test results to JSON file")
+        _ = parser.add_argument(
+            "--test-file", type=str, help="Path to YAML file containing test cases (default: test_cases.yaml)"
+        )
+        _ = parser.add_argument("--list-tests", action="store_true", help="List all test cases without running them")
+        _ = parser.add_argument(
+            "--dev-mode",
+            action="store_true",
+            help="Run in development mode (DISABLES all SSRF protections - DO NOT use in production!)",
+        )
+
+        # Parse arguments - argparse.Namespace has Any-typed attributes
+        # This is a known limitation of argparse in Python's type system
+        namespace = parser.parse_args()
+
+        # Convert namespace attributes to properly typed values
+        # argparse guarantees these attributes exist with the correct types
+        # based on our argument definitions, but the type system cannot verify this
+        return Args(
+            host=str(namespace.host),  # pyright: ignore[reportAny]
+            port=int(namespace.port),  # pyright: ignore[reportAny]
+            no_container=bool(namespace.no_container),  # pyright: ignore[reportAny]
+            save_results=bool(namespace.save_results),  # pyright: ignore[reportAny]
+            test_file=namespace.test_file or None,  # pyright: ignore[reportAny]
+            list_tests=bool(namespace.list_tests),  # pyright: ignore[reportAny]
+            dev_mode=bool(namespace.dev_mode),  # pyright: ignore[reportAny]
+        )
+
+    args = parse_args()
+
+    tester = SSRFProxyTester(args.host, args.port, args.test_file, args.dev_mode)
+
+    # If --list-tests flag is set, just list the tests and exit
+    if args.list_tests:
+        test_cases = tester.get_test_cases()
+        mode_str = " (DEVELOPMENT MODE)" if args.dev_mode else ""
+        print("\n" + "=" * 50)
+        print(f"         Available Test Cases{mode_str}")
+        print("=" * 50)
+        if args.dev_mode:
+            print(f"\n{Colors.RED}WARNING: Dev mode test cases expect ALL requests to be ALLOWED!{Colors.NC}")
+
+        # Group by category for display
+        categories: dict[str, list[TestCase]] = {}
+        for test in test_cases:
+            if test.category not in categories:
+                categories[test.category] = []
+            categories[test.category].append(test)
+
+        for category, tests in categories.items():
+            print(f"\n{Colors.YELLOW}{category}:{Colors.NC}")
+            for test in tests:
+                blocked_status = "BLOCK" if test.expected_blocked else "ALLOW"
+                color = Colors.RED if test.expected_blocked else Colors.GREEN
+                print(f"  {color}[{blocked_status}]{Colors.NC} {test.name}")
+                if test.description:
+                    print(f"        {test.description}")
+                print(f"        URL: {test.url}")
+
+        print(f"\nTotal: {len(test_cases)} test cases")
+        sys.exit(0)
+
+    try:
+        # Start container unless --no-container flag is set
+        if not args.no_container:
+            if not tester.start_proxy_container():
+                sys.exit(1)
+
+        # Run tests
+        tester.run_all_tests()
+
+        # Print summary
+        success = tester.print_summary()
+
+        # Save results if requested
+        if args.save_results:
+            tester.save_results()
+
+        # Exit with appropriate code
+        sys.exit(0 if success else 1)
+
+    finally:
+        # Cleanup
+        if not args.no_container:
+            print(f"\n{Colors.YELLOW}Cleaning up...{Colors.NC}")
+            tester.stop_proxy_container()
+
+
+if __name__ == "__main__":
+    main()

docker/docker-compose-template.yaml

@@ -245,12 +245,10 @@ services:
     volumes:
       - ./ssrf_proxy/squid.conf.template:/etc/squid/squid.conf.template
       - ./ssrf_proxy/docker-entrypoint.sh:/docker-entrypoint-mount.sh
-    entrypoint:
-      [
-        "sh",
-        "-c",
-        "cp /docker-entrypoint-mount.sh /docker-entrypoint.sh && sed -i 's/\r$$//' /docker-entrypoint.sh && chmod +x /docker-entrypoint.sh && /docker-entrypoint.sh",
-      ]
+      # Optional: Mount custom config directory for additional rules
+      # Uncomment the line below and create conf.d directory with custom .conf files
+      # - ./ssrf_proxy/conf.d:/etc/squid/conf.d:ro
+    entrypoint: [ 'sh', '-c', "cp /docker-entrypoint-mount.sh /docker-entrypoint.sh && sed -i 's/\r$$//' /docker-entrypoint.sh && chmod +x /docker-entrypoint.sh && /docker-entrypoint.sh" ]
     environment:
       # pls clearly modify the squid env vars to fit your network environment.
       HTTP_PORT: ${SSRF_HTTP_PORT:-3128}

docker/docker-compose.middleware.yaml

@@ -156,6 +156,7 @@ services:
     restart: always
     volumes:
       - ./ssrf_proxy/squid.conf.template:/etc/squid/squid.conf.template
+      - ./ssrf_proxy/squid.conf.dev.template:/etc/squid/squid.conf.dev.template
       - ./ssrf_proxy/docker-entrypoint.sh:/docker-entrypoint-mount.sh
     entrypoint:
       [

docker/docker-compose.yaml

@@ -842,12 +842,10 @@ services:
     volumes:
       - ./ssrf_proxy/squid.conf.template:/etc/squid/squid.conf.template
       - ./ssrf_proxy/docker-entrypoint.sh:/docker-entrypoint-mount.sh
-    entrypoint:
-      [
-        "sh",
-        "-c",
-        "cp /docker-entrypoint-mount.sh /docker-entrypoint.sh && sed -i 's/\r$$//' /docker-entrypoint.sh && chmod +x /docker-entrypoint.sh && /docker-entrypoint.sh",
-      ]
+      # Optional: Mount custom config directory for additional rules
+      # Uncomment the line below and create conf.d directory with custom .conf files
+      # - ./ssrf_proxy/conf.d:/etc/squid/conf.d:ro
+    entrypoint: [ 'sh', '-c', "cp /docker-entrypoint-mount.sh /docker-entrypoint.sh && sed -i 's/\r$$//' /docker-entrypoint.sh && chmod +x /docker-entrypoint.sh && /docker-entrypoint.sh" ]
     environment:
       # pls clearly modify the squid env vars to fit your network environment.
       HTTP_PORT: ${SSRF_HTTP_PORT:-3128}

docker/middleware.env.example

@@ -64,6 +64,10 @@ SSRF_HTTP_PORT=3128
 SSRF_COREDUMP_DIR=/var/spool/squid
 SSRF_REVERSE_PROXY_PORT=8194
 SSRF_SANDBOX_HOST=sandbox
+# Development mode switch - set to true to disable all SSRF protections
+# WARNING: This allows access to localhost, private networks, and all ports!
+# Only use this in development environments, NEVER in production!
+SSRF_PROXY_DEV_MODE=false
 
 # ------------------------------
 # Environment Variables for weaviate Service

docker/ssrf_proxy/README.md

@@ -0,0 +1,204 @@
+# SSRF Proxy Configuration
+
+This directory contains the Squid proxy configuration used to prevent Server-Side Request Forgery (SSRF) attacks in Dify.
+
+## Security by Default
+
+The default configuration (`squid.conf.template`) prevents SSRF attacks while allowing normal internet access:
+
+- **Blocks all private/internal networks** (RFC 1918, loopback, link-local, etc.)
+- **Only allows HTTP (80) and HTTPS (443) ports**
+- **Allows all public internet resources** (operates as a blacklist for private networks)
+- **Additional restrictions can be added** via custom configurations in `/etc/squid/conf.d/`
+
+## Customizing the Configuration
+
+### For Development/Local Environments
+
+To allow additional domains or relax restrictions for your local environment:
+
+1. Create a `conf.d` directory in your deployment
+1. Copy example configurations from `conf.d.example/` and modify as needed
+1. Mount the config files to `/etc/squid/conf.d/` in the container
+
+### Example: Docker Compose
+
+```yaml
+services:
+  ssrf-proxy:
+    volumes:
+      - ./my-proxy-configs:/etc/squid/conf.d:ro
+```
+
+### Example: Kubernetes ConfigMap
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: squid-custom-config
+data:
+  20-allow-external-domains.conf: |
+    acl allowed_external dstdomain .example.com
+    http_access allow allowed_external
+---
+apiVersion: apps/v1
+kind: Deployment
+spec:
+  template:
+    spec:
+      containers:
+      - name: ssrf-proxy
+        volumeMounts:
+        - name: custom-config
+          mountPath: /etc/squid/conf.d
+      volumes:
+      - name: custom-config
+        configMap:
+          name: squid-custom-config
+```
+
+## Available Example Configurations
+
+The `conf.d.example/` directory contains example configurations:
+
+- **00-testing-environment.conf.example**: Configuration for CI/testing environments (NOT for production)
+- **10-allow-internal-services.conf.example**: Allow internal services (use with caution!)
+- **20-allow-external-domains.conf.example**: Allow specific external domains
+- **30-allow-additional-ports.conf.example**: Allow additional ports
+- **40-restrict-to-allowlist.conf.example**: Convert to whitelist mode (block all except allowed)
+
+## Security Considerations
+
+⚠️ **WARNING**: Relaxing these restrictions can expose your system to SSRF attacks!
+
+- **Never allow access to private networks in production** unless absolutely necessary
+- **Carefully review any domains you whitelist** to ensure they cannot be used for SSRF
+- **Avoid allowing high port ranges** (1025-65535) as they can bypass security restrictions
+- **Monitor proxy logs** for suspicious activity
+
+## Default Blocked Networks
+
+The following networks are blocked by default to prevent SSRF:
+
+- `0.0.0.0/8` - "This" network
+- `10.0.0.0/8` - Private network (RFC 1918)
+- `127.0.0.0/8` - Loopback
+- `169.254.0.0/16` - Link-local (RFC 3927)
+- `172.16.0.0/12` - Private network (RFC 1918)
+- `192.168.0.0/16` - Private network (RFC 1918)
+- `224.0.0.0/4` - Multicast
+- `fc00::/7` - IPv6 unique local addresses
+- `fe80::/10` - IPv6 link-local
+- `::1/128` - IPv6 loopback
+
+## Development Mode
+
+⚠️ **WARNING: Development mode DISABLES all SSRF protections! Only use in development environments!**
+
+Development mode provides a zero-configuration environment that:
+
+- Allows access to ALL private networks and localhost
+- Allows access to cloud metadata endpoints
+- Allows connections to any port
+- Disables all SSRF protections for easier development
+
+### Using Development Mode
+
+#### Option 1: Environment Variable (Recommended)
+
+Simply set the `SSRF_PROXY_DEV_MODE` environment variable to `true`:
+
+```bash
+# In your .env or middleware.env file
+SSRF_PROXY_DEV_MODE=true
+
+# Then start normally
+docker-compose -f docker-compose.middleware.yaml up ssrf_proxy
+```
+
+Or set it directly in docker-compose:
+
+```yaml
+services:
+  ssrf_proxy:
+    environment:
+      SSRF_PROXY_DEV_MODE: true
+```
+
+**Important Note about Docker Networking:**
+
+When accessing services on your host machine from within Docker containers:
+
+- Do NOT use `127.0.0.1` or `localhost` (these refer to the container itself)
+- Instead use:
+  - `host.docker.internal:port` (recommended, works on Mac/Windows/Linux with Docker 20.10+)
+  - Your host machine's actual IP address
+  - On Linux: the Docker bridge gateway (usually `172.17.0.1`)
+
+Example:
+
+```bash
+# Wrong (won't work from inside container):
+http://127.0.0.1:1234
+
+# Correct (will work):
+http://host.docker.internal:1234
+```
+
+The development mode uses `squid.conf.dev.template` which allows all connections.
+
+## Testing
+
+Comprehensive integration tests are available to validate the SSRF proxy configuration:
+
+```bash
+# Run from the api/ directory
+cd ../../api
+uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py
+
+# List available test cases
+uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py --list-tests
+
+# Use extended test suite
+uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py --test-file test_cases_extended.yaml
+
+# Test development mode (all requests should be allowed)
+uv run python tests/integration_tests/ssrf_proxy/test_ssrf_proxy.py --dev-mode
+```
+
+The test suite validates:
+
+- Blocking of private networks and loopback addresses
+- Blocking of cloud metadata endpoints
+- Allowing of public internet resources
+- Port restriction enforcement
+
+See `api/tests/integration_tests/ssrf_proxy/TEST_CASES_README.md` for detailed testing documentation.
+
+## Troubleshooting
+
+If your application needs to access a service that's being blocked:
+
+1. Check the Squid logs to identify what's being blocked
+1. Create a custom configuration in `/etc/squid/conf.d/`
+1. Only allow the minimum necessary access
+1. Test thoroughly to ensure security is maintained
+
+## File Structure
+
+```
+docker/ssrf_proxy/
+├── squid.conf.template       # SSRF protection configuration  
+├── docker-entrypoint.sh      # Container entrypoint script
+├── conf.d.example/           # Example override configurations
+│   ├── 00-testing-environment.conf.example
+│   ├── 10-allow-internal-services.conf.example
+│   ├── 20-allow-external-domains.conf.example
+│   ├── 30-allow-additional-ports.conf.example
+│   └── 40-restrict-to-allowlist.conf.example
+├── conf.d.dev/               # Development mode configuration
+│   └── 00-development-mode.conf  # Disables all SSRF protections
+├── docker-compose.dev.yaml   # Docker Compose overlay for dev mode
+└── README.md                 # This file
+```

docker/ssrf_proxy/conf.d.example/00-testing-environment.conf.example

@@ -0,0 +1,12 @@
+# Configuration for CI/Testing Environment
+# Copy this file to /etc/squid/conf.d/00-testing-environment.conf when running tests
+# WARNING: This configuration is ONLY for testing and should NOT be used in production
+
+# Allow access to sandbox service for integration tests
+acl sandbox_service dst sandbox
+http_access allow sandbox_service
+
+# Allow access to Docker internal networks for testing
+# This is needed when services communicate within Docker networks
+acl docker_internal dst 172.16.0.0/12
+http_access allow docker_internal

docker/ssrf_proxy/conf.d.example/10-allow-internal-services.conf.example

@@ -0,0 +1,15 @@
+# Example: Allow access to internal services (USE WITH CAUTION!)
+# Copy this file to /etc/squid/conf.d/20-allow-internal-services.conf to enable
+# WARNING: This reduces SSRF protection. Only use if you understand the security implications.
+
+# Example: Allow specific internal service
+# acl internal_api_service dst 10.0.1.100
+# http_access allow internal_api_service
+
+# Example: Allow Docker network (172.17.0.0/16 is Docker's default bridge network)
+# acl docker_network dst 172.17.0.0/16
+# http_access allow docker_network
+
+# Example: Allow localhost access (DANGEROUS - can bypass SSRF protection)
+# acl localhost_dst dst 127.0.0.1
+# http_access allow localhost_dst

docker/ssrf_proxy/conf.d.example/20-allow-external-domains.conf.example

@@ -0,0 +1,18 @@
+# Example: Allow access to specific external domains
+# Copy this file to /etc/squid/conf.d/30-allow-external-domains.conf to enable
+
+# Allow specific domains for API integrations
+# acl allowed_apis dstdomain .api.openai.com .anthropic.com .googleapis.com
+# http_access allow allowed_apis
+
+# Allow webhook endpoints
+# acl webhook_endpoints dstdomain .webhook.site .zapier.com
+# http_access allow webhook_endpoints
+
+# Allow storage services
+# acl storage_services dstdomain .s3.amazonaws.com .blob.core.windows.net .storage.googleapis.com
+# http_access allow storage_services
+
+# Allow by specific IP address (use with caution)
+# acl trusted_ip dst 203.0.113.10
+# http_access allow trusted_ip

docker/ssrf_proxy/conf.d.example/30-allow-additional-ports.conf.example

@@ -0,0 +1,17 @@
+# Example: Allow additional ports for specific protocols
+# Copy this file to /etc/squid/conf.d/40-allow-additional-ports.conf to enable
+# WARNING: Opening additional ports can increase security risks
+
+# Allow additional safe ports
+# acl Safe_ports port 8080    # http-alt
+# acl Safe_ports port 8443    # https-alt
+# acl Safe_ports port 3000    # common development port
+# acl Safe_ports port 5000    # common API port
+
+# Allow additional SSL ports for CONNECT method
+# acl SSL_ports port 8443     # https-alt
+# acl SSL_ports port 3443     # custom ssl
+
+# Allow high ports (1025-65535) - DANGEROUS! Can be used to bypass restrictions
+# acl Safe_ports port 1025-65535
+# acl SSL_ports port 1025-65535

docker/ssrf_proxy/conf.d.example/40-restrict-to-allowlist.conf.example

@@ -0,0 +1,22 @@
+# Example: Convert proxy to whitelist mode (strict mode)
+# Copy this file to /etc/squid/conf.d/40-restrict-to-allowlist.conf to enable
+# WARNING: This will block ALL internet access except explicitly allowed domains
+#
+# This changes the default behavior from blacklist (block private, allow public)
+# to whitelist (block everything, allow specific domains only)
+
+# First, insert specific allowed domains BEFORE the final "allow all" rule
+# The include statement is processed sequentially, so rules here take precedence
+
+# Example: Only allow specific services
+# acl allowed_services dstdomain .openai.com .anthropic.com .google.com
+# http_access allow allowed_services
+
+# Example: Allow Dify marketplace
+# acl allowed_marketplace dstdomain .marketplace.dify.ai
+# http_access allow allowed_marketplace
+
+# Then deny all other requests (converting to whitelist mode)
+# This rule will override the default "allow all" at the end
+# Uncomment the following line to enable strict whitelist mode:
+# http_access deny all

docker/ssrf_proxy/docker-entrypoint.sh

@@ -26,8 +26,26 @@ tail -F /var/log/squid/error.log 2>/dev/null &
 tail -F /var/log/squid/store.log 2>/dev/null &
 tail -F /var/log/squid/cache.log 2>/dev/null &
 
+# Select the appropriate template based on DEV_MODE
+echo "[ENTRYPOINT] SSRF_PROXY_DEV_MODE is set to: '${SSRF_PROXY_DEV_MODE}'"
+if [ "${SSRF_PROXY_DEV_MODE}" = "true" ] || [ "${SSRF_PROXY_DEV_MODE}" = "True" ] || [ "${SSRF_PROXY_DEV_MODE}" = "TRUE" ] || [ "${SSRF_PROXY_DEV_MODE}" = "1" ]; then
+    echo "[ENTRYPOINT] WARNING: Development mode is ENABLED! All SSRF protections are DISABLED!"
+    echo "[ENTRYPOINT] This allows access to localhost, private networks, and all ports."
+    echo "[ENTRYPOINT] DO NOT USE IN PRODUCTION!"
+    TEMPLATE_FILE="/etc/squid/squid.conf.dev.template"
+else
+    echo "[ENTRYPOINT] Using production configuration with SSRF protections enabled"
+    TEMPLATE_FILE="/etc/squid/squid.conf.template"
+fi
+
+# Check if the selected template exists
+if [ ! -f "$TEMPLATE_FILE" ]; then
+    echo "[ENTRYPOINT] ERROR: Template file $TEMPLATE_FILE not found"
+    exit 1
+fi
+
 # Replace environment variables in the template and output to the squid.conf
-echo "[ENTRYPOINT] replacing environment variables in the template"
+echo "[ENTRYPOINT] replacing environment variables in the template: $TEMPLATE_FILE"
 awk '{
     while(match($0, /\${[A-Za-z_][A-Za-z_0-9]*}/)) {
         var = substr($0, RSTART+2, RLENGTH-3)
@@ -35,7 +53,24 @@ awk '{
         $0 = substr($0, 1, RSTART-1) val substr($0, RSTART+RLENGTH)
     }
     print
-}' /etc/squid/squid.conf.template > /etc/squid/squid.conf
+}' "$TEMPLATE_FILE" > /etc/squid/squid.conf
+
+# Log first few lines of generated config for debugging
+echo "[ENTRYPOINT] First 30 lines of generated squid.conf:"
+head -n 30 /etc/squid/squid.conf
+
+# Create an empty conf.d directory if it doesn't exist
+if [ ! -d /etc/squid/conf.d ]; then
+    echo "[ENTRYPOINT] creating /etc/squid/conf.d directory"
+    mkdir -p /etc/squid/conf.d
+fi
+
+# If conf.d directory is empty, create a placeholder file to prevent include errors
+# Only needed for production template which has the include directive
+if [ "${SSRF_PROXY_DEV_MODE}" != "true" ] && [ -z "$(ls -A /etc/squid/conf.d/*.conf 2>/dev/null)" ]; then
+    echo "[ENTRYPOINT] conf.d directory is empty, creating placeholder"
+    echo "# Placeholder file to prevent include errors" > /etc/squid/conf.d/placeholder.conf
+fi
 
 /usr/sbin/squid -Nz
 echo "[ENTRYPOINT] starting squid"

docker/ssrf_proxy/setup-testing.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+# Setup script for SSRF proxy in testing/CI environments
+# This script creates the necessary configuration to allow sandbox access during tests
+
+echo "Setting up SSRF proxy for testing environment..."
+
+# Create conf.d directory if it doesn't exist
+mkdir -p "$(dirname "$0")/conf.d"
+
+# Copy testing configuration
+cat > "$(dirname "$0")/conf.d/00-testing-environment.conf" << 'EOF'
+# CI/Testing Environment Configuration
+# This configuration is automatically generated for testing
+# DO NOT USE IN PRODUCTION
+
+# Allow access to sandbox service for integration tests
+acl sandbox_service dst sandbox
+http_access allow sandbox_service
+
+# Allow access to Docker internal networks for testing
+acl docker_internal dst 172.16.0.0/12
+http_access allow docker_internal
+
+# Allow localhost connections for testing
+acl test_localhost dst 127.0.0.1 ::1
+http_access allow test_localhost
+EOF
+
+echo "SSRF proxy testing configuration created successfully."

docker/ssrf_proxy/squid.conf.dev.template

@@ -0,0 +1,30 @@
+################################## DEVELOPMENT MODE CONFIGURATION ##################################
+# WARNING: This configuration DISABLES all SSRF protections!
+# Only use this in development environments. NEVER use in production!
+
+# Allow all requests - put this FIRST before any other rules
+http_access allow all
+
+################################## Proxy Server Configuration ##################################
+http_port ${HTTP_PORT}
+coredump_dir ${COREDUMP_DIR}
+
+# Refresh patterns
+refresh_pattern ^ftp:		1440	20%	10080
+refresh_pattern ^gopher:	1440	0%	1440
+refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
+refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims
+refresh_pattern \/Release(|\.gpg)$ 0 0% 0 refresh-ims
+refresh_pattern \/InRelease$ 0 0% 0 refresh-ims
+refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims
+refresh_pattern .		0	20%	4320
+
+################################## Reverse Proxy To Sandbox ##################################
+http_port ${REVERSE_PROXY_PORT} accel vhost
+cache_peer ${SANDBOX_HOST} parent ${SANDBOX_PORT} 0 no-query originserver
+
+# Buffer size for file uploads
+client_request_buffer_max_size 100 MB
+
+# Debug logging for development
+debug_options ALL,1

docker/ssrf_proxy/squid.conf.template

@@ -1,37 +1,79 @@
-acl localnet src 0.0.0.1-0.255.255.255	# RFC 1122 "this" network (LAN)
-acl localnet src 10.0.0.0/8		# RFC 1918 local private network (LAN)
-acl localnet src 100.64.0.0/10		# RFC 6598 shared address space (CGN)
-acl localnet src 169.254.0.0/16 	# RFC 3927 link-local (directly plugged) machines
-acl localnet src 172.16.0.0/12		# RFC 1918 local private network (LAN)
-acl localnet src 192.168.0.0/16		# RFC 1918 local private network (LAN)
-acl localnet src fc00::/7       	# RFC 4193 local private network range
-acl localnet src fe80::/10      	# RFC 4291 link-local (directly plugged) machines
+################################## SSRF Protection Configuration ##################################
+# This configuration prevents SSRF attacks by blocking access to private/internal networks
+# while allowing normal access to public internet resources.
+# To add additional restrictions or allowances, create config files in /etc/squid/conf.d/
+
+################################## Security ACLs ##################################
+# Define private/local networks that should be BLOCKED by default
+acl private_networks dst 0.0.0.0/8          # "This" network
+acl private_networks dst 10.0.0.0/8         # RFC 1918 private network
+acl private_networks dst 100.64.0.0/10      # RFC 6598 shared address space
+acl private_networks dst 127.0.0.0/8        # Loopback
+acl private_networks dst 169.254.0.0/16     # RFC 3927 link-local
+acl private_networks dst 172.16.0.0/12      # RFC 1918 private network  
+acl private_networks dst 192.168.0.0/16     # RFC 1918 private network
+acl private_networks dst 224.0.0.0/4        # Multicast
+acl private_networks dst 240.0.0.0/4        # Reserved for future use
+acl private_networks dst 255.255.255.255/32 # Broadcast
+acl private_networks dst fc00::/7           # IPv6 unique local addresses
+acl private_networks dst fe80::/10          # IPv6 link-local addresses
+acl private_networks dst ::1/128            # IPv6 loopback
+acl private_networks dst ff00::/8           # IPv6 multicast
+
+# Define localhost source
+acl localhost src 127.0.0.1/32 ::1
+
+# Define localnet ACL for compatibility with debian.conf (if present in ubuntu/squid image)
+acl localnet src 10.0.0.0/8
+acl localnet src 172.16.0.0/12
+acl localnet src 192.168.0.0/16
+
+# Define ports
 acl SSL_ports port 443
-# acl SSL_ports port 1025-65535   # Enable the configuration to resolve this issue: https://github.com/langgenius/dify/issues/12792
-acl Safe_ports port 80		# http
-acl Safe_ports port 21		# ftp
-acl Safe_ports port 443		# https
-acl Safe_ports port 70		# gopher
-acl Safe_ports port 210		# wais
-acl Safe_ports port 1025-65535	# unregistered ports
-acl Safe_ports port 280		# http-mgmt
-acl Safe_ports port 488		# gss-http
-acl Safe_ports port 591		# filemaker
-acl Safe_ports port 777		# multiling http
+acl Safe_ports port 80          # http
+acl Safe_ports port 443         # https
 acl CONNECT method CONNECT
-acl allowed_domains dstdomain .marketplace.dify.ai
-http_access allow allowed_domains
+
+################################## Access Control Rules ##################################
+# IMPORTANT: Order matters! First matching rule wins.
+
+# Special rule for reverse proxy port (sandbox access)
+# This must come FIRST to bypass other restrictions for sandbox connectivity
+acl reverse_proxy_port myport ${REVERSE_PROXY_PORT}
+http_access allow reverse_proxy_port
+
+# DENY access to all private/local networks - prevents SSRF attacks
+http_access deny private_networks
+
+# DENY non-safe ports
 http_access deny !Safe_ports
+
+# DENY CONNECT to non-SSL ports
 http_access deny CONNECT !SSL_ports
+
+# Allow manager access from localhost only
 http_access allow localhost manager
 http_access deny manager
-http_access allow localhost
-include /etc/squid/conf.d/*.conf
-http_access deny all
 
-################################## Proxy Server ################################
+# Note: We don't have a blanket "allow localhost" rule to prevent bypassing SSRF protection
+# Localhost connections will still be subject to the same restrictions as other clients
+
+# User overrides in /etc/squid/conf.d/*.conf should be placed here
+# These can be used to add additional restrictions or allowances
+# Note: debian.conf may be present by default in the ubuntu/squid image
+# The include directive uses a script to handle optional includes
+include /etc/squid/conf.d/*.conf
+
+# Allow all other requests (public internet resources)
+# This makes the proxy work as a blacklist (blocking private networks)
+# rather than a whitelist (blocking everything except allowed)
+http_access allow all
+
+################################## Proxy Server Configuration ##################################
 http_port ${HTTP_PORT}
 coredump_dir ${COREDUMP_DIR}
+
+# Refresh patterns
 refresh_pattern ^ftp:		1440	20%	10080
 refresh_pattern ^gopher:	1440	0%	1440
 refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
@@ -41,16 +83,14 @@ refresh_pattern \/InRelease$ 0 0% 0 refresh-ims
 refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims
 refresh_pattern .		0	20%	4320
 
+# Upstream proxy configuration (uncomment and configure if needed)
+# cache_peer <upstream_proxy_ip> parent 3128 0 no-query no-digest no-netdb-exchange default
 
-# cache_dir ufs /var/spool/squid 100 16 256
-# upstream proxy, set to your own upstream proxy IP to avoid SSRF attacks
-# cache_peer 172.1.1.1 parent 3128 0 no-query no-digest no-netdb-exchange default
-
-################################## Reverse Proxy To Sandbox ################################
+################################## Reverse Proxy To Sandbox ##################################
+# This configuration allows the sandbox to be accessed through the reverse proxy
+# The reverse proxy port is separate from the main proxy port and has different rules
 http_port ${REVERSE_PROXY_PORT} accel vhost
 cache_peer ${SANDBOX_HOST} parent ${SANDBOX_PORT} 0 no-query originserver
-acl src_all src all
-http_access allow src_all
 
-# Unless the option's size is increased, an error will occur when uploading more than two files.
-client_request_buffer_max_size 100 MB
+# Buffer size for file uploads
+client_request_buffer_max_size 100 MB