Proxy Concurrency Saturation Test: Find the Safe Throughput Knee

The fastest-looking proxy configuration is often already overloaded. Throughput may still rise while queue wait, p95 latency, retries and cost per valid result rise faster. The useful operating point is the knee just before added concurrency produces little additional verified work.

This guide provides a repeatable capacity test for residential, rotating, ISP and datacenter proxy routes. Run it only against systems you own or have explicit permission to load-test. It is not a method for pushing past a website's limits.

Define the four numbers people call “concurrency”

Record these separately:

• Offered concurrency: jobs the scheduler is trying to run.
• In-flight requests: requests that have left the application and not completed.
• Active connections: established or connecting transport sockets; multiplexing can place several requests on one connection.
• Queued jobs: work waiting inside the application, HTTP client, proxy gateway or downstream processor.

If only worker count is recorded, a client connection pool can silently queue most work. The proxy is then blamed for delay created before a request ever reached it.

Map every possible bottleneck

Draw the path before testing:

scheduler -> resolver -> client connection pool -> proxy account/gateway
          -> network and TLS -> authorized target -> parser -> storage

A plateau at any layer can look like “proxy saturation.” Capture client queue wait, connection acquisition, DNS, proxy connect, destination connect, TLS, time to first byte, transfer, validation and downstream processing independently.

Prepare a deterministic and authorized fixture

Use an endpoint designed for the test with a stable response size, known digest and controllable response delay. Agree on the test window and ceiling with its owner. Keep payload, method, headers, expected status and validation rule fixed.

Build two controls:

1. Direct control: the same client and fixture without a proxy, where permitted. This exposes client, target and downstream ceilings.
2. Proxy path: the identical workload through one documented route class and account policy.

Do not mix markets, session modes, address families or target behaviors in one curve. Create separate runs for Global, North America, Europe and APAC routes when they matter operationally.

Instrument useful work, not raw requests

For every attempt, capture a sanitized record:

run_id, route_class, market, address_family, session_mode
offered_concurrency, active_connections, in_flight, client_queue_ms
dns_ms, proxy_connect_ms, tls_ms, ttfb_ms, total_ms
status_class, retry_after_seen, attempt_number, response_digest_ok
bytes_transferred, useful_result, failure_phase

Report per step:

• verified useful results per second;
• p50, p95 and p99 queue and total latency;
• first-attempt and final success rates;
• response-integrity rate;
• retries and transferred bytes per useful result;
• 407, 429 and 5xx rates by phase;
• cost per useful result, not merely cost per request.

A response that is truncated, from the wrong market, unauthenticated or invalid is not useful throughput.

Run a bounded concurrency ladder

Warm the client in a documented way, then test a ladder such as 1, 2, 4, 8, 16 and 32. The exact ceiling must come from your authorization, provider plan and target policy—not from this example.

At each step:

1. Hold the work definition and route class constant.
2. Run long enough to include normal latency variation, with equal measurement windows.
3. Separate warm-up samples from measured samples.
4. Repeat the step to distinguish a capacity change from transient route noise.
5. Stop immediately when the agreed ceiling, error budget or latency guardrail is crossed.

Use a closed-loop test first: each worker starts a new job only after the previous job finishes. An open-loop arrival-rate test can reveal queue growth, but it requires stricter safeguards because work may accumulate faster than the system can complete it.

Locate the knee instead of chasing the peak

For adjacent steps, calculate:

throughput_gain = useful_rps_new - useful_rps_previous
gain_per_added_worker = throughput_gain / added_concurrency
cost_per_useful_result = total_proxy_cost / verified_results

The knee is the last step before marginal throughput collapses or a guardrail breaks. Typical signals include:

• useful throughput grows less than 10–15% while concurrency doubles;
• p95 total or queue latency rises sharply;
• first-attempt success or response integrity drops;
• retry volume and bytes per useful result increase;
• 429, 407, reset or timeout rates accelerate;
• downstream parsing or storage develops a persistent queue.

Choose a conservative operating point below the knee. A practical safety factor may be 70–85% of the observed stable level, but the factor must reflect route volatility and business tolerance rather than a universal rule.

Identify the saturated layer

Use the curve and phase timings together:

HTTP/2 or HTTP/3 multiplexing can decouple request concurrency from socket count. Record negotiated protocol and stream behavior; do not assume one socket equals one request.

Size the production worker pool

Capacity is constrained by the smallest authorized limit:

safe concurrency = observed stable capacity per route × safety factor

Also cap it by client resources, provider contract, target authorization and downstream capacity. Keep separate budgets by market and workload type. A single global worker pool can let one slow route consume every slot.

Add adaptive backpressure:

• stop admitting new work when queues exceed a time or depth budget;
• reduce concurrency on sustained tail-latency or first-attempt failure changes;
• honor server retry guidance and use bounded backoff with jitter;
• cap retry attempts and include them in cost and load accounting;
• recover gradually after several healthy windows instead of jumping to the old peak.

Rotating to another IP must never be used to bypass a rate limit or access policy.

Acceptance checklist

• [ ] Test target, window and load ceiling are explicitly authorized.
• [ ] Direct and proxy controls use the same deterministic work unit.
• [ ] Markets, address families and session modes have separate curves.
• [ ] Queue wait, active connections and in-flight requests are measured separately.
• [ ] First-attempt success, integrity and useful throughput are reported.
• [ ] Retries, bytes and cost are normalized per useful result.
• [ ] Stop conditions are automatic and tested.
• [ ] Production concurrency sits below the repeatable knee.
• [ ] Backpressure and gradual recovery are configured.

FAQ

Is the highest requests-per-second step the right setting?

Usually not. It may hide extreme tail latency, retries, invalid responses and unstable queues. Select the highest repeatable step that remains inside all quality, policy and cost guardrails.

Why does increasing workers not increase active connections?

The HTTP client may enforce per-origin or global connection limits and queue excess work. Multiplexed protocols can also carry several streams over one connection. Inspect pool metrics and negotiated protocol.

Should every route use the same concurrency?

No. Capacity can differ by market, address family, proxy product, session policy and workload. Maintain measured budgets for each operational route class.

Can retries be excluded from the benchmark?

No. They consume bandwidth, proxy usage and target capacity. Report first-attempt outcomes separately, then include every attempt in final cost and load.

Related 98IP guides

• Test proxy idle timeouts and keep-alive reuse
• Validate proxy response integrity
• Audit the HTTPS proxy TLS chain

Compliance note

Load-test only systems you own or are explicitly authorized to test. Respect contracts, robots and access policies, stated rate limits, privacy requirements and retry guidance. Do not use proxy rotation to evade blocks, quotas or identity controls. Store only sanitized operational telemetry and never log proxy credentials or personal data.