How to Use a Proxy in Python, Node.js and cURL: Code Examples

Using a proxy in a script takes only a few lines, as long as you know each tool's syntax and the traps that cost hours: the wrong dictionary key, a badly encoded password, a forgotten timeout. Here are complete examples for cURL, Python and Node.js, along with production best practices and fixes for the most common errors.

Before you start: the proxy URL

Proxy access comes down to four pieces of information: host, port, username and password. Airproxy delivers them in host:port:username:password format, but most libraries expect a URL:

# Line as delivered
proxy.example.com:8080:username:password

# URL for an HTTP(S) proxy
http://username:password@proxy.example.com:8080

# URL for a SOCKS5 proxy, DNS resolved by the proxy
socks5h://username:password@proxy.example.com:8080

The scheme describes the protocol of the proxy, not of the website. An HTTP proxy carries HTTPS traffic perfectly well: your tool asks it to open a tunnel (the CONNECT method) and encryption is negotiated directly with the website. So keep http://, even for https:// pages. To choose between the two protocols, see our HTTP vs SOCKS5 proxy comparison.

cURL: check your access in one line

The api.ipify.org service, used in every example, returns the IP address that contacted it: you should see the proxy's IP, not your own.

# HTTP proxy, credentials in the URL
curl -x "http://username:password@proxy.example.com:8080" https://api.ipify.org

# Same thing, credentials passed separately with -U
curl -x "http://proxy.example.com:8080" -U "username:password" https://api.ipify.org

# SOCKS5, the proxy resolves domain names
curl --socks5-hostname proxy.example.com:8080 -U "username:password" https://api.ipify.org

# Equivalent with a single URL
curl -x "socks5h://username:password@proxy.example.com:8080" https://api.ipify.org

--socks5-hostname, like the socks5h:// scheme, lets the proxy resolve domain names, which --socks5 and socks5:// do not. In Windows PowerShell, type curl.exe: there, curl is an alias for a different command.

Python: requests and httpx

requests

The requests library expects a dictionary that maps each type of target URL to a proxy:

import requests

proxy = "http://username:password@proxy.example.com:8080"
proxies = {"http": proxy, "https": proxy}

r = requests.get("https://api.ipify.org", proxies=proxies, timeout=10)
print(r.text)  # should print the proxy's IP

A classic trap: the "https" key refers to https:// websites, not to the proxy's protocol, so its value stays an http:// URL. And without a timeout, requests can wait forever.

For SOCKS5, install the optional dependency and change the scheme:

pip install "requests[socks]"

proxy = "socks5h://username:password@proxy.example.com:8080"
proxies = {"http": proxy, "https": proxy}

With socks5://, requests would resolve names on your machine: that is a DNS leak, and our guide on how to test a proxy shows you how to detect one.

httpx

With httpx, the proxy is set on the client, through the proxy parameter. The old proxies parameter was removed in version 0.28, which breaks many examples published online.

import httpx

proxy = "http://username:password@proxy.example.com:8080"

with httpx.Client(proxy=proxy, timeout=10.0) as client:
    r = client.get("https://api.ipify.org")
    print(r.text)

httpx.AsyncClient takes the same parameter. For SOCKS5, install httpx[socks] and pass a socks5:// URL. By default, httpx gives up after five seconds of network inactivity: adjust this timeout to the websites you target.

Node.js: fetch, axios and SOCKS5

The following examples are ES modules (a .mjs file, or "type": "module" in package.json), which allows top-level await.

Native fetch with undici

The fetch built into Node.js since version 18 does not use any proxy by default, even when HTTPS_PROXY is set. It is built on undici: install that package and pass a ProxyAgent in the dispatcher option.

npm install undici

import { ProxyAgent } from 'undici';

const dispatcher = new ProxyAgent('http://username:password@proxy.example.com:8080');

const res = await fetch('https://api.ipify.org?format=json', {
  dispatcher,
  signal: AbortSignal.timeout(10000), // give up after 10 seconds
});
const { ip } = await res.json();
console.log(ip);

Create the agent once and reuse it. To rule out any version mismatch with the built-in fetch, you can also import fetch from undici.

axios with https-proxy-agent

npm install axios https-proxy-agent

import axios from 'axios';
import { HttpsProxyAgent } from 'https-proxy-agent';

const agent = new HttpsProxyAgent('http://username:password@proxy.example.com:8080');

const client = axios.create({
  httpsAgent: agent,
  proxy: false, // turns off axios's built-in proxy handling
  timeout: 10000,
});

const { data } = await client.get('https://api.ipify.org?format=json');
console.log(data.ip);

proxy: false prevents a conflict: axios reads proxy environment variables on its own, and that setting could override your agent. HttpsProxyAgent covers https:// websites; for plain http://, the http-proxy-agent package does the same job through httpAgent.

SOCKS5 with socks-proxy-agent

npm install socks-proxy-agent

import axios from 'axios';
import { SocksProxyAgent } from 'socks-proxy-agent';

const agent = new SocksProxyAgent('socks5h://username:password@proxy.example.com:8080');

const { data } = await axios.get('https://api.ipify.org?format=json', {
  httpAgent: agent,
  httpsAgent: agent,
  proxy: false,
  timeout: 10000,
});
console.log(data.ip);

This agent also works with Node.js's http and https modules (the agent option), but not with native fetch.

Production best practices

No passwords in your code

A hard-coded credential sooner or later ends up in a Git repository or a screenshot. Store the proxy URL in an environment variable, fed by a .env file kept out of the repository or by a secrets manager. The standard HTTPS_PROXY and HTTP_PROXY variables are picked up automatically by requests and httpx; cURL reads them too, but only in lowercase (http_proxy) for the second one:

export HTTPS_PROXY="http://username:password@proxy.example.com:8080"
export http_proxy="$HTTPS_PROXY"

curl https://api.ipify.org
python script.py  # requests and httpx also use these variables

These variables apply to every program started from that terminal, pip and git included. In PowerShell, write $env:HTTPS_PROXY = "...". In Node.js, pass the value to the agent: new ProxyAgent(process.env.HTTPS_PROXY).

Encode special characters

In a URL, @, :, /, #, ? and % have a specific meaning. If they appear in the username or password, the URL is split in the wrong place and authentication fails. Percent-encode both fields (@ becomes %40). This function turns a line in Airproxy format into a ready-to-use URL:

from urllib.parse import quote

def proxy_url(line, scheme="http"):
    host, port, username, password = line.strip().split(":", 3)
    return f"{scheme}://{quote(username, safe='')}:{quote(password, safe='')}@{host}:{port}"

print(proxy_url("proxy.example.com:8080:username:password"))
print(proxy_url("proxy.example.com:8080:username:password", "socks5h"))

In JavaScript, encodeURIComponent() does the same job.

Timeouts and retries

Neither requests nor axios sets a timeout by default: a silent proxy or website can hang your script. Set a timeout on every request and add a few spaced-out retries to absorb transient errors:

import os
import requests
from requests.adapters import HTTPAdapter
from urllib3.util import Retry

proxy = os.environ["HTTPS_PROXY"]
proxies = {"http": proxy, "https": proxy}

retries = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504])
adapter = HTTPAdapter(max_retries=retries)

session = requests.Session()
session.mount("http://", adapter)
session.mount("https://", adapter)

r = session.get("https://api.ipify.org", proxies=proxies, timeout=(5, 30))
print(r.text)

The (5, 30) pair separates the connect timeout from the read timeout, in seconds. Retries are spaced further and further apart and honor the Retry-After header of a 429 response. With cURL, the equivalent is --connect-timeout 10 --max-time 30 --retry 3.

One session per proxy

Keep one client per proxy (a requests.Session, an httpx.Client, a ProxyAgent or an axios instance): connections get reused and identities stay separate, because one account's cookies should never travel through another account's IP. For data collection, respect each website's terms of service, its robots.txt and the GDPR; our guide to proxies for web scraping explains how to spread the load.

Troubleshooting: 407, timeouts, certificates

407 Proxy Authentication Required

The proxy did not receive valid credentials. Check that they were copied without spaces or line breaks, that special characters are encoded, that they are actually present in the URL or option you use, and that the access is active in your customer dashboard. With SOCKS5, the client reports an authentication failure instead of a 407 code. curl -v shows exactly which step fails.

Timeouts

A connect timeout usually means the proxy cannot be reached: wrong host or port, or a firewall blocking that outbound port, which is common on corporate networks. A read timeout points instead to a slow website or too many simultaneous requests. Check the access with our proxy checker: if it responds normally, the problem lies elsewhere.

Certificate errors

Inside an HTTP proxy's tunnel, encryption runs end to end: your tool receives the website's certificate, not the proxy's. A certificate error therefore comes from somewhere else:

• a proxy URL starting with https://: the tool attempts an encrypted connection to the proxy itself, which expects plain HTTP (an SSL error, often “wrong version number”). Switch to http://;
• TLS inspection by an antivirus or a corporate firewall: register its root certificate (--cacert for cURL, the verify parameter in requests, the NODE_EXTRA_CA_CERTS variable for Node.js).

Never turn off verification (-k, verify=False) in production: your traffic could then be intercepted.

Airproxy's dedicated ISP proxies (France, Spain and the EU offer) are delivered in host:port:username:password format and answer over HTTP(S) and SOCKS5 on the same host and port, so these examples apply as they are. For a browser or desktop software, follow our guide on how to set up a proxy instead. Available locations are listed on the offers page.