Remember the days when you could connect to a web server using telnet or netcat and manually type HTTP requests? That interactive experience was a great way to understand how HTTP/1.1 worked under the hood. With HTTP/2 and HTTP/3, that direct interaction became impossible because both protocols use binary framing instead of plain text.
In this post I’m exploring how to build interactive HTTP/2 and HTTP/3 clients in Python - not for any production use but to better understand what’s going on in those connections.
Binary Protocols
When learning and debugging HTTP/1.1 you could open a connection and type:
GET / HTTP/1.1
Host: example.com
The server would respond in plain text, and you could see exactly what was happening. This made HTTP/1.1 easy to understand, debug, and teach.
HTTP/2 changed this by introducing binary framing. Instead of text-based requests and responses, HTTP/2 splits communications into frames with specific binary structures. While this brought significant performance improvements (multiplexing, header compression, server push), it meant you could no longer just type requests manually.
HTTP/3 went even further by running over QUIC instead of TCP, using UDP as the underlying transport. This brought additional benefits like improved connection establishment and better handling of packet loss, but it also meant an even more complex binary protocol.
HTTP/2: Building with HTTPX
For HTTP/2, the HTTPX library provides excellent support. HTTPX is a modern, feature-rich HTTP client for Python with a familiar API similar to the popular requests library.
Setting Up HTTP/2 Support
First, install HTTPX with HTTP/2 support:
pip install httpx[http2]
This installs the h2 library which provides the HTTP/2 protocol implementation.
Using the HTTPX Command-Line Tool
Before diving into custom Python scripts, HTTPX provides a built-in command-line tool that works like curl but with HTTP/2 support. Install it with:
pip install 'httpx[cli]'
Now you can use the httpx command directly from your terminal:
# Basic GET request
httpx https://www.google.com
# Verbose output showing headers and connection details
httpx --verbose https://www.google.com
# Specify HTTP method
httpx --method POST https://httpbin.org/post
# Add custom headers
httpx https://httpbin.org/headers --headers X-Custom-Header example-value
# Follow redirects
httpx --follow-redirects https://github.com
The CLI tool is perfect for quick testing and learning about HTTP/2 behavior without writing code. However, for more control and educational purposes, building your own client reveals more about how the protocols work.
A Simple Interactive HTTP/2 Client
Here’s a basic interactive client that lets you make HTTP/2 requests:
#!/usr/bin/env python3
import httpx
import sys
def interactive_http2_client():
print("Interactive HTTP/2 Client")
print("=" * 40)
# Create a client with HTTP/2 enabled
with httpx.Client(http2=True) as client:
while True:
try:
# Get URL from user
url = input("\nEnter URL (or 'quit' to exit): ").strip()
if url.lower() == 'quit':
break
# Get method from user
method = input("Method (GET/POST/PUT/DELETE) [GET]: ").strip().upper() or "GET"
# Get headers if desired
headers = {}
print("Enter headers (key: value), empty line to finish:")
while True:
header = input(" ").strip()
if not header:
break
if ':' in header:
key, value = header.split(':', 1)
headers[key.strip()] = value.strip()
# Make the request
print(f"\n→ {method} {url}")
response = client.request(method, url, headers=headers)
# Display response details
print(f"\n← HTTP/{response.http_version} {response.status_code} {response.reason_phrase}")
print("\nResponse Headers:")
for key, value in response.headers.items():
print(f" {key}: {value}")
print(f"\nContent Length: {len(response.content)} bytes")
# Show body if it's text
content_type = response.headers.get('content-type', '')
if 'text' in content_type or 'json' in content_type or 'xml' in content_type:
show_body = input("\nShow response body? (y/N): ").strip().lower()
if show_body == 'y':
print("\n" + "-" * 40)
print(response.text)
print("-" * 40)
except KeyboardInterrupt:
print("\nExiting...")
break
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
if __name__ == "__main__":
interactive_http2_client()
What Makes This HTTP/2?
The key is the http2=True parameter when creating the client. HTTPX automatically negotiates the protocol version using ALPN (Application-Layer Protocol Negotiation) during the TLS handshake. If the server supports HTTP/2, the connection will use it; otherwise, it falls back to HTTP/1.1.
You can verify the HTTP version in use by checking response.http_version:
print(f"HTTP Version: {response.http_version}") # "HTTP/2" or "HTTP/1.1"
Enhanced Version with Connection Reuse
One of HTTP/2’s key features is connection multiplexing - multiple requests can share a single connection. Here’s an enhanced version that demonstrates this:
#!/usr/bin/env python3
import httpx
import sys
from datetime import datetime
def enhanced_http2_client():
print("Enhanced HTTP/2 Client")
print("Demonstrates connection reuse and multiplexing")
print("=" * 50)
with httpx.Client(http2=True) as client:
request_count = 0
while True:
try:
url = input("\nURL (or 'quit'): ").strip()
if url.lower() == 'quit':
break
method = input("Method [GET]: ").strip().upper() or "GET"
start_time = datetime.now()
response = client.request(method, url)
duration = (datetime.now() - start_time).total_seconds()
request_count += 1
print(f"\n[Request #{request_count}]")
print(f"HTTP Version: {response.http_version}")
print(f"Status: {response.status_code} {response.reason_phrase}")
print(f"Duration: {duration:.3f}s")
# Show if connection was reused
if hasattr(response.extensions, 'get'):
stream_id = response.extensions.get('http2_stream_id')
if stream_id:
print(f"HTTP/2 Stream ID: {stream_id}")
print(f"Content Length: {len(response.content)} bytes")
except KeyboardInterrupt:
print("\n\nSession Statistics:")
print(f"Total requests: {request_count}")
break
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
if __name__ == "__main__":
enhanced_http2_client()
Advanced Version: Viewing Binary HTTP/2 Frames
One of the most educational aspects of working with HTTP/2 is seeing the actual binary frames being exchanged. Here’s an enhanced client that displays real HTTP/2 frames in a hexdump format similar to xxd, showing both the binary data and its ASCII representation.
To capture actual frame data, we need to intercept the socket-level communication. This requires creating a custom transport that wraps the default httpcore transport:
#!/usr/bin/env python3
import httpx
import httpcore
import sys
from typing import Iterable, Optional
def hexdump(data, offset=0, prefix=""):
"""Format binary data as xxd-style hexdump"""
lines = []
for i in range(0, len(data), 16):
chunk = data[i:i+16]
hex_part = ' '.join(f'{b:02x}' for b in chunk)
# Pad hex part to align ASCII
hex_part = hex_part.ljust(47)
# ASCII representation
ascii_part = ''.join(chr(b) if 32 <= b < 127 else '.' for b in chunk)
lines.append(f"{prefix}{offset+i:08x}: {hex_part} {ascii_part}")
return '\n'.join(lines)
# Frame type names for display
FRAME_TYPES = {
0x00: "DATA",
0x01: "HEADERS",
0x02: "PRIORITY",
0x03: "RST_STREAM",
0x04: "SETTINGS",
0x05: "PUSH_PROMISE",
0x06: "PING",
0x07: "GOAWAY",
0x08: "WINDOW_UPDATE",
0x09: "CONTINUATION",
}
def parse_frame_header(data):
"""Parse HTTP/2 frame header (first 9 bytes)"""
if len(data) < 9:
return None
length = (data[0] << 16) | (data[1] << 8) | data[2]
frame_type = data[3]
flags = data[4]
stream_id = ((data[5] & 0x7F) << 24) | (data[6] << 16) | (data[7] << 8) | data[8]
return {
'length': length,
'type': frame_type,
'type_name': FRAME_TYPES.get(frame_type, f"UNKNOWN({frame_type})"),
'flags': flags,
'stream_id': stream_id,
}
class LoggingNetworkStream(httpcore.NetworkStream):
"""Wrapper around a network stream that logs all data"""
def __init__(self, stream: httpcore.NetworkStream):
self._stream = stream
self._frame_count = 0
def read(self, max_bytes: int, timeout: Optional[float] = None) -> bytes:
"""Read data and log it"""
data = self._stream.read(max_bytes, timeout)
if data:
self._log_data(data, "←")
return data
def write(self, buffer: bytes, timeout: Optional[float] = None) -> None:
"""Write data and log it"""
if buffer:
self._log_data(buffer, "→")
return self._stream.write(buffer, timeout)
def _log_data(self, data: bytes, symbol: str):
"""Log frame data with parsing"""
# Skip TLS handshake data (starts with 0x16 for handshake)
if len(data) > 0 and data[0] == 0x16:
print(f"\n{symbol} TLS Handshake Data ({len(data)} bytes)")
return
# Look for HTTP/2 connection preface
if data.startswith(b'PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n'):
print(f"\n{symbol} HTTP/2 Connection Preface")
print("=" * 70)
print(hexdump(data, prefix=" "))
print("=" * 70)
return
offset = 0
data_bytes = bytes(data) if not isinstance(data, bytes) else data
while offset < len(data_bytes):
# Try to parse frame header
frame_info = parse_frame_header(data_bytes[offset:])
if frame_info and frame_info['length'] <= 16384: # Max frame size
frame_length = 9 + frame_info['length']
if offset + frame_length <= len(data_bytes):
frame_data = data_bytes[offset:offset+frame_length]
self._frame_count += 1
print(f"\n{symbol} Frame #{self._frame_count} - {frame_info['type_name']}")
print(f" Length: {frame_info['length']}, Flags: 0x{frame_info['flags']:02x}, "
f"Stream: {frame_info['stream_id']}")
print("=" * 70)
print(hexdump(frame_data, prefix=" "))
print("=" * 70)
offset += frame_length
continue
# Can't parse or remaining data - dump it
remaining = data_bytes[offset:]
if remaining and len(remaining) < 100: # Only show small chunks
print(f"\n{symbol} Raw data ({len(remaining)} bytes)")
print("=" * 70)
print(hexdump(remaining, prefix=" "))
print("=" * 70)
break
def close(self) -> None:
return self._stream.close()
def start_tls(
self,
ssl_context,
server_hostname: Optional[str] = None,
timeout: Optional[float] = None,
) -> "LoggingNetworkStream":
"""Start TLS and wrap the resulting stream"""
new_stream = self._stream.start_tls(ssl_context, server_hostname, timeout)
return LoggingNetworkStream(new_stream)
def get_extra_info(self, info: str):
return self._stream.get_extra_info(info)
class LoggingNetworkBackend(httpcore.NetworkBackend):
"""Custom network backend that wraps streams with logging"""
def __init__(self, backend: Optional[httpcore.NetworkBackend] = None):
self._backend = backend or httpcore.SyncBackend()
def connect_tcp(
self,
host: str,
port: int,
timeout: Optional[float] = None,
local_address: Optional[str] = None,
socket_options: Optional[Iterable[httpcore.SOCKET_OPTION]] = None,
) -> httpcore.NetworkStream:
"""Connect and wrap the stream with logging"""
print(f"\n{'=' * 70}")
print(f"Connecting to {host}:{port}")
print(f"{'=' * 70}")
stream = self._backend.connect_tcp(
host, port, timeout, local_address, socket_options
)
return LoggingNetworkStream(stream)
def connect_unix_socket(
self,
path: str,
timeout: Optional[float] = None,
socket_options: Optional[Iterable[httpcore.SOCKET_OPTION]] = None,
) -> httpcore.NetworkStream:
"""Connect to unix socket and wrap with logging"""
stream = self._backend.connect_unix_socket(path, timeout, socket_options)
return LoggingNetworkStream(stream)
def sleep(self, seconds: float) -> None:
return self._backend.sleep(seconds)
def http2_client_with_real_frame_dump():
"""HTTP/2 client that captures and displays real binary frame data"""
print("HTTP/2 Client with Real Binary Frame Capture")
print("=" * 70)
print("This client captures and displays actual HTTP/2 frames")
print()
# Create custom network backend with logging
network_backend = LoggingNetworkBackend()
# Create httpcore connection pool with custom backend
with httpcore.ConnectionPool(
http2=True,
network_backend=network_backend,
) as pool:
while True:
try:
url_input = input("\nURL (or 'quit'): ").strip()
if url_input.lower() == 'quit':
break
method = input("Method [GET]: ").strip().upper() or "GET"
print(f"\n{'=' * 70}")
print(f"Making {method} request to {url_input}")
print(f"{'=' * 70}")
# Parse URL and create request
url = httpcore.URL(url_input.encode('utf-8'))
headers = [
(b"host", url.host),
(b"user-agent", b"HTTP2-Frame-Dumper/1.0"),
]
request = httpcore.Request(
method=method.encode('utf-8'),
url=url,
headers=headers,
)
# Make the request - frames will be logged automatically
response = pool.handle_request(request)
# Show response summary
print(f"\n{'=' * 70}")
print("Response Summary")
print(f"{'=' * 70}")
print(f"Status: {response.status}")
print(f"\nResponse Headers:")
for name, value in response.headers:
print(f" {name.decode('utf-8', errors='replace')}: "
f"{value.decode('utf-8', errors='replace')}")
# Read response body
body_parts = []
for chunk in response.stream:
body_parts.append(chunk)
body = b''.join(body_parts)
print(f"\nContent Length: {len(body)} bytes")
# Offer to show response body
if len(body) > 0:
content_type = ''
for name, value in response.headers:
if name.lower() == b'content-type':
content_type = value.decode('utf-8', errors='replace')
break
if any(t in content_type for t in ['text', 'json', 'xml', 'html']):
show = input("\nShow response body? (y/N): ").strip().lower()
if show == 'y':
print("\n" + "-" * 70)
try:
text = body.decode('utf-8', errors='replace')[:2000]
print(text)
if len(body) > 2000:
print(f"\n... ({len(body) - 2000} more bytes)")
except Exception:
print(f"Binary data ({len(body)} bytes)")
print("-" * 70)
except KeyboardInterrupt:
print("\n\nExiting...")
break
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
import traceback
traceback.print_exc()
if __name__ == "__main__":
http2_client_with_real_frame_dump()
This implementation captures real HTTP/2 frames by:
- Custom Network Backend: Creates a
LoggingNetworkBackendthat extendshttpcore.NetworkBackend - Stream Wrapping: Wraps the
NetworkStreamwith aLoggingNetworkStreamthat intercepts read/write operations - Frame Parsing: Parses the 9-byte HTTP/2 frame header to identify frame types, lengths, flags, and stream IDs
- Real-time Display: Shows each frame as it’s sent or received with:
- Frame type (DATA, HEADERS, SETTINGS, etc.)
- Frame metadata (length, flags, stream ID)
- Complete hexdump of the binary data
- Direct httpcore Usage: Uses
httpcore.ConnectionPooldirectly with the custom backend for low-level control
The hexdump format shows:
- Offset: The byte position in hexadecimal
- Hex data: The raw bytes in hexadecimal (16 bytes per line)
- ASCII representation: Printable characters shown, dots for non-printable bytes
Each HTTP/2 frame has a 9-byte header containing:
- 3 bytes: payload length
- 1 byte: frame type (SETTINGS=0x04, HEADERS=0x01, DATA=0x00, etc.)
- 1 byte: flags
- 4 bytes: stream identifier
What you’ll see:
When you run this, you’ll see frames like:
- SETTINGS frames: Initial connection parameters
- WINDOW_UPDATE frames: Flow control
- HEADERS frames: Request/response headers (HPACK encoded)
- DATA frames: Actual response body data
Important Note about HTTPS/TLS:
When connecting to HTTPS servers, you might wonder: “Am I seeing encrypted or decrypted data?”
The answer: You’re seeing decrypted HTTP/2 frames.
Here’s why: The LoggingNetworkStream wrapper architecture works like this:
- Initial TCP connection → First
LoggingNetworkStreamwrapper (sees TLS handshake bytes) start_tls()is called → TLS encryption is applied to the underlying stream- New
LoggingNetworkStreamwrapper is created → This sits above the TLS layer
The network stack looks like:
[Your Code]
↕ (plaintext HTTP/2)
[LoggingNetworkStream] ← You are here (after start_tls)
↕ (plaintext HTTP/2)
[TLS Layer] ← Encryption/decryption happens here
↕ (encrypted bytes)
[TCP Socket]
↕ (encrypted bytes)
[Network]
This means the logging happens after TLS decryption on reads and before TLS encryption on writes. You see the actual binary HTTP/2 protocol data, not encrypted bytes. This is exactly what you want for understanding how HTTP/2 works - the TLS layer is transparent to your inspection.
If you wanted to see the encrypted bytes instead, you’d need to wrap the stream before start_tls() is called, or use a network capture tool like Wireshark on the raw socket.
This gives you a complete view of the HTTP/2 binary protocol in action, making the invisible visible.
Can HTTP/2 work without encryption (plain HTTP)?
Yes! This is called h2c (HTTP/2 Cleartext), but it’s rarely used in practice:
- Browsers don’t support it: Chrome, Firefox, Safari, and Edge all require HTTP/2 to use HTTPS
- Most servers don’t offer it: Since browsers won’t use it, public web servers typically don’t enable h2c
- Where it’s used: Internal microservices, testing, server-to-server APIs, and programmatic clients
For the frame dumping tool, plain HTTP actually works better because there’s no TLS layer to complicate things - you see every byte directly without any handshake noise.
If you want to test h2c, you’d need a server that supports it (like a local test server), and the httpcore client will use it automatically when connecting to an http:// URL instead of https://. The frame dumps will be even cleaner since there’s no TLS handshake to filter out.
Protocol Negotiation:
- With TLS (h2): Uses ALPN (Application-Layer Protocol Negotiation) during TLS handshake
- Without TLS (h2c): Either uses HTTP/1.1 Upgrade header or requires “prior knowledge” that the server supports HTTP/2
HTTP/3: Using aioquic
HTTP/3 is more challenging because it runs over QUIC/UDP rather than TCP. The aioquic library provides a QUIC and HTTP/3 implementation for Python.
Installing aioquic
pip install aioquic
A Basic HTTP/3 Client Example
Building an HTTP/3 client is more complex because aioquic follows a “bring your own I/O” pattern. Here’s a working example that properly handles the HTTP/3 protocol:
#!/usr/bin/env python3
import asyncio
import sys
from urllib.parse import urlparse
from typing import Dict, List, Tuple
from aioquic.asyncio.client import connect
from aioquic.asyncio.protocol import QuicConnectionProtocol
from aioquic.h3.connection import H3_ALPN, H3Connection
from aioquic.h3.events import HeadersReceived, DataReceived, H3Event
from aioquic.quic.configuration import QuicConfiguration
from aioquic.quic.events import QuicEvent
class HTTP3Client(QuicConnectionProtocol):
"""HTTP/3 client that collects response data"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._http = H3Connection(self._quic)
self._request_events: Dict[int, Dict] = {}
self._request_waiter: Dict[int, asyncio.Future] = {}
async def make_request(
self,
method: str,
url: str,
headers: List[Tuple[bytes, bytes]] = None,
content: bytes = b""
) -> Dict:
"""Make an HTTP/3 request and return response"""
parsed = urlparse(url)
path = parsed.path or "/"
if parsed.query:
path += f"?{parsed.query}"
# Prepare headers
request_headers = [
(b":method", method.encode()),
(b":scheme", parsed.scheme.encode()),
(b":authority", parsed.netloc.encode()),
(b":path", path.encode()),
]
if headers:
request_headers.extend(headers)
# Get stream ID and send request
stream_id = self._quic.get_next_available_stream_id()
self._http.send_headers(
stream_id=stream_id,
headers=request_headers,
end_stream=not content,
)
if content:
self._http.send_data(stream_id=stream_id, data=content, end_stream=True)
# Transmit
self.transmit()
# Prepare to collect response
self._request_events[stream_id] = {
"headers": [],
"data": b"",
"status": None,
}
waiter = self._loop.create_future()
self._request_waiter[stream_id] = waiter
print(f"\n→ {method} {path}")
print(f"Stream ID: {stream_id}")
# Wait for response
return await waiter
def quic_event_received(self, event: QuicEvent) -> None:
"""Handle QUIC events"""
# Process HTTP/3 events
for http_event in self._http.handle_event(event):
self._http_event_received(http_event)
def _http_event_received(self, event: H3Event) -> None:
"""Handle HTTP/3 events"""
if isinstance(event, HeadersReceived):
stream_id = event.stream_id
if stream_id in self._request_events:
headers = event.headers
self._request_events[stream_id]["headers"] = headers
# Extract status
for name, value in headers:
if name == b":status":
self._request_events[stream_id]["status"] = int(value.decode())
print(f"\n← Received HEADERS on stream {stream_id}")
print(f"Status: {self._request_events[stream_id]['status']}")
elif isinstance(event, DataReceived):
stream_id = event.stream_id
if stream_id in self._request_events:
self._request_events[stream_id]["data"] += event.data
print(f"← Received DATA on stream {stream_id}: {len(event.data)} bytes")
# Check if stream is finished
if event.stream_ended:
print(f"← Stream {stream_id} ended")
# Complete the request
if stream_id in self._request_waiter:
waiter = self._request_waiter.pop(stream_id)
if not waiter.done():
waiter.set_result(self._request_events[stream_id])
async def http3_request(url: str, method: str = "GET"):
"""Make an HTTP/3 request"""
parsed = urlparse(url)
host = parsed.hostname
port = parsed.port or 443
# Configure QUIC
configuration = QuicConfiguration(
alpn_protocols=H3_ALPN,
is_client=True,
verify_mode=None, # Don't verify certificates for testing
)
print(f"\nConnecting to {host}:{port} via HTTP/3...")
try:
async with connect(
host,
port,
configuration=configuration,
create_protocol=HTTP3Client,
) as client:
# client is an HTTP3Client instance due to create_protocol parameter
# Make the request
response = await client.make_request(method, url)
# Show response
print(f"\n{'=' * 70}")
print("Response Summary")
print(f"{'=' * 70}")
print(f"Status: {response['status']}")
print(f"\nResponse Headers:")
for name, value in response["headers"]:
if not name.startswith(b":"):
print(f" {name.decode('utf-8', errors='replace')}: "
f"{value.decode('utf-8', errors='replace')}")
print(f"\nContent Length: {len(response['data'])} bytes")
# Show body preview
if len(response['data']) > 0:
content_type = ""
for name, value in response["headers"]:
if name.lower() == b"content-type":
content_type = value.decode('utf-8', errors='replace')
break
if any(t in content_type for t in ['text', 'json', 'xml', 'html']):
show = input("\nShow response body? (y/N): ").strip().lower()
if show == 'y':
print("\n" + "-" * 70)
try:
text = response['data'].decode('utf-8', errors='replace')[:2000]
print(text)
if len(response['data']) > 2000:
print(f"\n... ({len(response['data']) - 2000} more bytes)")
except Exception:
print(f"Binary data ({len(response['data'])} bytes)")
print("-" * 70)
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
import traceback
traceback.print_exc()
async def interactive_http3_client():
"""Interactive HTTP/3 client shell"""
print("Interactive HTTP/3 Client")
print("=" * 40)
print()
while True:
try:
url = input("\nEnter URL (or 'quit' to exit): ").strip()
if url.lower() == 'quit':
break
method = input("Method [GET]: ").strip().upper() or "GET"
await http3_request(url, method)
except KeyboardInterrupt:
print("\nExiting...")
break
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
if __name__ == "__main__":
try:
asyncio.run(interactive_http3_client())
except KeyboardInterrupt:
print("\nExiting...")
For more information have a look at the examples in the aioquic repository.
Conclusion
While we can’t use telnet to interact with HTTP/2 and HTTP/3 servers, Python libraries like HTTPX and aioquic let us build interactive clients that restore that hands-on learning experience. HTTPX makes HTTP/2 accessible with a simple, familiar API, while aioquic provides the foundation for HTTP/3 exploration.
These tools not only help us understand modern HTTP protocols but also enable practical debugging, testing, and educational applications. As HTTP/3 adoption grows and tooling matures, the experience will only get better.
Whether you’re learning about protocol evolution, debugging server behavior, or just curious about how modern web protocols work, building your own interactive client is an excellent way to get hands-on experience with HTTP/2 and HTTP/3.
References
- HTTPX Documentation
- HTTPX HTTP/2 Support
- HTTPX PyPI Package
- HTTPX GitHub Repository
- httpcore Documentation
- aioquic - QUIC and HTTP/3 implementation in Python
- python-hyper/hyper - HTTP/2 for Python
- Getting Started with HTTP/3 in Python
- Getting Started with HTTPX: Python’s Modern HTTP Client
- How to use httpx, a web client for Python
- Python http.client Documentation