Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion docs/source/getting-started/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,12 +81,46 @@ data = await response.json()
print(data)
```

### Binary data

`response.bytes()` returns a read-only `memoryview`, not a `bytes` object. The view shares Rust-owned data without a copy into Python bytes. Reading a complete response can still allocate memory to combine body chunks.

```python
import hashlib

view = await response.bytes()
await response.close()
print(view.readonly) # True; closing the response does not invalidate the view
print(hashlib.sha256(view).hexdigest()) # Reads the buffer directly
```

The blocking API returns the same type, without `await`. Stream data frames, WebSocket binary fields, header names and values, and peer certificates also return read-only memoryviews. Each view retains its backing data even after the source object is closed or deleted.

Use views directly with APIs that accept the buffer protocol, such as `file.write(view)` or `hashlib.sha256(view)`. For text, `str(view, "utf-8")` decodes into a string without an intermediate `bytes` object.

#### Copying data

Only convert when you need an independent `bytes` object or an API requires one:

```python
data = bytes(view) # Copies the data; view.tobytes() also copies
view.release()
```

This changes the binary return type. `memoryview` has no `.decode()` method or byte-string concatenation. When you finish using a view, you can call `view.release()`; this does not release other views or slices sharing the data. Input types are unchanged.

Built-in `bytes` and `str` inputs can share their storage. Their subclasses are copied from the actual contents to avoid hidden reference cycles; deleting a view releases its ownership normally, without requiring an explicit `release()`.

When passing a view back to wreq's binary inputs (`body`, `Part`, `Message` constructors, or `CertStore`), convert it with `bytes(view)`. These inputs do not treat a memoryview as binary data.

### Response headers

Response headers are available as a [HeaderMap](../api/header/?h=HeaerMap#wreq.header.HeaderMap) object:

```python
print(response.headers.get("content-type"))
content_type = response.headers.get("content-type")
if content_type is not None:
print(str(content_type, "ascii"))
# application/json
```

Expand Down
19 changes: 14 additions & 5 deletions docs/source/guide/basic.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,13 +140,17 @@ headers.append("Accept", "application/json")
headers.append("Accept", "text/html")

# Retrieve a single value
print(headers.get("Content-Type"))
content_type = headers.get("Content-Type")
if content_type is not None:
print(str(content_type, "ascii"))
# application/json

# Retrieve all values for a multi-value header
print(list(headers.get_all("Accept")))
print([str(value, "ascii") for value in headers.get_all("Accept")])
# ['application/json', 'text/html']
```

Header names and values are read-only `memoryview` objects. Decode text with `str(view, encoding)` without first copying it into `bytes`.

Pass the `HeaderMap` to any request method via the `headers` argument:

Expand All @@ -161,16 +165,21 @@ response = await wreq.get("https://httpbin.org/headers", headers=headers)
For large responses, you can read the body incrementally instead of loading it all into memory at once. Use `resp.stream()` as an async iterator:

```python
from wreq import Client
import sys

from wreq import Client, HeaderMap

async def main():
client = Client()
response = await client.get("https://httpbin.org/stream/10")

async for chunk in response.stream():
print(chunk.decode("utf-8"))
if isinstance(chunk, memoryview):
sys.stdout.buffer.write(chunk)
elif isinstance(chunk, HeaderMap):
print("Trailers:", chunk)
```

Each `chunk` is a `bytes` object. Decode it to a string only if you know the response body is text.
Data chunks are read-only `memoryview` objects; trailer frames are `HeaderMap` objects. The example writes data directly through the buffer protocol. Each view stays valid after the stream is closed. See [Binary data](../getting-started/quickstart.md#binary-data) for copying and releasing views.

---
9 changes: 8 additions & 1 deletion docs/source/guide/blocking.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,8 @@ if __name__ == "__main__":
### Streaming Response

```python
import sys

from wreq.blocking import Client


Expand All @@ -175,9 +177,14 @@ def main():
with resp:
with resp.stream() as streamer:
for chunk in streamer:
print(chunk)
if isinstance(chunk, memoryview):
sys.stdout.buffer.write(chunk)
else:
print("Trailers:", chunk)


if __name__ == "__main__":
main()
```

Data chunks are read-only `memoryview` objects that stay valid after the stream is closed. `resp.bytes()` returns the same type. Pass views directly to APIs that accept the buffer protocol; use `bytes(view)` or `view.tobytes()` only when you need a copy.
2 changes: 2 additions & 0 deletions docs/source/guide/websocket.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
- HTTP/1.1 WebSocket
- HTTP/2 WebSocket

`Message.data`, `.binary`, `.ping`, and `.pong` return read-only `memoryview` objects when present. The views remain valid after the message is deleted or the connection is closed. Use `bytes(view)` when a consumer requires a `bytes` object; `Message.text` still returns a string.

### HTTP/1.1 WebSocket Connection

```python
Expand Down
6 changes: 5 additions & 1 deletion examples/blocking/stream.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import sys
import time

import wreq
Expand All @@ -8,7 +9,10 @@ def main():
with wreq.blocking.get("https://httpbin.io/stream/20") as resp:
with resp.stream() as streamer:
for chunk in streamer:
print(chunk)
if isinstance(chunk, memoryview):
sys.stdout.buffer.write(chunk)
else:
print("Trailers:", chunk)
time.sleep(0.1)


Expand Down
6 changes: 4 additions & 2 deletions examples/header_map.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@
# Add Accept header (second value)
headers.insert("Accept", "text/html")
# Get all values for 'Accept' header
print("All Accept:", list(headers.get_all("Accept")))
print("All Accept:", [str(value, "ascii") for value in headers.get_all("Accept")])
# Get the value for 'Content-Type' header
print("Content-Type:", headers.get("Content-Type"))
content_type = headers.get("Content-Type")
if content_type is not None:
print("Content-Type:", str(content_type, "ascii"))
# Print total number of values in the map
print("len (all values):", headers.len())
# Print number of unique keys in the map
Expand Down
10 changes: 6 additions & 4 deletions examples/request.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,15 @@ async def main():
print("Cookies: ", resp.cookies)
print("Content-Length: ", resp.content_length)
print("Remote Address: ", resp.remote_addr)
print("Headers set-cookie: ", resp.headers["set-cookie"])
set_cookie = resp.headers["set-cookie"]
if set_cookie is not None:
print("Headers set-cookie: ", str(set_cookie, "latin-1"))

for key in resp.headers:
print(key)
for key in resp.headers.keys():
print(str(key, "ascii"))

for key, value in resp.headers:
print(f"{key}: {value}")
print(f"{str(key, 'ascii')}: {str(value, 'latin-1')}")

for cookie in resp.cookies:
print(cookie)
Expand Down
6 changes: 5 additions & 1 deletion examples/stream.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import asyncio
import sys
import wreq
from wreq import Response

Expand All @@ -8,7 +9,10 @@ async def main():
async with resp:
async with resp.stream() as streamer:
async for chunk in streamer:
print(chunk)
if isinstance(chunk, memoryview):
sys.stdout.buffer.write(chunk)
else:
print("Trailers:", chunk)
await asyncio.sleep(0.1)


Expand Down
8 changes: 5 additions & 3 deletions python/wreq/blocking.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ def raise_for_status(self) -> None:

def stream(self) -> Streamer:
r"""
Get the response into a `Streamer` of `bytes` from the body.
Stream the body as read-only memoryviews, with HeaderMap frames for trailers.
"""
...

Expand All @@ -104,9 +104,11 @@ def json(self) -> Any:
Get the JSON content of the response.
"""

def bytes(self) -> bytes:
def bytes(self) -> memoryview:
r"""
Get the bytes content of the response.
Read the body as a read-only memoryview without copying it into Python bytes.
The view remains valid after the response is closed or deleted.
Use bytes(view) or view.tobytes() for a copy; view.release() releases this view.
"""
...

Expand Down
33 changes: 19 additions & 14 deletions python/wreq/header.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,12 @@ class HeaderMap:
The implementation follows HTTP/1.1 specifications for header handling
and provides both dictionary-like access and specialized methods for
HTTP header manipulation.

Header names and values are returned as read-only memoryviews.
Each view retains its backing data even if the map is changed or deleted.
"""

def __getitem__(self, key: str) -> bytes | None:
def __getitem__(self, key: str) -> memoryview | None:
"""Get the first value for a header name (case-insensitive)."""
...

Expand All @@ -49,7 +52,7 @@ def __len__(self) -> int:
"""Return the total number of header values (not unique names)."""
...

def __iter__(self) -> Iterator[Tuple[bytes, bytes]]:
def __iter__(self) -> Iterator[Tuple[memoryview, memoryview]]:
"""Iterate all header(name, value) pairs, including duplicates for multiple values."""
...

Expand Down Expand Up @@ -140,7 +143,7 @@ def remove(self, key: str) -> None:
"""
...

def get(self, key: str, default: bytes | None = None) -> bytes | None:
def get(self, key: str, default: bytes | None = None) -> memoryview | None:
r"""
Get the first value for a header name with optional default.

Expand All @@ -153,41 +156,41 @@ def get(self, key: str, default: bytes | None = None) -> bytes | None:
default: Value to return if header doesn't exist

Returns:
The first header value as bytes, or the default value
A read-only view of the first header value, or of the default value
"""
...

def get_all(self, key: str) -> Iterator[bytes]:
def get_all(self, key: str) -> list[memoryview]:
r"""
Get all values for a header name.

Returns an iterator over all values associated with the header name.
Returns a list of read-only views of all values associated with the header name.
This is useful for headers that can have multiple values, such as
Set-Cookie, Accept-Encoding, or custom headers.

Args:
key: The header name (case-insensitive)

Returns:
An iterator over all header values
A list of read-only header value views
"""
...

def values(self) -> Iterator[bytes]:
def values(self) -> list[memoryview]:
"""
Iterate over all header values.
Get all header values.

Returns:
An iterator over all header values as bytes.
A list of read-only header value views.
"""
...

def keys(self) -> Iterator[bytes]:
def keys(self) -> list[memoryview]:
"""
Iterate over unique header names.
Get all unique header names.

Returns:
An iterator over unique header names as bytes.
A list of read-only header name views.
"""
...

Expand Down Expand Up @@ -247,6 +250,8 @@ class OrigHeaderMap:
The map stores a mapping between the case-insensitive (standard) header name and the
original case-sensitive header name as it appeared in the HTTP message.

Iteration returns pairs of read-only memoryviews that retain their backing data.

Example:
If an HTTP message included the following headers:

Expand Down Expand Up @@ -277,7 +282,7 @@ def __init__(
"""
...

def __iter__(self) -> Iterator[Tuple[bytes, bytes]]:
def __iter__(self) -> Iterator[Tuple[memoryview, memoryview]]:
"""
Returns an iterator over the (standard_name, original_name) pairs.

Expand Down
4 changes: 2 additions & 2 deletions python/wreq/tls.py
Original file line number Diff line number Diff line change
Expand Up @@ -446,8 +446,8 @@ class TlsInfo:
Information about the established TLS connection.
"""

def peer_certificate(self) -> bytes | None:
def peer_certificate(self) -> memoryview | None:
"""
Get the DER encoded leaf certificate of the peer.
Get a read-only memoryview of the peer's DER-encoded leaf certificate.
"""
...
Loading
Loading