apigenie-generator client python (AsyncAPI)
Tier 2 · Release 0.12.0: Level 1 ✓ · Level 2 ✓ · Level 3 ✓ · Verification
Generate a Python asyncio Kafka client package for the Kafka channels of an AsyncAPI document.
Usage
apigenie-generator client python [...OPTIONS] INPUT_FILE
Description
apigenie-generator client python generates from OpenAPI documents (see
apigenie-generator client python) and from AsyncAPI 3.x
documents, telling them apart by their openapi or asyncapi field. Documents referencing other
files are bundled first.
From an AsyncAPI document it generates a Python package (Python 3.10 or later) on
aiokafka, with Pydantic 2
models of the payloads: the models of
apigenie-generator server faststream. By default the
client is the counterpart of the application the document describes: it sends the messages of
the application’s receive operations and receives those of its send operations. Set the
perspective option to application to generate the described application’s side instead.
The generator writes the Kafka channels: the channels of servers with the kafka or
kafka-secure protocol, and channels with a Kafka binding in documents without servers. Channels
of other protocols are skipped and reported as info.
With perspective: application, the client of the Carpet Express Event Platform sends the
shipment events and requests labels:
options = CarpetExpressEventsKafkaClientOptions(bootstrap_servers="localhost:9092")
async with CarpetExpressEventsKafkaClient(options) as client:
await client.send_shipment_delivered(delivered)
# request/reply: returns the reply carrying the request's correlation id
reply = await client.request_label_render(request, timeout=10.0)
if isinstance(reply, messages.LabelRendered):
print(reply.payload.url)
In the default perspective, it receives them and renders the labels:
class LabelFarm(RequestLabelRenderHandler):
async def on_render_label(self, message: messages.RenderLabel) -> messages.RequestLabelRenderReply | None:
# sent to the labelRenderResults topic with the request's correlation id
return messages.LabelRendered(payload=render(message.payload), key=message.key)
subscription = await client.listen_request_label_render(LabelFarm())
await subscription.assigned() # once the consumer group assigned partitions
The package is the snake case of --package-name, or of the document’s title followed by
_client; the distribution name is its kebab case:
| File | Contents |
|---|---|
models.py, _models.py |
the Pydantic models of the payloads |
messages.py |
a dataclass per message (payload, headers, key), its headers model with the constant headers (CE_TYPE, …), and a type alias per reply union |
client.py |
<Name>KafkaClient, <Name>KafkaClientOptions and a handler class per received operation |
_kafka.py, _runtime.py |
the Kafka runtime (records, subscriptions, replies) and the support of headers and correlation ids |
security.py |
with security schemes: a function per scheme returning its aiokafka settings, and security_from_environment(scheme) |
avro.py, _avro.py |
with Avro payloads: the Pydantic models of the records, and their codec on fastavro with the schema registry |
pyproject.toml |
the distribution, requiring aiokafka and Pydantic (and fastavro with Avro payloads) |
- The client. It is an async context manager;
start()starts a producer per sending and replying operation, with the operation’s client id, andclose()stops the subscriptions, the reply consumer and the producers. The options default to the document’s server, topics, client ids and group ids. They also take the reply client id and timeout (30 seconds), thesecurityscheme, theschema_registryof the Avro payloads it sends, and keyword arguments of the aiokafka producers and consumers that replace the client’s. - Sent operations.
send_<message>(message)writes the message’s key, its headers (constant headers such as thece_specversionandce_typeof CloudEvents included) and its JSON payload to the operation’s topic, and returns once the broker acknowledged it. - Request/reply with a reply channel:
request_<operation>(message, *, timeout=None)sets a new correlation id unless the message has one (in its header, at its payload pointer, or inkafka_correlationId), sends the request and returns the reply, or raisesTimeoutError. With several reply messages, the reply is one of them. One consumer, without a consumer group, receives the replies of all requests and matches them by correlation id. An operation named after the verb isn’t repeated:requestLabelRendergivesrequest_label_render. - Received operations.
listen_<operation>(handler)consumes the operation’s topic in a task, in its consumer group (new groups start at the first record), until the returned subscription or the client is closed. It commits the offsets of the records it handled.- The handler class (
PublishShipmentEventsHandler) has an abstracton_<message>method per message, anon_unknownand anon_error. - Several messages are told apart by a header or payload property with a distinct constant
value (e.g.
ce_type). Otherwise the first message whose payload validates without unknown properties is chosen. - Records that aren’t valid messages, and exceptions of the handler, go to
on_error. - A reply returned by the handler is sent to the reply topic with the request’s correlation id.
- The handler class (
Messages are validated by their models when they are built and when they are received; headers
models ignore the record’s other headers. The package passes mypy --strict.
Security
security.py has a function per SASL (scramSha512, scramSha256, plain, userPassword) or
client certificate (X509) scheme of the Kafka servers, returning its aiokafka settings
(security_protocol, sasl_mechanism, sasl_plain_username, ssl_context, …), with TLS on
kafka-secure servers. The security option names the scheme the producers and consumers connect
with (security="kafkaScram"); security_from_environment(scheme) reads its credentials from
environment variables named after the scheme: KAFKA_SCRAM_USERNAME and KAFKA_SCRAM_PASSWORD, or
the paths of the PEM files in <KEY>_CERT, <KEY>_KEY and <KEY>_CA. The settings in
producer_options and consumer_options replace the scheme’s.
Avro payloads
A message with an Avro payload (schemaFormat: application/vnd.apache.avro;version=1.9.0) carries a
Pydantic model of its record (avro.CarrierScan), with enums of its enum types, written and read
with fastavro. The client sends the schema id the
schema_registry option gives, by default a ConfluentSchemaRegistry of the schemaRegistryUrl of
the first server whose Kafka binding has one, which registers the schema under the subject of the message’s
schemaLookupStrategy (TopicIdStrategy by default). The id is in the payload (the Confluent
framing) or in a header, as the message’s Kafka binding says (schemaIdLocation); the schema in the
document reads the received records. A root schema that isn’t a record, records with decimal
fields and unions other than ["null", T] are skipped with a warning.
Not supported, and reported as warnings: payloads of other non-JSON schema formats and Avro messages of request/reply operations (the messages are skipped, and operations left without a message too), reply addresses given by runtime expressions, and Kafka topics with parameters.
Extensions
| Extension | Effect |
|---|---|
x-apigenie.codegen.name |
name of the model class of a component schema |
x-enum-varnames, x-enumNames |
names of the members of an enum |
x-apigenie.codegen.disabled |
true on a channel or an operation leaves it out (with the operations of a channel left out) |
The extensions can live in an overlay for this target instead of in the document, e.g.
python.overlay.yaml:
overlay: 1.0.0
info:
title: Python names
version: 1.0.0
actions:
- target: $.components.schemas.Order
update:
x-apigenie:
codegen:
name: PurchaseOrder
- target: $.components.schemas.OrderStatus
update:
x-enum-varnames: [CREATED, PAID, SHIPPED]
apigenie-generator client python asyncapi.yaml --overlay python.overlay.yaml -o out
Options
-c, –config
Path to the configuration file to use. See the “Configuration” section for more information.
–overlay OVERLAY
Path to an OpenAPI Overlay (1.0 to 1.2) applied to the input before generating, e.g. one setting the
extensions of the “Extensions” section. Can be repeated: the overlays apply in order, after those
listed under commands.generate.client_python.overlay in the configuration file (with paths
relative to it). Warnings and errors about a value an overlay set point to the overlay. The overlays
apply to the AsyncAPI document bundled with the files it references.
–strict
Makes an overlay action whose target selects nothing an error (exit status 2, nothing is generated)
instead of a warning (OVERLAY:LINE:COL: action #N (TARGET) selects no nodes.).
–check
Writes nothing: compares what the generator would write with the output directory, prints the files that would be created or changed, and exits with 1 when there are any (since 0.12.0). See Check that generated code is up to date.
-o, –output
Directory where the package is generated.
–package-name
Name of the distribution, by default the document’s title followed by -client. Its kebab case
is the base of default group and client ids, and its Pascal case the base of the class names.
–package-version
Version of the distribution; defaults to the version of the API.
Configuration
The options are read from the asyncapi section of the configuration of
apigenie-generator client python; its other keys configure the generation from OpenAPI documents.
version: '1.0'
commands:
generate:
client_python:
asyncapi:
perspective: counterpart
enum_style: literal
| Option | Description |
|---|---|
perspective |
counterpart (default): the counterpart of the described application; application: the described application, sending what it sends |
enum_style, field_prefix, class_overrides |
Settings of the Pydantic models, as for apigenie-generator client python |
Verification
The codegen conformance suite runs this target on the Carpet Express AsyncAPI fixture. Level 1 compares the output with its snapshots and checks each feature, Level 2 runs the language's type checker and linter, and Level 3 runs the generated code in contract tests against the fixture. All targets.
This target's support tier is Tier 2. It goes through all three levels before every release, like Tier 1. It doesn't aim to support every feature: a feature of the fixture can stay unsupported, with the reason its manifest gives.
| Results | Level 1 | Level 2 | Level 3 |
|---|---|---|---|
Release 0.12.0, client python | ✓ snapshots of 3.0; 31 supported, 0 not yet | ✓ pass (python 3.12.11 aiokafka 0.14.0 pydantic 2.13.5 mypy 2.4.0, ruff 0.16.10) | ✓ 13 of 13 operations pass (python 3.12.11 aiokafka 0.14.0 pydantic 2.13.5 mypy 2.4.0, ruff 0.16.10) |
Release 0.12.0, client python (application) | ✓ snapshots of 3.0; 31 supported, 0 not yet | ✓ pass (python 3.12.11 aiokafka 0.14.0 pydantic 2.13.5 mypy 2.4.0, ruff 0.16.10) | ✓ 13 of 13 operations pass (python 3.12.11 aiokafka 0.14.0 pydantic 2.13.5 mypy 2.4.0, ruff 0.16.10) |
Development version, client python | ✓ snapshots of 3.0; 31 supported, 0 not yet | runs before a release | runs before a release |
Development version, client python (application) | ✓ snapshots of 3.0; 31 supported, 0 not yet | runs before a release | runs before a release |