Connections

Until now, the lessons printed generated values locally. A connection tells Synthtraffic where to send those events.

Connections have two parts:

1. connections contains named, reusable destination settings such as Kafka brokers or an HTTP base URL.
2. Each generator selects one of those names with connection and provides the fields for one event, such as a Kafka topic and value.

This keeps credentials and client settings separate from the data generated for every event.

Shape an event for Kafka

Kafka needs more than the order value. It also needs a topic and, optionally, a key and headers. We want the preview to show this envelope:

{
  "topic": "shop.orders",
  "key": "ORD-000",
  "headers": null,
  "value": {
    "orderId": "ORD-000",
    "status": "placed",
    "total": 49.99
  }
}

Orders ready for a Kafka topic

connections:
  eventBus:
    type: kafka
    brokers: [localhost:9092]

generators:
  - name: orders
    connection: eventBus
    config:
      maxEvents: 2
    vars:
      orderId: =seq(format=ORD-%03d)
    topic: shop.orders
    key: $orderId
    value:
      orderId: $orderId
      status: placed
      total: =cycle(49.99, 84.50)

Press Try it. Previewing does not contact localhost:9092; it safely shows the envelope above.

Connection settings and event fields

The introduced lines belong to those two parts:

• connections.eventBus names reusable destination settings.
• type: kafka selects the Kafka connection behavior.
• brokers lists the bootstrap address.
• connection: eventBus selects that connection for orders.
• topic, key, and value describe the event sent to Kafka.

The shared $orderId comes from vars, so the Kafka key and payload always identify the same order. Expressions are calculated before delivery; the connection receives finished fields and does not change them.

The event fields depend on the destination:

• Kafka: topic, optional key and headers, and value
• PostgreSQL: table, row, and optional schema
• HTTP: optional method, required path, optional query and headers, and optional body
• File and cloud storage: prefix and value

Do not move these fields into connections. One connection can be reused by generators that publish to different topics, tables, paths, or artifact prefixes.

Preview before sending

Use sample while shaping an event. It skips real-time waits and never opens Kafka, PostgreSQL, HTTP, file, or cloud-storage connections. Use run --stdout when you also want real pacing and delivery behavior without contacting the destination. A connected run opens every declared connection and sends events.

sample prints a short preview to your terminal. It does not wait for real-time pacing, and it never opens Kafka, PostgreSQL, or other destinations.

synthtraffic sample kafka-orders.yaml --events 2 --seed 42

docker run --rm \
  --env-file ./license.env \
  --volume "${PWD}/kafka-orders.yaml:/work/kafka-orders.yaml:ro" \
  synthtraffic/synthtraffic:latest \
  sample /work/kafka-orders.yaml --events 2 --seed 42

run --stdout prints the destination-shaped envelope locally without opening the configured connection. Remove --stdout when you are ready to send to that destination.

synthtraffic run kafka-orders.yaml --stdout --events 2 --seed 42

docker run --rm \
  --env-file ./license.env \
  --volume "${PWD}/kafka-orders.yaml:/work/kafka-orders.yaml:ro" \
  synthtraffic/synthtraffic:latest \
  run /work/kafka-orders.yaml --stdout --events 2 --seed 42

Full flag lists: sample and run. Install and license: Install.

After the envelope looks right:

1. Start Kafka and create shop.orders.
2. Replace the example broker address if Kafka is elsewhere.
3. Run the shown run --stdout command once more to verify the full paced output.
4. Remove --stdout to publish through the configured connection.

Choose a destination

Each guide starts with a runnable scenario, then lists every supported setting and the setup required for real delivery:

• Kafka — topic, optional key and headers, and value
• PostgreSQL — table and row, with an explicit schema policy
• HTTP — method, path, optional request fields, and body
• File — rolling local artifacts
• S3, GCS, and Azure Blob — rolling cloud-storage objects

Next: simulate delayed, missing, and repeated deliveries.