AvroJson

dev.constructive.eo.avro.circe.AvroJson
object AvroJson

Structural Avro ↔ circe bridge: move between an Avro generic runtime value and a circe io.circe.Json document '''without''' a typed case class in the middle. The Avro mirror of dev.constructive.eo.circe. Lives inside cats-eo-avro with circe as an Optional dependency — the API surface names io.circe.Json, so any caller already depends on circe directly; add circe-core to use this package.

The bidirectional entry points are record, a lawful Prism[Json, IndexedRecord] per schema — getOption is the strict schema-guided parse (Json → record, misses on any shape the schema does not pin), reverseGet the total structural walk avroToJson (record → Json) — and its polymorphic diagonal family rooted at valuePrismpPrism, bytesPrism, recordPrism and pRecord pre-compose its inputs via tearFrom / mendFrom, tearing generic values / payload bytes / records into a typed A and mending back out as Json.

==Rendering conventions (record → Json)==

avroToJson is a purely structural walk of the Avro value model:

  • IndexedRecord → object (field name from getSchema.getFields, recurse on value, schema declaration order);
  • java.util.Map → object (stringify keys, recurse on values) — matches Encoder[Map[String, ?]];
  • java.util.List → array (recurse);
  • CharSequence (incl. org.apache.avro.util.Utf8) → Json.fromString;
  • Integer / LongJson.fromLong (Json.fromInt(i) == Json.fromLong(i.toLong));
  • DoubleJson.fromDoubleOrNull (matches circe's Encoder[Double]);
  • FloatJson.fromFloatOrNull (matches circe's Encoder[Float]). NB widening float→double first would change the value (0.1f0.10000000149…) and break JsonNumber equality, so the float branch must not;
  • BooleanJson.fromBoolean; a resolved null union branch → Json.Null;
  • GenericEnumSymbolJson.fromString;
  • ByteBuffer / GenericFixed → array of signed byte ints (circe's Encoder[Array[Byte]] convention);
  • any other runtime type → Json.fromString(value.toString) — a lenient last resort (the walk is total), not a convention to rely on.

Unions are resolved at the value level (the runtime value IS the branch), so dispatch on the runtime type needs no union special-casing.

==Parsing conventions (Json → record, the prism's getOption)==

The parse inverts the walk under the schema, and is '''strict''' so the prism stays lawful (getOption(j).map(reverseGet) must reproduce j, so nothing may be guessed, defaulted, or dropped):

  • a record object's key set must equal the schema's field names exactly — an extra key would be silently dropped on re-render, a missing one has NO schema default applied; both miss;
  • int / long parse via JsonNumber.toInt / toLong — non-integral or out-of-range numbers miss; float / double accept any JSON number (toFloat / toDouble);
  • enum requires a string among the schema's symbols; fixed requires the exact byte length; bytes / fixed parse the signed-byte-int array rendering;
  • a union is matched '''first branch that parses wins''' (Json.Null only ever matches a null branch). For the ubiquitous ["null", X] this is exact; a union whose branches overlap structurally (e.g. ["int", "long"]) resolves to the first — and the prism laws only hold up to that choice.

==Non-goals (deliberate)==

The bridge sees only the '''runtime''' Avro value, never the logical type or a case-class field type — so a source whose circe encoder does a '''non-structural''' transform is not reproducible structurally and the bridge is not a drop-in there. Two representative cases:

  • '''logical types''': an Instant stored as timestamp-millis is a long at runtime; the bridge renders Json.fromLong, not the ISO-8601 string a Encoder[Instant] would emit;
  • '''stringified numerics''': an encoder that renders a long as a decimal '''string''' has no structural counterpart — the bridge renders Json.fromLong.

These are the caller's concern (post-process the Json, or decode the typed value): the walk is defined by the wire shape, not the intended semantic type.

Attributes

Source
AvroJson.scala
Graph
Supertypes
class Object
trait Matchable
class Any
Self type
AvroJson.type

Members list

Grouped members

Bidirectional prism (Json ↔ record)

def record(schema: Schema): MendTearPrism[Json, Json, IndexedRecord, IndexedRecord]

The bidirectional bridge: a Prism[Json, IndexedRecord] for schema. Reading (getOption / to) is the strict schema-guided parse — see ''Parsing conventions''; it misses (Left of the untouched Json) on anything the schema does not pin. Writing (reverseGet) is the total structural walk avroToJson — use it directly for the record → Json direction.

The bidirectional bridge: a Prism[Json, IndexedRecord] for schema. Reading (getOption / to) is the strict schema-guided parse — see ''Parsing conventions''; it misses (Left of the untouched Json) on anything the schema does not pin. Writing (reverseGet) is the total structural walk avroToJson — use it directly for the record → Json direction.

Attributes

Source
AvroJson.scala

Codec diagonals (tearFrom / mendFrom of valuePrism)

def bytesPrism[A](using codec: AvroCodec[A]): MendTearPrism[Array[Byte], Json, A, A]

Typed-both-ways byte diagonal — pPrism with the mend routed through the codec's encode, so modify(f: A => A): Array[Byte] => Json works in one hop with no generic record at the call site.

Typed-both-ways byte diagonal — pPrism with the mend routed through the codec's encode, so modify(f: A => A): Array[Byte] => Json works in one hop with no generic record at the call site.

Attributes

Source
AvroJson.scala
def bytesPrism[A](writer: Schema)(using codec: AvroCodec[A]): MendTearPrism[Array[Byte], Json, A, A]

bytesPrism for a stream written under a '''different''' (but compatible) writer schema: the tear resolves writer → codec.schema (Avro schema resolution — field reordering, defaults, promotions) before decoding.

bytesPrism for a stream written under a '''different''' (but compatible) writer schema: the tear resolves writer → codec.schema (Avro schema resolution — field reordering, defaults, promotions) before decoding.

Attributes

Source
AvroJson.scala
def pPrism[A](using codec: AvroCodec[A]): MendTearPrism[Array[Byte], Json, A, IndexedRecord]

valuePrism torn from a binary parse — tear Avro '''payload bytes''' into a typed A, mend a generic record out as Json.

valuePrism torn from a binary parse — tear Avro '''payload bytes''' into a typed A, mend a generic record out as Json.

Same parse contract as bytesToJson: position-based under codec.schema, no writer/reader resolution — malformed or schema-mismatched bytes throw rather than miss. For a mixed-schema stream use the writer-schema overload of bytesPrism (or ConfluentWire.resolvingBytes).

Attributes

Source
AvroJson.scala
def pRecord(schema: Schema): MendTearPrism[Array[Byte], Json, IndexedRecord, IndexedRecord]

Untyped byte diagonal — no codec, no case class: bytesPrism instantiated at IndexedRecord via a trivial per-schema codec (encode = identity, decode = runtime-type check). For registry / dynamic-schema consumers; effectively bytesToJson upgraded to a writable prism.

Untyped byte diagonal — no codec, no case class: bytesPrism instantiated at IndexedRecord via a trivial per-schema codec (encode = identity, decode = runtime-type check). For registry / dynamic-schema consumers; effectively bytesToJson upgraded to a writable prism.

Attributes

Source
AvroJson.scala
def recordPrism[A](using codec: AvroCodec[A]): MendTearPrism[IndexedRecord, Json, A, A]

Record-sourced diagonal — for streams already resolved to generic records (e.g. the output of ConfluentWire.recordReader): tear an IndexedRecord into a typed A, mend A out as Json.

Record-sourced diagonal — for streams already resolved to generic records (e.g. the output of ConfluentWire.recordReader): tear an IndexedRecord into a typed A, mend A out as Json.

Attributes

Source
AvroJson.scala
def valuePrism[A](using codec: AvroCodec[A]): MendTearPrism[Any, Json, A, Any]

The fundamental codec diagonal — every prism below is this one with its inputs pre-composed via MendTearPrism.tearFrom / mendFrom. Tears an Avro '''generic runtime value''' into a typed A via the codec's decode; a miss surrenders the '''structural Json view''' of the value (the avroToJson walk generalised to any value) instead of the raw input, so a payload that is valid Avro but not a valid A still lands somewhere inspectable. The mend renders any generic value back as Json — the same structural walk.

The fundamental codec diagonal — every prism below is this one with its inputs pre-composed via MendTearPrism.tearFrom / mendFrom. Tears an Avro '''generic runtime value''' into a typed A via the codec's decode; a miss surrenders the '''structural Json view''' of the value (the avroToJson walk generalised to any value) instead of the raw input, so a payload that is valid Avro but not a valid A still lands somewhere inspectable. The mend renders any generic value back as Json — the same structural walk.

The family below varies the two '''input''' slots only: what the tear consumes (generic value / payload bytes / record) and what the mend accepts (generic value / A / IndexedRecord). The outputs — typed focus A, Json fallback — are fixed here.

Attributes

Source
AvroJson.scala

Structural walk

def avroToJson(record: IndexedRecord): Json

The whole substance of the write side: a recursive structural walk of an Avro generic record into a circe io.circe.Json object. Allocates no typed case class; field order is the schema's field declaration order. Also record's reverseGet.

The whole substance of the write side: a recursive structural walk of an Avro generic record into a circe io.circe.Json object. Allocates no typed case class; field order is the schema's field declaration order. Also record's reverseGet.

Attributes

Source
AvroJson.scala

Read optic (bytes → Json)

def bytesToJson(schema: Schema): Getter[Array[Byte], Json]

The read optic: the base avroToJson composed onto a bytes → record read dev.constructive.eo.optics.Getter. The parse step reads payload bytes to a generic IndexedRecord (no typed decode), and eo's fused Getter.andThen(Getter) maps the structural walk over it, yielding a total Getter[Array[Byte], Json].

The read optic: the base avroToJson composed onto a bytes → record read dev.constructive.eo.optics.Getter. The parse step reads payload bytes to a generic IndexedRecord (no typed decode), and eo's fused Getter.andThen(Getter) maps the structural walk over it, yielding a total Getter[Array[Byte], Json].

The schema must be the exact writer schema the bytes were encoded under: the parse is position-based and does no writer/reader resolution, so a mismatched schema silently misreads. For a mixed-schema stream, resolve writer→reader first (e.g. ConfluentWire.recordReader / ConfluentWire.resolvingBytes) and walk the resolved record / bytes.

Attributes

Source
AvroJson.scala