AvroJson

dev.constructive.eo.avro.circe.AvroJson
object AvroJson

Structural Avro ↔ circe bridge: move between an Avro generic runtime value and a circe io.circe.Json document '''without''' a typed case class in the middle. The Avro mirror of dev.constructive.eo.circe. Lives inside cats-eo-avro with circe as an Optional dependency — the API surface names io.circe.Json, so any caller already depends on circe directly; add circe-core to use this package.

The bidirectional entry points are record, a lawful Prism[Json, IndexedRecord] per schema — getOption is the strict schema-guided parse (Json → record, misses on any shape the schema does not pin), reverseGet the total structural walk avroToJson (record → Json) — and its polymorphic diagonal family rooted at valuePrismpPrism, bytesPrism, recordPrism and pRecord pre-compose its inputs via tearFrom / mendFrom, tearing generic values / payload bytes / records into a typed A and mending back out as Json.

==Rendering conventions (record → Json)==

avroToJson is a purely structural walk of the Avro value model:

  • IndexedRecord → object (field name from getSchema.getFields, recurse on value, schema declaration order);
  • java.util.Map → object (stringify keys, recurse on values) — matches Encoder[Map[String, ?]];
  • java.util.List → array (recurse);
  • CharSequence (incl. org.apache.avro.util.Utf8) → Json.fromString;
  • Integer / LongJson.fromLong (Json.fromInt(i) == Json.fromLong(i.toLong));
  • DoubleJson.fromDoubleOrNull (matches circe's Encoder[Double]);
  • FloatJson.fromFloatOrNull (matches circe's Encoder[Float]). NB widening float→double first would change the value (0.1f0.10000000149…) and break JsonNumber equality, so the float branch must not;
  • BooleanJson.fromBoolean; a resolved null union branch → Json.Null;
  • GenericEnumSymbolJson.fromString;
  • ByteBuffer / GenericFixed → array of signed byte ints (circe's Encoder[Array[Byte]] convention);
  • any other runtime type → Json.fromString(value.toString) — a lenient last resort (the walk is total), not a convention to rely on.

Unions are resolved at the value level (the runtime value IS the branch), so dispatch on the runtime type needs no union special-casing.

==Parsing conventions (Json → record, the prism's getOption)==

The parse inverts the walk under the schema, and is '''strict''' so the prism stays lawful (getOption(j).map(reverseGet) must reproduce j, so nothing may be guessed, defaulted, or dropped):

  • a record object's key set must equal the schema's field names exactly — an extra key would be silently dropped on re-render, a missing one has NO schema default applied; both miss;
  • int / long parse via JsonNumber.toInt / toLong — non-integral or out-of-range numbers miss; float / double accept any JSON number (toFloat / toDouble);
  • enum requires a string among the schema's symbols; fixed requires the exact byte length; bytes / fixed parse the signed-byte-int array rendering;
  • a union is matched '''first branch that parses wins''' (Json.Null only ever matches a null branch). For the ubiquitous ["null", X] this is exact; a union whose branches overlap structurally (e.g. ["int", "long"]) resolves to the first — and the prism laws only hold up to that choice.

==Non-goals (deliberate)==

The bridge sees only the '''runtime''' Avro value, never the logical type or a case-class field type — so a source whose circe encoder does a '''non-structural''' transform is not reproducible structurally and the bridge is not a drop-in there. Two representative cases:

  • '''logical types''': an Instant stored as timestamp-millis is a long at runtime; the bridge renders Json.fromLong, not the ISO-8601 string a Encoder[Instant] would emit;
  • '''stringified numerics''': an encoder that renders a long as a decimal '''string''' has no structural counterpart — the bridge renders Json.fromLong.

These are the caller's concern (post-process the Json, or decode the typed value): the walk is defined by the wire shape, not the intended semantic type.

==Drilled cursors (.json and .avro faces)==

The full AvroPrism cursor sugar — .field(_.x) / .fields(...) / .at(i) / .union[B] / .each / Dynamic selection — reaches this bridge through the json extensions on AvroPrism / AvroTraversal (drill first, flip last, like .record): reads yield the typed focus, writes render the whole modified document as Json. render is the focus-as-standalone-Json terminal for the read side. The reverse cursor is the avro extension on dev.constructive.eo.circe.JsonPrism: the drilled focus itself converts — subtree ↔ Avro binary, both directions the structural walks above. Same face family as dev.constructive.eo.avro.jsoniter, landing on the AST instead of bytes; the .avro face is why cats-eo-circe is a second Optional dependency next to circe-core.

Attributes

Source
AvroJson.scala
Graph
Supertypes
class Object
trait Matchable
class Any
Self type
AvroJson.type

Members list

Grouped members

Bidirectional prism (Json ↔ record)

def record(schema: Schema): MendTearPrism[Json, Json, IndexedRecord, IndexedRecord]

The bidirectional bridge: a Prism[Json, IndexedRecord] for schema. Reading (getOption / to) is the strict schema-guided parse — see ''Parsing conventions''; it misses (Left of the untouched Json) on anything the schema does not pin. Writing (reverseGet) is the total structural walk avroToJson — use it directly for the record → Json direction.

The bidirectional bridge: a Prism[Json, IndexedRecord] for schema. Reading (getOption / to) is the strict schema-guided parse — see ''Parsing conventions''; it misses (Left of the untouched Json) on anything the schema does not pin. Writing (reverseGet) is the total structural walk avroToJson — use it directly for the record → Json direction.

Attributes

Source
AvroJson.scala

Codec diagonals (tearFrom / mendFrom of valuePrism)

def bytesPrism[A](using codec: AvroCodec[A]): MendTearPrism[AvroBytes, Json, A, A]

Typed-both-ways byte diagonal — pPrism with the mend routed through the codec's encode, so modify(f: A => A): AvroBytes => Json works in one hop with no generic record at the call site.

Typed-both-ways byte diagonal — pPrism with the mend routed through the codec's encode, so modify(f: A => A): AvroBytes => Json works in one hop with no generic record at the call site.

Attributes

Source
AvroJson.scala
def bytesPrism[A](writer: Schema)(using codec: AvroCodec[A]): MendTearPrism[AvroBytes, Json, A, A]

bytesPrism for a stream written under a '''different''' (but compatible) writer schema: the tear resolves writer → codec.schema (Avro schema resolution — field reordering, defaults, promotions) before decoding.

bytesPrism for a stream written under a '''different''' (but compatible) writer schema: the tear resolves writer → codec.schema (Avro schema resolution — field reordering, defaults, promotions) before decoding.

Attributes

Source
AvroJson.scala
def pPrism[A](using codec: AvroCodec[A]): MendTearPrism[AvroBytes, Json, A, IndexedRecord]

valuePrism torn from a binary parse — tear Avro '''payload bytes''' into a typed A, mend a generic record out as Json.

valuePrism torn from a binary parse — tear Avro '''payload bytes''' into a typed A, mend a generic record out as Json.

Same parse contract as bytesToJson: position-based under codec.schema, no writer/reader resolution — malformed or schema-mismatched bytes throw rather than miss. For a mixed-schema stream use the writer-schema overload of bytesPrism (or ConfluentWire.resolvingBytes).

Attributes

Source
AvroJson.scala
def pRecord(schema: Schema): MendTearPrism[AvroBytes, Json, IndexedRecord, IndexedRecord]

Untyped byte diagonal — no codec, no case class: bytesPrism instantiated at IndexedRecord via a trivial per-schema codec (encode = identity, decode = runtime-type check). For registry / dynamic-schema consumers; effectively bytesToJson upgraded to a writable prism.

Untyped byte diagonal — no codec, no case class: bytesPrism instantiated at IndexedRecord via a trivial per-schema codec (encode = identity, decode = runtime-type check). For registry / dynamic-schema consumers; effectively bytesToJson upgraded to a writable prism.

Attributes

Source
AvroJson.scala
def recordPrism[A](using codec: AvroCodec[A]): MendTearPrism[IndexedRecord, Json, A, A]

Record-sourced diagonal — for streams already resolved to generic records (e.g. the output of ConfluentWire.recordReader): tear an IndexedRecord into a typed A, mend A out as Json.

Record-sourced diagonal — for streams already resolved to generic records (e.g. the output of ConfluentWire.recordReader): tear an IndexedRecord into a typed A, mend A out as Json.

Attributes

Source
AvroJson.scala
def valuePrism[A](using codec: AvroCodec[A]): MendTearPrism[Any, Json, A, Any]

The fundamental codec diagonal — every prism below is this one with its inputs pre-composed via MendTearPrism.tearFrom / mendFrom. Tears an Avro '''generic runtime value''' into a typed A via the codec's decode; a miss surrenders the '''structural Json view''' of the value (the avroToJson walk generalised to any value) instead of the raw input, so a payload that is valid Avro but not a valid A still lands somewhere inspectable. The mend renders any generic value back as Json — the same structural walk.

The fundamental codec diagonal — every prism below is this one with its inputs pre-composed via MendTearPrism.tearFrom / mendFrom. Tears an Avro '''generic runtime value''' into a typed A via the codec's decode; a miss surrenders the '''structural Json view''' of the value (the avroToJson walk generalised to any value) instead of the raw input, so a payload that is valid Avro but not a valid A still lands somewhere inspectable. The mend renders any generic value back as Json — the same structural walk.

The family below varies the two '''input''' slots only: what the tear consumes (generic value / payload bytes / record) and what the mend accepts (generic value / A / IndexedRecord). The outputs — typed focus A, Json fallback — are fixed here.

Attributes

Source
AvroJson.scala

Structural walk

def avroToJson(record: IndexedRecord): Json

The whole substance of the write side: a recursive structural walk of an Avro generic record into a circe io.circe.Json object. Allocates no typed case class; field order is the schema's field declaration order. Also record's reverseGet.

The whole substance of the write side: a recursive structural walk of an Avro generic record into a circe io.circe.Json object. Allocates no typed case class; field order is the schema's field declaration order. Also record's reverseGet.

Attributes

Source
AvroJson.scala

Read optic (bytes → Json)

def bytesToJson(schema: Schema): Getter[AvroBytes, Json]

The read optic: the base avroToJson composed onto a bytes → record read dev.constructive.eo.optics.Getter. The parse step reads payload bytes to a generic IndexedRecord (no typed decode), and eo's fused Getter.andThen(Getter) maps the structural walk over it, yielding a total Getter[AvroBytes, Json].

The read optic: the base avroToJson composed onto a bytes → record read dev.constructive.eo.optics.Getter. The parse step reads payload bytes to a generic IndexedRecord (no typed decode), and eo's fused Getter.andThen(Getter) maps the structural walk over it, yielding a total Getter[AvroBytes, Json].

The schema must be the exact writer schema the bytes were encoded under: the parse is position-based and does no writer/reader resolution, so a mismatched schema silently misreads. For a mixed-schema stream, resolve writer→reader first (e.g. ConfluentWire.recordReader / ConfluentWire.resolvingBytes) and walk the resolved record / bytes.

Attributes

Source
AvroJson.scala
def render[A](using codec: AvroCodec[A]): Getter[A, Json]

Focus-as-JSON terminal: render a typed focus as a standalone io.circe.Json through the codec's encode + the structural walk. Compose it after any drilled optic to read just the focused field as Json:

Focus-as-JSON terminal: render a typed focus as a standalone io.circe.Json through the codec's encode + the structural walk. Compose it after any drilled optic to read just the focused field as Json:

 codecPrism[Person].field(_.address).andThen(AvroJson.render[Address])
   .getOption(avroBytes)                       // Option[Json] of the address alone

For the whole-document face (drilled writes included) use the json extension instead.

Attributes

Source
AvroJson.scala