AvroEx is a library for encoding and decoding data with Avro schemas. Supports parsing schemas, encoding data, and decoding data.
For encoding and decoding, the following type chart should be referenced:
| Avro Types | Elixir Types |
|---|---|
| boolean | boolean |
| integer | integer |
| long | integer |
| float | decimal |
| double | decimal |
| bytes | binary |
| string | String.t, atom |
| null | nil |
| Record | map |
| Enum | String.t, atom (corresponding to the enum's symbol list) |
Summary
Functions
Given an encoded message and its accompanying schema, decodes the message.
Same as decode/2, but returns raw decoded value.
Given an Elixir or JSON-encoded schema, parses the schema and returns a AvroEx.Schema.t/0 struct representing the schema.
Same as AvroEx.decode_schema/1, but raises an exception on failure instead of
returning an error tuple.
Checks to see if the given data is encodable using the given schema. Helpful for unit testing.
Given AvroEx.Schema.t/0 and term(), takes the data and encodes it according to the schema.
Same as encode/2, but returns the encoded value directly.
Encodes the given schema to JSON
Types
@type encoded_avro() :: binary()
Functions
@spec decode(AvroEx.Schema.t(), encoded_avro(), keyword()) :: {:ok, term()} | {:error, AvroEx.DecodeError.t()}
Given an encoded message and its accompanying schema, decodes the message.
iex> schema = AvroEx.decode_schema!("boolean")
iex> AvroEx.decode(schema, <<1>>)
{:ok, true}Tagged unions
When decoding one can set the option tagged_unions: true to decode union
values as a tagged tuple of {name, value} instead of just the plain value.
This allows to retain the information about which union schema was used for
encoding when this cannot be infered from the value alone.
Decimals
Specify the option decimals: :exact to use Decimal.new/3 to parse decimals
into a Decimal struct with arbitrary precision.
Otherwise, an approximate number is calculated.
@spec decode!(AvroEx.Schema.t(), encoded_avro(), keyword()) :: term()
Same as decode/2, but returns raw decoded value.
Raises AvroEx.DecodeError.t/0 on error.
For documentation of opts see decode/3.
Examples
iex> schema = AvroEx.decode_schema!("string")
iex> encoded = AvroEx.encode!(schema, "hello")
iex> AvroEx.decode!(schema, encoded)
"hello"
@spec decode_schema(term(), Keyword.t()) :: {:ok, AvroEx.Schema.t()} | {:error, AvroEx.Schema.DecodeError.t()}
Given an Elixir or JSON-encoded schema, parses the schema and returns a AvroEx.Schema.t/0 struct representing the schema.
Errors for invalid JSON, invalid schemas, and bad name references.
Options
:strict- whether to strictly validate the schema, defaults tofalse. Recommended to turn this on for locally owned schemas, but not for interop with external schemas.
Examples
iex> AvroEx.decode_schema("string")
{:ok, %AvroEx.Schema{schema: %AvroEx.Schema.Primitive{type: :string}}}
iex> json= ~S({"fields":[{"name":"a","type":"string"}],"name":"my_type","type":"record"})
iex> {:ok, %Schema{schema: record}} = AvroEx.decode_schema(json)
iex> match?(%Record{}, record)
true
@spec decode_schema!(term(), Keyword.t()) :: AvroEx.Schema.t()
Same as AvroEx.decode_schema/1, but raises an exception on failure instead of
returning an error tuple.
Examples
iex> AvroEx.decode_schema!("int")
%AvroEx.Schema{schema: %AvroEx.Schema.Primitive{type: :int}}
@spec encodable?(AvroEx.Schema.t(), any()) :: boolean()
Checks to see if the given data is encodable using the given schema. Helpful for unit testing.
iex> AvroEx.encodable?(%Schema{schema: %Primitive{type: :string}}, "wut")
true
iex> AvroEx.encodable?(%Schema{schema: %Primitive{type: :string}}, 12345)
false
@spec encode(AvroEx.Schema.t(), term(), keyword()) :: {:ok, encoded_avro()} | {:error, AvroEx.EncodeError.t() | Exception.t()}
Given AvroEx.Schema.t/0 and term(), takes the data and encodes it according to the schema.
Examples
iex> schema = AvroEx.decode_schema!("int")
iex> AvroEx.encode(schema, 1234)
{:ok, <<164, 19>>}Tagged unions
When supplying a union value one can optionally supply a tagged tuple of {name, value}
instead of just the plain value to force encoding the value as the named type
found using name instead of matching by the shape of value. This can improve
performance and allows forcing a selected named type even if the shape of the
data is the same. See also the "Tagged unions" section on decode/3.
Array encoding
Array encoding may add an additional long encoded integer to put the byte size
of blocks with their counts. This allows consumers of the encoded data to skip
over those blocks in an efficient manner. Using the option include_block_byte_size: true
enables adding those additional values.
@spec encode!(AvroEx.Schema.t(), term(), keyword()) :: encoded_avro()
Same as encode/2, but returns the encoded value directly.
Raises AvroEx.EncodeError.t/0 on error.
For documentation of opts see encode/3.
Examples
iex> schema = AvroEx.decode_schema!("boolean")
iex> AvroEx.encode!(schema, true)
<<1>>
@spec encode_schema(AvroEx.Schema.t(), Keyword.t()) :: String.t()
Encodes the given schema to JSON
Options
canonical- Encodes the schema into its Parsing Canonical Form, defaultfalse
Examples
iex> schema = AvroEx.decode_schema!(%{"type" => "int", "logicalType" => "date"})
iex> AvroEx.encode_schema(schema)
~S({"type":"int","logicalType":"date"})
iex> schema = AvroEx.decode_schema!(%{"type" => "int", "logicalType" => "date"})
iex> AvroEx.encode_schema(schema, canonical: true)
~S("int")
@spec named_type( AvroEx.Schema.full_name(), AvroEx.Schema.t() | AvroEx.Schema.Context.t() ) :: nil | AvroEx.Schema.schema_types()
@spec parse_schema(AvroEx.Schema.json_schema()) :: {:ok, AvroEx.Schema.t()} | {:error, AvroEx.Schema.DecodeError.t()}
@spec parse_schema!(AvroEx.Schema.json_schema()) :: AvroEx.Schema.t() | no_return()