AvroEx (AvroEx v2.2.1)

Copy Markdown View Source

AvroEx is a library for encoding and decoding data with Avro schemas. Supports parsing schemas, encoding data, and decoding data.

For encoding and decoding, the following type chart should be referenced:

Avro TypesElixir Types
booleanboolean
integerinteger
longinteger
floatdecimal
doubledecimal
bytesbinary
stringString.t, atom
nullnil
Recordmap
EnumString.t, atom (corresponding to the enum's symbol list)

Summary

Functions

Given an encoded message and its accompanying schema, decodes the message.

Same as decode/2, but returns raw decoded value.

Given an Elixir or JSON-encoded schema, parses the schema and returns a AvroEx.Schema.t/0 struct representing the schema.

Same as AvroEx.decode_schema/1, but raises an exception on failure instead of returning an error tuple.

Checks to see if the given data is encodable using the given schema. Helpful for unit testing.

Given AvroEx.Schema.t/0 and term(), takes the data and encodes it according to the schema.

Same as encode/2, but returns the encoded value directly.

Encodes the given schema to JSON

parse_schema(json) deprecated
parse_schema!(json) deprecated

Types

encoded_avro()

@type encoded_avro() :: binary()

Functions

decode(schema, message, opts \\ [])

@spec decode(AvroEx.Schema.t(), encoded_avro(), keyword()) ::
  {:ok, term()} | {:error, AvroEx.DecodeError.t()}

Given an encoded message and its accompanying schema, decodes the message.

iex> schema = AvroEx.decode_schema!("boolean")
iex> AvroEx.decode(schema, <<1>>)
{:ok, true}

Tagged unions

When decoding one can set the option tagged_unions: true to decode union values as a tagged tuple of {name, value} instead of just the plain value. This allows to retain the information about which union schema was used for encoding when this cannot be infered from the value alone.

Decimals

Specify the option decimals: :exact to use Decimal.new/3 to parse decimals into a Decimal struct with arbitrary precision.

Otherwise, an approximate number is calculated.

decode!(schema, message, opts \\ [])

@spec decode!(AvroEx.Schema.t(), encoded_avro(), keyword()) :: term()

Same as decode/2, but returns raw decoded value.

Raises AvroEx.DecodeError.t/0 on error.

For documentation of opts see decode/3.

Examples

iex> schema = AvroEx.decode_schema!("string")
iex> encoded = AvroEx.encode!(schema, "hello")
iex> AvroEx.decode!(schema, encoded)
"hello"

decode_schema(schema, opts \\ [])

@spec decode_schema(term(), Keyword.t()) ::
  {:ok, AvroEx.Schema.t()} | {:error, AvroEx.Schema.DecodeError.t()}

Given an Elixir or JSON-encoded schema, parses the schema and returns a AvroEx.Schema.t/0 struct representing the schema.

Errors for invalid JSON, invalid schemas, and bad name references.

Options

  • :strict - whether to strictly validate the schema, defaults to false. Recommended to turn this on for locally owned schemas, but not for interop with external schemas.

Examples

iex> AvroEx.decode_schema("string")
{:ok, %AvroEx.Schema{schema: %AvroEx.Schema.Primitive{type: :string}}}

iex> json= ~S({"fields":[{"name":"a","type":"string"}],"name":"my_type","type":"record"})
iex> {:ok, %Schema{schema: record}} = AvroEx.decode_schema(json)
iex> match?(%Record{}, record)
true

decode_schema!(schema, opts \\ [])

@spec decode_schema!(term(), Keyword.t()) :: AvroEx.Schema.t()

Same as AvroEx.decode_schema/1, but raises an exception on failure instead of returning an error tuple.

Examples

iex> AvroEx.decode_schema!("int")
%AvroEx.Schema{schema: %AvroEx.Schema.Primitive{type: :int}}

encodable?(schema, data)

@spec encodable?(AvroEx.Schema.t(), any()) :: boolean()

Checks to see if the given data is encodable using the given schema. Helpful for unit testing.

iex> AvroEx.encodable?(%Schema{schema: %Primitive{type: :string}}, "wut")
true

iex> AvroEx.encodable?(%Schema{schema: %Primitive{type: :string}}, 12345)
false

encode(schema, data, opts \\ [])

@spec encode(AvroEx.Schema.t(), term(), keyword()) ::
  {:ok, encoded_avro()} | {:error, AvroEx.EncodeError.t() | Exception.t()}

Given AvroEx.Schema.t/0 and term(), takes the data and encodes it according to the schema.

Examples

iex> schema = AvroEx.decode_schema!("int")
iex> AvroEx.encode(schema, 1234)
{:ok, <<164, 19>>}

Tagged unions

When supplying a union value one can optionally supply a tagged tuple of {name, value} instead of just the plain value to force encoding the value as the named type found using name instead of matching by the shape of value. This can improve performance and allows forcing a selected named type even if the shape of the data is the same. See also the "Tagged unions" section on decode/3.

Array encoding

Array encoding may add an additional long encoded integer to put the byte size of blocks with their counts. This allows consumers of the encoded data to skip over those blocks in an efficient manner. Using the option include_block_byte_size: true enables adding those additional values.

encode!(schema, data, opts \\ [])

@spec encode!(AvroEx.Schema.t(), term(), keyword()) :: encoded_avro()

Same as encode/2, but returns the encoded value directly.

Raises AvroEx.EncodeError.t/0 on error.

For documentation of opts see encode/3.

Examples

iex> schema = AvroEx.decode_schema!("boolean")
iex> AvroEx.encode!(schema, true)
<<1>>

encode_schema(schema, opts \\ [])

@spec encode_schema(AvroEx.Schema.t(), Keyword.t()) :: String.t()

Encodes the given schema to JSON

Options

Examples

iex> schema = AvroEx.decode_schema!(%{"type" => "int", "logicalType" => "date"})
iex> AvroEx.encode_schema(schema)
~S({"type":"int","logicalType":"date"})

iex> schema = AvroEx.decode_schema!(%{"type" => "int", "logicalType" => "date"})
iex> AvroEx.encode_schema(schema, canonical: true)
~S("int")

named_type(name, context)

This function is deprecated. Use AvroEx.Schema.Context.lookup/2.

parse_schema(json)

This function is deprecated. Use AvroEx.decode_schema/1 instead.
@spec parse_schema(AvroEx.Schema.json_schema()) ::
  {:ok, AvroEx.Schema.t()} | {:error, AvroEx.Schema.DecodeError.t()}

parse_schema!(json)

This function is deprecated. Use AvroEx.decode_schema!/1 instead.
@spec parse_schema!(AvroEx.Schema.json_schema()) :: AvroEx.Schema.t() | no_return()