Pydantic serialisation

Tortoise ORM has a Pydantic plugin that will generate Pydantic Models from Tortoise Models, and then provides helper functions to serialise that model and its related objects.

We currently only support generating Pydantic objects for serialisation, and no deserialisation at this stage.

See the Pydantic Examples

Tutorial

1: Basic usage

Here we introduce:

  • Creating a Pydantic model from a Tortoise model

  • Docstrings & doc-comments are used

  • Evaluating the generated schema

  • Simple serialisation with both .model_dump() and .model_dump_json()

Source to example: 1: Basic usage

Lets start with a basic Tortoise Model:

from tortoise import fields
from tortoise.models import Model

class Tournament(Model):
    """
    This references a Tournament
    """
    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=100)
    #: The date-time the Tournament record was created at
    created_at = fields.DatetimeField(auto_now_add=True)
To create a Pydantic model from that one would call:
tortoise.contrib.pydantic.creator.pydantic_model_creator()
from tortoise.contrib.pydantic import pydantic_model_creator

Tournament_Pydantic = pydantic_model_creator(Tournament)

And now have a Pydantic Model that can be used for representing schema and serialisation.

The JSON-Schema of Tournament_Pydantic is now:

>>> print(Tournament_Pydantic.schema())
{
    'title': 'Tournament',
    'description': 'This references a Tournament',
    'type': 'object',
    'properties': {
        'id': {
            'title': 'Id',
            'type': 'integer'
        },
        'name': {
            'title': 'Name',
            'type': 'string'
        },
        'created_at': {
            'title': 'Created At',
            'description': 'The date-time the Tournament record was created at',
            'type': 'string',
            'format': 'date-time'
        }
    }
}

Note how the class docstring and doc-comment #: is included as descriptions in the Schema.

To serialise an object it is simply (in an async context):

tournament = await Tournament.create(name="New Tournament")
tourpy = await Tournament_Pydantic.from_tortoise_orm(tournament)

And one could get the contents by using regular Pydantic-object methods, such as .model_dump() or .model_dump_json()

>>> print(tourpy.model_dump())
{
    'id': 1,
    'name': 'New Tournament',
    'created_at': datetime.datetime(2020, 3, 1, 20, 28, 9, 346808)
}
>>> print(tourpy.model_dump_json())
{
    "id": 1,
    "name": "New Tournament",
    "created_at": "2020-03-01T20:28:09.346808"
}

2: Querysets & Lists

Here we introduce:

  • Creating a list-model to serialise a queryset

  • Default sorting is honoured

Source to example: 2: Querysets & Lists

from tortoise import fields
from tortoise.models import Model

class Tournament(Model):
    """
    This references a Tournament
    """
    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=100)
    #: The date-time the Tournament record was created at
    created_at = fields.DatetimeField(auto_now_add=True)

    class Meta:
        # Define the default ordering
        #  the pydantic serialiser will use this to order the results
        ordering = ["name"]
To create a Pydantic list-model from that one would call:
tortoise.contrib.pydantic.creator.pydantic_queryset_creator()
from tortoise.contrib.pydantic import pydantic_queryset_creator

Tournament_Pydantic_List = pydantic_queryset_creator(Tournament)

And now have a Pydantic Model that can be used for representing schema and serialisation.

The JSON-Schema of Tournament_Pydantic_List is now:

>>> print(Tournament_Pydantic_List.schema())
{
    'title': 'Tournaments',
    'description': 'This references a Tournament',
    'type': 'array',
    'items': {
        '$ref': '#/definitions/Tournament'
    },
    'definitions': {
        'Tournament': {
            'title': 'Tournament',
            'description': 'This references a Tournament',
            'type': 'object',
            'properties': {
                'id': {
                    'title': 'Id',
                    'type': 'integer'
                },
                'name': {
                    'title': 'Name',
                    'type': 'string'
                },
                'created_at': {
                    'title': 'Created At',
                    'description': 'The date-time the Tournament record was created at',
                    'type': 'string',
                    'format': 'date-time'
                }
            }
        }
    }
}

Note that the Tournament is now not the root. A simple list is.

To serialise an object it is simply (in an async context):

# Create objects
await Tournament.create(name="New Tournament")
await Tournament.create(name="Another")
await Tournament.create(name="Last Tournament")

tourpy = await Tournament_Pydantic_List.from_queryset(Tournament.all())

And one could get the contents by using regular Pydantic-object methods, such as .model_dump() or .model_dump_json()

>>> print(tourpy.model_dump())
{
    'root': [
        {
            'id': 2,
            'name': 'Another',
            'created_at': datetime.datetime(2020, 3, 2, 6, 53, 39, 776504)
        },
        {
            'id': 3,
            'name': 'Last Tournament',
            'created_at': datetime.datetime(2020, 3, 2, 6, 53, 39, 776848)
        },
        {
            'id': 1,
            'name': 'New Tournament',
            'created_at': datetime.datetime(2020, 3, 2, 6, 53, 39, 776211)
        }
    ]
}
>>> print(tourpy.model_dump_json())
[
    {
        "id": 2,
        "name": "Another",
        "created_at": "2020-03-02T06:53:39.776504"
    },
    {
        "id": 3,
        "name": "Last Tournament",
        "created_at": "2020-03-02T06:53:39.776848"
    },
    {
        "id": 1,
        "name": "New Tournament",
        "created_at": "2020-03-02T06:53:39.776211"
    }
]

Note how .model_dump() has a root element with the list, but the .model_dump_json() has the list as root. Also note how the results are sorted alphabetically by name.

3: Relations & Early-init

Here we introduce:

  • Relationships

  • Early model init

Note

The part of this tutorial about early-init is only required if you need to generate the pydantic models before you have initialised Tortoise ORM.

Look at Basic Pydantic (in function run) to see where the *_creator is only called after we initialised Tortoise ORM properly, in that case an early init is not needed.

Source to example: 3: Relations & Early-init

We define our models with a relationship:

from tortoise import fields
from tortoise.models import Model

class Tournament(Model):
    """
    This references a Tournament
    """

    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=100)
    #: The date-time the Tournament record was created at
    created_at = fields.DatetimeField(auto_now_add=True)

class Event(Model):
    """
    This references an Event in a Tournament
    """

    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=100)
    created_at = fields.DatetimeField(auto_now_add=True)

    tournament = fields.ForeignKeyField(
        "models.Tournament", related_name="events", description="The Tournament this happens in"
    )

Next we create our Pydantic Model using pydantic_model_creator:

from tortoise.contrib.pydantic import pydantic_model_creator

Tournament_Pydantic = pydantic_model_creator(Tournament)

The JSON-Schema of Tournament_Pydantic is now:

>>> print(Tournament_Pydantic.schema())
{
    'title': 'Tournament',
    'description': 'This references a Tournament',
    'type': 'object',
    'properties': {
        'id': {
            'title': 'Id',
            'type': 'integer'
        },
        'name': {
            'title': 'Name',
            'type': 'string'
        },
        'created_at': {
            'title': 'Created At',
            'description': 'The date-time the Tournament record was created at',
            'type': 'string',
            'format': 'date-time'
        }
    }
}

Oh no! Where is the relation?

Because the models have not fully initialised, it doesn’t know about the relations at this stage.

We need to initialise our model relationships early using tortoise.Tortoise.init_models()

from tortoise import Tortoise

Tortoise.init_models(["__main__"], "models")
# Now lets try again
Tournament_Pydantic = pydantic_model_creator(Tournament)

The JSON-Schema of Tournament_Pydantic is now:

>>> print(Tournament_Pydantic.schema())
{
    'title': 'Tournament',
    'description': 'This references a Tournament',
    'type': 'object',
    'properties': {
        'id': {
            'title': 'Id',
            'type': 'integer'
        },
        'name': {
            'title': 'Name',
            'type': 'string'
        },
        'created_at': {
            'title': 'Created At',
            'description': 'The date-time the Tournament record was created at',
            'type': 'string',
            'format': 'date-time'
        },
        'events': {
            'title': 'Events',
            'description': 'The Tournament this happens in',
            'type': 'array',
            'items': {
                '$ref': '#/definitions/Event'
            }
        }
    },
    'definitions': {
        'Event': {
            'title': 'Event',
            'description': 'This references an Event in a Tournament',
            'type': 'object',
            'properties': {
                'id': {
                    'title': 'Id',
                    'type': 'integer'
                },
                'name': {
                    'title': 'Name',
                    'type': 'string'
                },
                'created_at': {
                    'title': 'Created At',
                    'type': 'string',
                    'format': 'date-time'
                }
            }
        }
    }
}

Aha! that’s much better.

Note we can also create a model for Event the same way, and it should just work:

Event_Pydantic = pydantic_model_creator(Event)

>>> print(Event_Pydantic.schema())
{
    'title': 'Event',
    'description': 'This references an Event in a Tournament',
    'type': 'object',
    'properties': {
        'id': {
            'title': 'Id',
            'type': 'integer'
        },
        'name': {
            'title': 'Name',
            'type': 'string'
        },
        'created_at': {
            'title': 'Created At',
            'type': 'string',
            'format': 'date-time'
        },
        'tournament': {
            'title': 'Tournament',
            'description': 'The Tournament this happens in',
            'allOf': [
                {
                    '$ref': '#/definitions/Tournament'
                }
            ]
        }
    },
    'definitions': {
        'Tournament': {
            'title': 'Tournament',
            'description': 'This references a Tournament',
            'type': 'object',
            'properties': {
                'id': {
                    'title': 'Id',
                    'type': 'integer'
                },
                'name': {
                    'title': 'Name',
                    'type': 'string'
                },
                'created_at': {
                    'title': 'Created At',
                    'description': 'The date-time the Tournament record was created at',
                    'type': 'string',
                    'format': 'date-time'
                }
            }
        }
    }
}

And that also has the relation defined!

Note how both schema’s don’t follow relations back. This is on by default, and in a later tutorial we will show the options.

Lets create and serialise the objects and see what they look like (in an async context):

# Create objects
tournament = await Tournament.create(name="New Tournament")
event = await Event.create(name="The Event", tournament=tournament)

# Serialise Tournament
tourpy = await Tournament_Pydantic.from_tortoise_orm(tournament)

>>> print(tourpy.model_dump_json())
{
    "id": 1,
    "name": "New Tournament",
    "created_at": "2020-03-02T07:23:27.731656",
    "events": [
        {
            "id": 1,
            "name": "The Event",
            "created_at": "2020-03-02T07:23:27.732492"
        }
    ]
}

And serialising the event (in an async context):

eventpy = await Event_Pydantic.from_tortoise_orm(event)

>>> print(eventpy.model_dump_json())
{
    "id": 1,
    "name": "The Event",
    "created_at": "2020-03-02T07:23:27.732492",
    "tournament": {
        "id": 1,
        "name": "New Tournament",
        "created_at": "2020-03-02T07:23:27.731656"
    }
}

4: PydanticMeta & Callables

Here we introduce:

  • Configuring model creator via PydanticMeta class.

  • Using callable functions to annotate extra data.

Source to example: 4: PydanticMeta & Callables

Let’s add some methods that calculate data, and tell the creators to use them:

class Tournament(Model):
    """
    This references a Tournament
    """

    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=100)
    created_at = fields.DatetimeField(auto_now_add=True)

    # It is useful to define the reverse relations manually so that type checking
    #  and auto completion work
    events: fields.ReverseRelation["Event"]

    def name_length(self) -> int:
        """
        Computed length of name
        """
        return len(self.name)

    def events_num(self) -> int:
        """
        Computed team size
        """
        try:
            return len(self.events)
        except NoValuesFetched:
            return -1

    class PydanticMeta:
        # Let's exclude the created timestamp
        exclude = ("created_at",)
        # Let's include two callables as computed columns
        computed = ("name_length", "events_num")


class Event(Model):
    """
    This references an Event in a Tournament
    """

    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=100)
    created_at = fields.DatetimeField(auto_now_add=True)

    tournament = fields.ForeignKeyField(
        "models.Tournament", related_name="events", description="The Tournament this happens in"
    )

    class Meta:
        ordering = ["name"]

    class PydanticMeta:
        exclude = ("created_at",)

There is much to unpack here.

Firstly, we defined a PydanticMeta block, and in there is configuration options for the pydantic model creator. See tortoise.contrib.pydantic.creator.PydanticMeta for the available options.

Secondly, we excluded created_at in both models, as we decided it provided no benefit.

Thirly, we added two callables: name_length and events_num. We want these as part of the result set. Note that callables/computed fields require manual specification of return type, as without this we can’t determine the record type which is needed to create a valid Pydantic schema. This is not needed for standard Tortoise ORM fields, as the fields already define a valid type.

Note that the Pydantic serializer can’t call async methods, but since the tortoise helpers pre-fetch relational data, it is available before serialization. So we don’t need to await the relation. We should however protect against the case where no prefetching was done, hence catching and handling the tortoise.exceptions.NoValuesFetched exception.

Next we create our Pydantic Model using pydantic_model_creator:

from tortoise import Tortoise

Tortoise.init_models(["__main__"], "models")
Tournament_Pydantic = pydantic_model_creator(Tournament)

The JSON-Schema of Tournament_Pydantic is now:

{
    "title": "Tournament",
    "description": "This references a Tournament",
    "type": "object",
    "properties": {
        "id": {
            "title": "Id",
            "type": "integer"
        },
        "name": {
            "title": "Name",
            "type": "string"
        },
        "events": {
            "title": "Events",
            "description": "The Tournament this happens in",
            "type": "array",
            "items": {
                "$ref": "#/definitions/Event"
            }
        },
        "name_length": {
            "title": "Name Length",
            "description": "Computes length of name",
            "type": "integer"
        },
        "events_num": {
            "title": "Events Num",
            "description": "Computes team size.",
            "type": "integer"
        }
    },
    "definitions": {
        "Event": {
            "title": "Event",
            "description": "This references an Event in a Tournament",
            "type": "object",
            "properties": {
                "id": {
                    "title": "Id",
                    "type": "integer"
                },
                "name": {
                    "title": "Name",
                    "type": "string"
                }
            }
        }
    }
}

Note that created_at is removed, and name_length & events_num is added.

Lets create and serialise the objects and see what they look like (in an async context):

# Create objects
tournament = await Tournament.create(name="New Tournament")
await Event.create(name="Event 1", tournament=tournament)
await Event.create(name="Event 2", tournament=tournament)

# Serialise Tournament
tourpy = await Tournament_Pydantic.from_tortoise_orm(tournament)

>>> print(tourpy.model_dump_json())
{
    "id": 1,
    "name": "New Tournament",
    "events": [
        {
            "id": 1,
            "name": "Event 1"
        },
        {
            "id": 2,
            "name": "Event 2"
        }
    ],
    "name_length": 14,
    "events_num": 2
}

Creators

tortoise.contrib.pydantic.creator.pydantic_model_creator(cls, *, name=None, exclude=(), include=(), computed=(), optional=(), allow_cycles=None, sort_alphabetically=None, _stack=(), exclude_readonly=False, meta_override=None, model_config=None, validators=None, module='tortoise.contrib.pydantic.creator')[source]

Function to build Pydantic Model off Tortoise Model.

Parameters:
_stack=()

Internal parameter to track recursion

cls

The Tortoise Model

name=None

Specify a custom name explicitly, instead of a generated name.

exclude=()

Extra fields to exclude from the provided model.

include=()

Extra fields to include from the provided model.

computed=()

Extra computed fields to include from the provided model.

optional=()

Extra optional fields for the provided model.

allow_cycles=None

Do we allow any cycles in the generated model? This is only useful for recursive/self-referential models.

A value of False (the default) will prevent any and all backtracking.

sort_alphabetically=None

Sort the parameters alphabetically instead of Field-definition order.

The default order would be:

  • Field definition order +

  • order of reverse relations (as discovered) +

  • order of computed functions (as provided).

exclude_readonly=False

Build a subset model that excludes any readonly fields

meta_override=None

A PydanticMeta class to override model’s values.

model_config=None

A custom config to use as pydantic config.

validators=None

A dictionary of methods that validate fields.

module='tortoise.contrib.pydantic.creator'

The name of the module that the model belongs to.

Note: Created pydantic model uses config_class parameter and PydanticMeta’s

config_class as its Config class’s bases(Only if provided!), but it ignores fields config. pydantic_model_creator will generate fields by include/exclude/computed parameters automatically.

Return type:

Type[PydanticModel]

tortoise.contrib.pydantic.creator.pydantic_queryset_creator(cls, *, name=None, exclude=(), include=(), computed=(), allow_cycles=None, sort_alphabetically=None)[source]

Function to build a Pydantic Model list off Tortoise Model.

Parameters:
cls

The Tortoise Model to put in a list.

name=None

Specify a custom name explicitly, instead of a generated name.

The list generated name is currently naive and merely adds a “s” to the end of the singular name.

exclude=()

Extra fields to exclude from the provided model.

include=()

Extra fields to include from the provided model.

computed=()

Extra computed fields to include from the provided model.

allow_cycles=None

Do we allow any cycles in the generated model? This is only useful for recursive/self-referential models.

A value of False (the default) will prevent any and all backtracking.

sort_alphabetically=None

Sort the parameters alphabetically instead of Field-definition order.

The default order would be:

  • Field definition order +

  • order of reverse relations (as discovered) +

  • order of computed functions (as provided).

Return type:

Type[PydanticListModel]

PydanticMeta

class tortoise.contrib.pydantic.creator.PydanticMeta[source]

The PydanticMeta class is used to configure metadata for generating the pydantic Model.

Usage:

class Foo(Model):
    ...

    class PydanticMeta:
        exclude = ("foo", "baa")
        computed = ("count_peanuts", )
allow_cycles : bool = False

Allow cycles in recursion - This can result in HUGE data - Be careful! Please use this with exclude/include and sane max_recursion

backward_relations : bool = True

Use backward relations without annotations - not recommended, it can be huge data without control

computed : tuple[str, ...] = ()

Computed fields can be listed here to use in pydantic model

exclude : tuple[str, ...] = ('Meta',)

Fields listed in this property will be excluded from pydantic model

exclude_raw_fields : bool = True

If we should exclude raw fields (the ones have _id suffixes) of relations

include : tuple[str, ...] = ()

If not empty, only fields this property contains will be in the pydantic model

max_recursion : int = 3

Maximum recursion level allowed

model_config : ConfigDict | None = None

Allows user to specify custom config for generated model

sort_alphabetically : bool = False

Sort fields alphabetically. If not set (or False) then leave fields in declaration order

Model classes

class tortoise.contrib.pydantic.base.PydanticListModel(root=PydanticUndefined, **data)[source]

Pydantic BaseModel for List of Tortoise Models

This provides an extra method above the usual Pydantic model properties

async classmethod from_queryset(queryset)[source]

Returns a serializable pydantic model instance that contains a list of models, from the provided queryset.

This will prefetch all the relations automatically.

Parameters:
queryset : QuerySet

a queryset on the model this PydanticListModel is based on.

Return type:

PydanticListModel

model_computed_fields : ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config : ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields : ClassVar[dict[str, FieldInfo]] = {'root': FieldInfo(annotation=~RootModelRootType, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class tortoise.contrib.pydantic.base.PydanticModel(**data)[source]

Pydantic BaseModel for Tortoise objects.

This provides an extra method above the usual Pydantic model properties

async classmethod from_queryset(queryset)[source]

Returns a serializable pydantic model instance that contains a list of models, from the provided queryset.

This will prefetch all the relations automatically.

Parameters:
queryset : QuerySet

a queryset on the model this PydanticModel is based on.

Return type:

List[PydanticModel]

async classmethod from_queryset_single(queryset)[source]

Returns a serializable pydantic model instance for a single model from the provided queryset.

This will prefetch all the relations automatically.

Parameters:
queryset : QuerySetSingle

a queryset on the model this PydanticModel is based on.

Return type:

PydanticModel

async classmethod from_tortoise_orm(obj)[source]

Returns a serializable pydantic model instance built from the provided model instance.

Note

This will prefetch all the relations automatically. It is probably what you want.

If you don’t want this, or require a sync method, look to using .from_orm().

In that case you’d have to manage prefetching yourself, or exclude relational fields from being part of the model using tortoise.contrib.pydantic.creator.PydanticMeta, or you would be getting OperationalError exceptions.

This is due to how the asyncio framework forces I/O to happen in explicit await statements. Hence we can only do lazy-fetching during an awaited method.

Parameters:
obj

The Model instance you want serialized.

Return type:

PydanticModel

model_computed_fields : ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config : ClassVar[ConfigDict] = {'from_attributes': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields : ClassVar[dict[str, FieldInfo]] = {}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.