In C#, it is really easy to convert between class and JSON object, which brings tremendous convenience for Web development. Recently, I have to use a Python framework Django, and found that there’s no such feature as C#, so I implemented a shabby replica for that by myself.

What you will learn

In this article, you will get both a serializer and a de-serializer in Python to convert JSON string or dict object into or from a Python object. 😋

Prerequisites

To start with, you must keep one concept in mind,

"Python object is merely a dictionary!"

Not that absolute, but generally speaking, yes.

We do not begin with scratch, and Python already make json package part of its standard. So we just use it for fundamental parsing.

Let’s Start Serialization!

1. JSON Encoder & Decoder

For basic conversion between JSON string and Python dict, we can simply use json module to do the stuffs. However, if you have datetime field in your Python object, you have to manually convert it to string, as JSON doesn’t have corresponding datetime value.

When you use json.dump, you need a custom encoder to handle datetime field.

1
2
3
4
5
6
7
class AdvancedEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return obj.strftime("%Y-%m-%d %H:%M:%S")
        elif isinstance(obj, datetime.date):
            return obj.strftime("%Y-%m-%d")
        return obj.__dict__

Notice that, datetime is derived from date, so you should place instance check of datetime ahead of date. And the format is custom, just make sure they corresponds each other in Encoder and Decoder. 🫡

Then, correspondingly, when you use json.loads, you need a custom decoder. However, what is different is that we need to override its object_hook to tell it to try convert str into datetime object. (It seems, in previous version, it is a method, instead of a member.)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
def object_hook(obj):
    for key, value in obj.items():
        if isinstance(value, str):
            try:
                obj[key] = datetime.datetime.strptime(value, '%Y-%m-%d %H:%M:%S')
            except ValueError:
                pass
            else:
                continue
            try:
                obj[key] = datetime.datetime.strptime(value, '%Y-%m-%d')
            except ValueError:
                pass
            else:
                continue
    return obj


class AdvancedDecoder(json.JSONDecoder):
    def __init__(self):
        super().__init__(object_hook=object_hook)

2. Serialization

Then, with JSONEncoder, you can simply serialize a object into JSON string.

1
2
3
4
5
def serialize(obj) -> str:
    try:
        return json.dumps(obj, cls=AdvancedEncoder)
    except Exception:
        raise JsonSerializeException("Failed to serialize", obj)

Serialization related exceptions will be talked about at the end, since they are not the main topic.

If you don’t want to serialize a object into raw string, you can also serialize it into a dict.

1
2
3
4
5
6
7
def serialize_as_dict(obj) -> dict:
    if not hasattr(obj, '__dict__'):
        return obj
    ret = obj.__dict__
    for key in obj.__dict__.keys():
        ret[key] = serialize_as_dict(obj.__dict__[key])
    return ret

Well, I guess you can also achieve this by serialize it first into a string, then deserialize it into a dict, which will then need no extra work.

3. Deserialization

Compared to serialization, deserialization got one more problem - how to deserialize deserialize a JSON object into a object with desired class? And this is the key point of the article.

Then, the deserialization can be implemented as such. If cls is assigned, it will try to convert JSON string or object into the given class. Otherwise, it will simply return a dict object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
def deserialize(obj, cls=None):
    if isinstance(obj, dict):
        pass
    elif isinstance(obj, str):
        try:
            obj = json.loads(obj)
        except:
            raise JsonDeserializeException("Failed to deserialize", obj)
    else:
        raise JsonDeserializeException("Type not supported", None)

    if cls is None:
        return obj

    try:
        _check_type(obj, cls())
        obj = _construct_cls(obj, cls)
    except AttributeError as e:
        print(e)
        raise JsonDeserializeException(f"Type mismatch, should be {cls.__name__}", obj)

    return obj

There are two functions that play an important role here, _check_type and _construct_cls. They make sure the JSON object strictly match the given class, and try to build such a class from the JSON object. The implementation of them may be a little hard to understand, though. 😣

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
def _check_type(dict_obj, cls_obj) -> bool:
    if not dict_obj.keys() == cls_obj.__dict__.keys():
        hint = "Attribute set does not match:\n\t"
        hint += f"Expected: {cls_obj.__dict__.keys()}\n\t"
        hint += f"     Got: {dict_obj.keys()}"
        raise AttributeError(hint)

    for key in cls_obj.__dict__.keys():
        cls_type = type(cls_obj.__dict__[key])
        if isinstance(dict_obj[key], cls_type):
            continue

        try:
            if not isinstance(dict_obj[key], dict):
                hint = "Attribute type mismatch:\n\t"
                hint += f"Expected: Attribute '{key}' should be '{type(cls_obj.__dict__[key])}'\n\t"
                hint += f"     Got: '{dict_obj[key]}' of type {type(dict_obj[key])}"
                raise AttributeError(hint)

            _check_type(dict_obj[key], cls_obj.__dict__[key])
        except AttributeError as e:
            raise e
    return True


def _construct_cls(dict_obj, cls):
    model = cls()
    obj = cls()
    for key in model.__dict__.keys():
        if isinstance(dict_obj[key], type(model.__dict__[key])):
            obj.__dict__[key] = dict_obj[key]
            continue
        attr = _construct_cls(dict_obj[key], type(model.__dict__[key]))
        if attr is None:
            raise AttributeError(f"Missing '{key}'")
        obj.__dict__[key] = attr
    return obj

So… again, Python object is just a dict with some extra fields. Emm… Hope it could make it easier for you to understand the code. 🥺

4. Exception Declaration

At last, I present to you the definition of custom exceptions that I used to fit the last piece.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
class JsonException(Exception):
    def __init__(self, msg):
        self.msg = msg

    def __str__(self):
        return f"JSON Error: {self.msg}"


class JsonSerializeException(JsonException):
    def __init__(self, msg, obj):
        super().__init__(msg)
        self.obj = obj

    def __str__(self):
        _str = super().__str__()
        _str += f"\n\tOn object: {'None' if self.obj is None else self.obj.__dict__}"
        return _str
    
    
class JsonDeserializeException(JsonException):
    def __init__(self, msg, obj):
        super().__init__(msg)
        self.obj = obj

    def __str__(self):
        _str = super().__str__()
        _str += f"\n\tOn string: {'None' if self.obj is None else self.obj}"
        return _str

So, this is it. And… I guess that Python is not that diabolical. It can be convenient sometimes. Only, some times. 😶‍🌫️