upgrade-marshmallow

Upgrade marshmallow is a tool to batch modify call expression with marshmallow.fields.Field.

Free software: MIT license
Documentation: https://upgrade-marshmallow.readthedocs.io.

Problem

After upgrade marshmallow>=3.10.0, the non-reserved arguments(e.g. title, description) should passing in by metadata argument.

Otherwise, it will pop RemovedInMarshmallow4Warning(when running pytest).

Duplicate

code structure:

$ tree .
.
├── requirements.txt  # marshmallow==3.3.0
├── schemas.py
└── test_warning.py

example code(schemas.py):

from marshmallow import Schema
from marshmallow import fields


class FooSchema(Schema):
    name = fields.String(
        required=True,
        title='foo',
        description='foo name')
    relationship = fields.String(
        title='bar', description='foobar')

test code(test_warning.py):

from schemas import FooSchema


def test_warn():
    INPUT = {
        'name': 'something',
        'relationship': 'others'
    }
    schema = FooSchema()
    data = schema.load(INPUT)
    assert data == INPUT

upgrade marshmallow and duplicate this problem:

$ pip install -U marshmallow
...
Successfully installed marshmallow-3.14.1

$ pytest test_warning.py
================== test session starts ================================
platform darwin -- Python 3.8.3, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /private/tmp/_
collected 1 item

test_warning.py .                                                                                                                              [100%]

================== warnings summary =================================
.venv/lib/python3.8/site-packages/marshmallow/fields.py:218
/private/tmp/_/.venv/lib/python3.8/site-packages/marshmallow/fields.py:218: RemovedInMarshmallow4Warning: Passing field metadata as keyword arguments is deprecated. Use the explicit `metadata=...` argument instead. Additional metadata: {'title': 'foo', 'description': 'foo name'}
warnings.warn(

.venv/lib/python3.8/site-packages/marshmallow/fields.py:218
/private/tmp/_/.venv/lib/python3.8/site-packages/marshmallow/fields.py:218: RemovedInMarshmallow4Warning: Passing field metadata as keyword arguments is deprecated. Use the explicit `metadata=...` argument instead. Additional metadata: {'title': 'bar', 'description': 'foobar'}
warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================== 1 passed, 2 warnings in 0.03s ===================

Issue

In some projects, we may use a lot of Field with “title”, “description” etc. arguments.

It will take lots of effort to change each one by one manually, and it’s not easy to “replace” by editor tools or use awk/sed.

Solution

upgrade-marshmallow use AST to parsing source code, then replacing arguments with expected code.

Features

upgrade-marshmallow tool could handler many cases.

the most common(simple) way to use marshmallow.fields.Field:

from marshmallow import Schema
from marshmallow import fields
from marshmallow import post_load


class FooSchema(Schema):
    name = fields.String(description='foo name')
    compare = fields.Integer(required=False)

    @post_load
    def do_nothing(self, data, **_):
        """docstring keeps"""
        return data


def normal_fun():
    """regression test: compatible different cases without initial"""
    pass


normal_fun()

alias fields:

from marshmallow import Schema
from marshmallow import fields as ma_fields


class FooSchema(Schema):
    name = ma_fields.String(title='foo name', description='foo name')
    # TODO: nested
    # fields = ma_fields.List(ma_fields.String(title='field'))

import marshmallow, using Field with absolute path:

import marshmallow


class FooSchema(marshmallow.Schema):
    name = marshmallow.fields.String(title='foo name', description="foo name")

alias marshmallow:

import marshmallow as ma


class FooSchema(ma.Schema):
    name = ma.fields.String(title='foo name', description="foo name")

directly import to Field

from marshmallow import Schema
from marshmallow.fields import Field
from marshmallow.fields import Raw
from marshmallow.fields import String
from marshmallow.fields import UUID
from marshmallow.fields import Number
from marshmallow.fields import Integer
from marshmallow.fields import Decimal
from marshmallow.fields import Boolean
from marshmallow.fields import Float
from marshmallow.fields import DateTime
from marshmallow.fields import NaiveDateTime
from marshmallow.fields import AwareDateTime
from marshmallow.fields import Time
from marshmallow.fields import Date
from marshmallow.fields import TimeDelta
from marshmallow.fields import Url
from marshmallow.fields import URL
from marshmallow.fields import Email
from marshmallow.fields import IP
from marshmallow.fields import IPv4
from marshmallow.fields import IPv6
from marshmallow.fields import IPInterface
from marshmallow.fields import IPv4Interface
from marshmallow.fields import IPv6Interface
from marshmallow.fields import Str
from marshmallow.fields import Bool
from marshmallow.fields import Int


class FooSchema(Schema):
    field = Field(title='Field')
    raw = Raw(title='Raw')
    string = String(title='String')
    uuid = UUID(title='UUID')
    number = Number(title='Number')
    integer = Integer(title='Integer')
    decimal = Decimal(title='Decimal')
    boolean = Boolean(title='Boolean')
    float = Float(title='Float')
    datetime = DateTime(title='DateTime')
    naivedatetime = NaiveDateTime(title='NaiveDateTime')
    awaredatetime = AwareDateTime(title='AwareDateTime')
    time = Time(title='Time')
    date = Date(title='Date')
    timedelta = TimeDelta(title='TimeDelta')
    url = Url(title='Url')
    url = URL(title='URL')
    email = Email(title='Email')
    ip = IP(title='IP')
    ipv4 = IPv4(title='IPv4')
    ipv6 = IPv6(title='IPv6')
    ipinterface = IPInterface(title='IPInterface')
    ipv4interface = IPv4Interface(title='IPv4Interface')
    ipv6interface = IPv6Interface(title='IPv6Interface')
    str = Str(title='Str')
    bool = Bool(title='Bool')
    int = Int(title='Int')

rename default to dump_default, missing to load_default

from marshmallow import Schema
from marshmallow import fields


class Foo(Schema):
    name = fields.String(title='name', description='foo name')
    tag = fields.String(default='foo', missing='foo')