Detailed explanation of enhanced arithmetic assignment '-=' operation

Related learning recommendations: python tutorial

Preface

This article is a series of articles on Python syntax sugar one. The latest source code can be found in the desugar project (github.com/brettcannon…

Introduction

Python has something called enhanced arithmetic assignment(augmented arithmetic assignment) . Maybe you are not familiar with this name. It actually means assignment while doing mathematical operations. For example, a -= b is the enhanced arithmetic assignment of subtraction.

Enhanced assignment was added in Python 2.0 version. (Annotation: Introduced in PEP-203)

Analysis-=

Because Python does not allow overwriting assignments, compared to other operations that have special/magic methods , the way it implements enhanced assignment may not be exactly the same as you imagine.

First of all, you must know that a -= b is semantically the same as a = a-b. But also realize that if you know in advance that you want to assign an object to a variable name, it may be more efficient than a - b blind operation.

For example, at least The advantage is that you can avoid creating a new object: if an object can be modified in place, then returning self is more efficient than reconstructing a new object.

Therefore, Python provides a __isub__() method. If it is defined on the left side of the assignment (usually called an lvalue), the value on the right side (usually called an rvalue) will be called. So for a -= b, it will try to call a. __isub__(b).

If the result of the call is NotImplemented, or there is no result at all, then Python will fall back to regular binary arithmetic operations: a - b. Article on binary operations, the translation is here)

In the end, no matter which method is used, the return value will be assigned to a.

The following is a simple pseudo code,a - = b is decomposed into:

# 实现 a -= b 的伪代码if hasattr(a, "__isub__"):
    _value = a.__isub__(b)    if _value is not NotImplemented:
        a = _value    else:
        a = a - b    del _value else:
     a = a - b复制代码

Generalize these methods

Since we already implemented binary arithmetic, it is not too complicated to generalize the enhanced arithmetic operation.

By passing in a binary arithmetic operation function, and doing some introspection (and handling possible TypeErrors), it can be neatly summarized as:

def _create_binary_inplace_op(binary_op: _BinaryOp) -> Callable[[Any, Any], Any]:

    binary_operation_name = binary_op.__name__[2:-2]
    method_name = f"__i{binary_operation_name}__"
    operator = f"{binary_op._operator}="

    def binary_inplace_op(lvalue: Any, rvalue: Any, /) -> Any:
        lvalue_type = type(lvalue)        try:
            method = debuiltins._mro_getattr(lvalue_type, method_name)        except AttributeError:            pass
        else:
            value = method(lvalue, rvalue)            if value is not NotImplemented:                return value        try:            return binary_op(lvalue, rvalue)        except TypeError as exc:            # If the TypeError is due to the binary arithmetic operator, suppress
            # it so we can raise the appropriate one for the agumented assignment.
            if exc._binary_op != binary_op._operator:                raise
        raise TypeError(            f"unsupported operand type(s) for {operator}: {lvalue_type!r} and {type(rvalue)!r}"
        )

    binary_inplace_op.__name__ = binary_inplace_op.__qualname__ = method_name
    binary_inplace_op.__doc__ = (        f"""Implement the augmented arithmetic assignment `a {operator} b`."""
    )    return binary_inplace_op复制代码

This makes the definition of -= support _create_binary_inplace_op(__ sub__) , and can infer other things: the function name, what __i*__ function is called, and which callable to call when a binary arithmetic operation goes wrong.

I find that almost no one uses it**=

While writing the code for this article, I encountered a strange test error of **=. Of all the tests that ensure that __pow__ is called appropriately, there is one that fails for the operator module in the Python standard library.

My code is usually fine, and if there are differences between the code and CPython's code, it usually means I'm doing something wrong.

However, no matter how carefully I troubleshoot the code, I can't pinpoint why my tests pass and the standard library fails.

I decided to take a closer look at what's going on under the hood of CPython. Starting with the disassembled bytecode:

>>> def test(): a **= b... >>> import dis>>> dis.dis(test)  1           0 LOAD_FAST                0 (a)              2 LOAD_GLOBAL              0 (b)              4 INPLACE_POWER              6 STORE_FAST               0 (a)              8 LOAD_CONST               0 (None)             10 RETURN_VALUE复制代码

Through it, I found INPLACE_POWER in the eval loop:

        case TARGET(INPLACE_POWER): {
            PyObject *exp = POP();
            PyObject *base = TOP();
            PyObject *res = PyNumber_InPlacePower(base, exp, Py_None);
            Py_DECREF(base);
            Py_DECREF(exp);
            SET_TOP(res);            if (res == NULL)                goto error;
            DISPATCH();
        }复制代码

Source: github.com/python/cpyt …

Then find PyNumber_InPlacePower():

PyObject *PyNumber_InPlacePower(PyObject *v, PyObject *w, PyObject *z){    if (v->ob_type->tp_as_number &&
        v->ob_type->tp_as_number->nb_inplace_power != NULL) {        return ternary_op(v, w, z, NB_SLOT(nb_inplace_power), "**=");
    }    else {        return ternary_op(v, w, z, NB_SLOT(nb_power), "**=");
    }
}复制代码

Source: github.com/python/cpyt…

Heaved a sigh of relief~The code shows if defined If __ipow__ is present, it will be called, but __pow__ will be called only if there is no __ipow__.

However, the correct approach should be: If there is a problem when calling __ipow__, NotImplemented is returned or there is no return at all, then __pow__ and __rpow__ should be called.

In other words, the above code unexpectedly skips the fallback semantics of a**b when __ipow__ is present!

Actually, this issue was partially discovered and a bug submitted about 11 months ago. I fixed the issue and documented it on python-dev.

As of now, it seems that this will be fixed in Python 3.10, we also need to add a notice about **= being buggy in the documentation for 3.8 and 3.9 (the problem may be early, but older The Python version is already in security-only maintenance mode, so the documentation will not change).

The fixed code will most likely not be ported because it is a semantic change, and it is difficult to tell if someone accidentally relied on the problematic semantics. But it took so long for this problem to be noticed, which suggests that **= is not widely used, otherwise the problem would have been discovered long ago.

If you want to know more about programming learning, please pay attention to the php training column!

The above is the detailed content of Detailed explanation of enhanced arithmetic assignment '-=' operation. For more information, please follow other related articles on the PHP Chinese website!