Python全栈（四）高级编程技巧之2.类与对象深度问题与解决技巧

文章目录

• 一、派生内置不可变类型并修改其实例化行为
• 二、创建大量实例节省内存
• 三、Python中的with语句
• 四、创建可管理的对象属性
• 五、类支持比较操作
• 六、环状数据结构中管理内存

一、派生内置不可变类型并修改其实例化行为

引入：
我们想自定义一种新类型的元组,对于传入的可迭代对象,我们只保留其中int类型且值大于0的元素，例如:
IntTuple([2,-2,'jr',['x','y'],4]) => (2,4)
如何继承内置tuple 实现IntTuple？
自定义IntTuple：

class IntTuple(tuple):
    def __init__(self,iterable):
        for i in iterable:
            if isinstance(i,int) and i > 0:
                super().__init__(i)

int_t = IntTuple([2,-2,'cl',['x','y'],4])
print(int_t)

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 7, in <module>
    int_t = IntTuple([2,-2,'cl',['x','y'],4])
  File "xxx/demo.py", line 5, in __init__
    super().__init__(i)
TypeError: object.__init__() takes exactly one argument (the instance to initialize)

问题：self对象到底是谁创建的：

class A:
    def __new__(cls, *args, **kwargs):
        print('A.__new__',cls,args)
        return object.__new__(cls)

    def __init__(self,*args):
        print('A.__init__')

a = A(1,2)

打印

A.__new__ <class '__main__.A'> (1, 2)
A.__init__

易知，真正创建对象的是__new__方法，self对象是通过__new__创建的。
上述示例默认继承object，也可以继承自其他类，__new__方法返回的是super().new(cls)。

class B:
    pass

class A(B):
    def __new__(cls, *args, **kwargs):
        print('A.__new__',cls,args)
        return super().__new__(cls)

    def __init__(self,*args):
        print('A.__init__')

a = A(1,2)

运行结果与前者相同，但最好采用第一种方式。
A的实例化可以转化为另两行等价的代码：

class A:
    def __new__(cls, *args, **kwargs):
        print('A.__new__',cls,args)
        return object.__new__(cls)

    def __init__(self,*args):
        print('A.__init__')

# a = A(1,2)相当于下面两行代码
a = A.__new__(A,1,2)
A.__init__(a,1,2)

创建列表转化等价代码：

l = list.__new__(list,'abc')
print(l)
list.__init__(l,'abc')
print(l)

打印

[]
['a', 'b', 'c']

但是元组不同：

t = tuple.__new__(tuple,'abc')
print(t)
tuple.__init__(t,'abc')
print(t)

打印

('a', 'b', 'c')
('a', 'b', 'c')

在第一步调用__new__方法时就已经实例化生成了元组。
这也解释了在最开始自定义IntTuple时__init__(self,iterable)会报错：
在__init__(self,iterable)之前调用__new__方法时元组已经被创建好了，并且元组是不能被修改的，所以会报错，不能实现预期效果。
对代码的修改：

class IntTuple(tuple):
    def __new__(cls,iterable):
        #生成器
        r = (i for i in iterable if isinstance(i,int) and i > 0)
        return super().__new__(cls,r)

int_t = IntTuple([2,-2,'cl',['x','y'],4])
print(int_t)

打印

(2, 4)

实现了预期功能。

二、创建大量实例节省内存

问题提出：
在游戏中,定义了玩家类player,每有一个在线玩家,在服务器内则有一个player的实例,当在线人数很多时,将产生大量实例(百万级)，如何降低这些大量实例的内存开销？
解决方案：
定义类的__slots__属性,声明实例有哪些属性(关闭动态绑定)。

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p2 = Player2('0002','Tom')
print(len(dir(p1)))
print(len(dir(p2)))
print(dir(p1))
print(dir(p2))
print(set(dir(p1)) - set(dir(p2)))

打印

30
29
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'level', 'name', 'status', 'uid']
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', 'level', 'name', 'status', 'uid']
{'__dict__', '__weakref__'}

显然，p1和p2的所有属性相比较，p2比p1少了属性，其中__weakref__是弱引用，__dict__是动态绑定属性，主要的内存浪费就在于此。

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p1.x = 6
p1.__dict__['y'] = 7 #p1.__dict__本身就是字典
print(p1.__dict__)

打印

{'uid': '0001', 'name': 'Tom', 'status': 0, 'level': 1, 'x': 6, 'y': 7}

可知，增加了属性x和y，此即动态绑定，即可以随意向对象添加属性。
Python可以通过这种动态绑定添加属性，但是这种方式是及其浪费内存的，添加__slots__后可消除动态绑定、解决这个问题。

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p2 = Player2('0002','Tom')
p1.x = 6
p2.y = 7
print(p1.__dict__)
print(p2.__dict__)

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 73, in <module>
    p2.y = 7
AttributeError: 'Player2' object has no attribute 'y'

即不允许再向Player2的实例任意添加属性。
导入sys模块查看具体内存：

import sys

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p1.x = 6
print(p1.__dict__)
print(sys.getsizeof(p1.__dict__))
print(sys.getsizeof(p1.uid))
print(sys.getsizeof(p1.name))
print(sys.getsizeof(p1.status))
print(sys.getsizeof(p1.level))
print(sys.getsizeof(p1.x))

打印

112
53
52
24
28
28

不允许对实例动态修改__slots__：

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

p1 = Player1('0001','Tom')
p2 = Player2('0002','Tom')
p1.x = 6
p1.__dict__['y'] = 7
p2.__slots__ = ('uid','name','status','level','y')
print(p1.__dict__)
print(p2.__dict__)

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 74, in <module>
    p2.__slots__ = ('uid','name','status','level','y')
AttributeError: 'Player2' object attribute '__slots__' is read-only

报错，__slots__是只读的。
导入库tracemalloc跟踪内存的使用：

import tracemalloc

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

tracemalloc.start()
p1 = [Player1('0001','Tom',2,3) for _ in range(100000)]
p2 = [Player1('0001','Tom',2,3) for _ in range(100000)]
end = tracemalloc.take_snapshot()
top = end.statistics('lineno')

for stat in top[:10]:
    print(stat)

打印

xxx/demo.py:59: size=21.4 MiB, count=399992, average=56 B
xxx/demo.py:74: size=6274 KiB, count=100003, average=64 B
xxx/demo.py:73: size=6274 KiB, count=100001, average=64 B
xxx/demo.py:75: size=432 B, count=1, average=432 B
xxx\Python\Python37\lib\tracemalloc.py:532: size=64 B, count=1, average=64 B

end.statistics()的参数为’lineno’时是对占内存的代码逐行分析：
59行和73行是对p1进行分析，占内存为21.4 MiB（__dict__属性内存）+6274 KiB=27.5MiB；
74行是对p2进行分析：6274 KiB=6.1MiB。
end.statistics()的参数为’filename’时是对整个文件内存进行分析：
此时需要对p1和p2分别分析：
对p1：

import tracemalloc

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

tracemalloc.start()
p1 = [Player1('0001','Tom',2,3) for _ in range(100000)]
# p2 = [Player1('0001','Tom',2,3) for _ in range(100000)]
end = tracemalloc.take_snapshot()
# top = end.statistics('lineno')
top = end.statistics('filename')

for stat in top[:10]:
    print(stat)

打印

xxx/demo.py:0: size=16.8 MiB, count=299994, average=59 B
xxx\Python\Python37\lib\tracemalloc.py:0: size=64 B, count=1, average=64 B

对p2：

import tracemalloc

class Player1(object):
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

class Player2(object):
    __slots__ = ('uid','name','status','level')
    def __init__(self,uid,name,status=0,level=1):
        self.uid = uid
        self.name = name
        self.status = status
        self.level = level

tracemalloc.start()
# p1 = [Player1('0001','Tom',2,3) for _ in range(100000)]
p2 = [Player1('0001','Tom',2,3) for _ in range(100000)]
end = tracemalloc.take_snapshot()
# top = end.statistics('lineno')
top = end.statistics('filename')

for stat in top[:10]:
    print(stat)

打印

xxx/demo.py:0: size=16.8 MiB, count=299994, average=59 B
xxx\Python\Python37\lib\tracemalloc.py:0: size=64 B, count=1, average=64 B

三、Python中的with语句

文件处理：

try:
    f = open('test.txt','w')
    raise KeyError
except KeyError as e:
    print('Key Error')
    f.close()
except IndexError as e:
    print('Index Error')
    f.close()
except Exception as e:
    print('Error')
    f.close()
finally:
    print('end')
    f.close()

打印

Key Error
end

代码中有多个except显得很冗余重复，所以引入with语句：

with open('test.txt','r') as f:
    f.read()

类关于上下文处理的尝试：

class Sample(object):
    def demo(self):
        print('This is a demo')

with Sample() as sample:
    sample.demo()

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 105, in <module>
    with Sample() as sample:
AttributeError: __enter__

报错AttributeError（没有__enter__方法），这个方法与上下文处理有关。
改进-增加__enter__方法：

class Sample(object):
    def __enter__(self):
        print('start')
        return self

    def demo(self):
        print('This is a demo')

with Sample() as sample:
    sample.demo()

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 109, in <module>
    with Sample() as sample:
AttributeError: __exit__

报错AttributeError,没有__exit__方法。
改进-增加__exit__方法：

class Sample(object):
    #获取资源
    def __enter__(self):
        print('start')
        return self

    def demo(self):
        print('This is a demo')
    
    #释放资源
    def __exit__(self, exc_type, exc_val, exc_tb):
        print('end')

with Sample() as sample:
    sample.demo()

打印

start
This is a demo
end

此时正常运行。
易知，要想用上下文处理器来处理类，要在类中增加__enter__和__exit__方法。
类中的3个方法可类比文件处理中的打开文件、处理文件、关闭文件，代替了异常处理（try…except…）语句。
__exit__的参数探索：
正常情况下：

class Sample(object):
    def __enter__(self):
        print('start')
        return self

    def demo(self):
        print('This is a demo')

    def __exit__(self, exc_type, exc_val, exc_tb):
        print('exc_type:',exc_type)
        print('exc_val:',exc_val)
        print('exc_tb:',exc_tb)
        print('end')

with Sample() as sample:
    sample.demo()

打印

start
This is a demo
exc_type: None
exc_val: None
exc_tb: None
end

有异常时：

class Sample(object):
    def __enter__(self):
        print('start')
        return self

    def demo(self):
        print('This is a demo')

    def __exit__(self, exc_type, exc_val, exc_tb):
        #异常类
        print('exc_type:',exc_type)
        #异常值
        print('exc_val:',exc_val)
        #追踪信息
        print('exc_tb:',exc_tb)
        print('end')

with Sample() as sample:
    sample.sample()

打印

Traceback (most recent call last):
start
  File "xxx/demo.py", line 119, in <module>
exc_type: <class 'AttributeError'>
    sample.sample()
exc_val: 'Sample' object has no attribute 'sample'
AttributeError: 'Sample' object has no attribute 'sample'
exc_tb: <traceback object at 0x00000210286C8388>
end

即__exit__三个参数携带了有关异常处理的信息。
contextlib库简化上下文管理器：

import contextlib

@contextlib.contextmanager
def file_open(filename):
    #相当于__enter__
    print('file open')
    yield {}
    # 相当于__exit__
    print('file close')

with file_open('test.txt') as f:
    print('file operation')

打印

file open
file operation
file close

此即上下文管理器协议；
file_open函数中必须要用yield，不能用return，否则会报错。

四、创建可管理的对象属性

在面向对象编程中,我们把方法看做对象的接口。
直接访问对象的属性可能是不安全的,或设计上不够灵活,但是使用调用方法在形式上不如访问属性简洁。
调用方法：

A.get_key()  #访问器
A.set_key()  #设置器

这种方式较麻烦。
属性访问：

A.key
A.key = 'xxx'

这种方式不安全。
寻找形式上属性访问、深圳市调用方法的方式：

class A:
    def __init__(self,age):
        self.age = age

    def get_age(self):
        return self.age

    def set_age(self,age):
        if not isinstance(age,int):
            raise TypeError('Type Error')
        self.age = age

a =A(18)
a.set_age('20')

打印

Traceback (most recent call last):
  File "xxx/demo.py", line 147, in <module>
    a.set_age('20')
  File "xxx/demo.py", line 143, in set_age
    raise TypeError('Type Error')
TypeError: Type Error

当设置年龄不为整型时会抛出异常，但是较麻烦。
property两种调用方式：
（1）property(fget=None, fset=None, fdel=None, doc=None)：

class C(object):
    def getx(self): return self._x
    def setx(self, value): self._x = value
    def delx(self): del self._x
    x = property(getx, setx, delx, "I'm the 'x' property.")

例如：

class A:
    def __init__(self,age):
        self.age = age

    def get_age(self):
        return self.age

    def set_age(self,age):
        if not isinstance(age,int):
            raise TypeError('Type Error')
        self.age = age

    R = property(get_age,set_age)

a =A(18)
#set
a.R = 20
#get
print(a.R)

打印

20

（2）加装饰器

class C(object):
    @property
    def x(self):
        "I am the 'x' property."
        return self._x
    @x.setter
    def x(self, value):
        self._x = value
    @x.deleter
    def x(self):
        del self._x

例如：

class A:
    def __init__(self,age):
        self.age = age

    def get_age(self):
        return self.age

    def set_age(self,age):
        if not isinstance(age,int):
            raise TypeError('Type Error')
        self.age = age

    R = property(get_age,set_age)

    @property
    def S(self):
        return self.age

    @S.setter
    def S(self,age):
        if not isinstance(age,int):
            raise TypeError('Type Error')
        self.age = age

a =A(18)
#set
a.S = 20
#get
print(a.S)

打印

20

五、类支持比较操作

有时我们希望自定义类的实例间可以使用>、<、<=、>=、==、!=符号进行比较,例如,有一个矩形的类,比较两个矩形的实例时,比较的是它们的面积。

a = 1
b = 2
print(a > b)
print(a < b)
print(a.__ge__(b))
print(a.__lt__(b))
c = 'abc'
d = 'abd'
print(c > d)
print(c < d)
print(c.__ge__(d))
print(c.__lt__(d))
e = {1,2,3}
f = {1,2}
g = {1,4}
#集合比较的是包含，包含则为True，否则为False
print(e > f)
print(e > g)

打印

False
True
False
True
False
True
False
True
True
False

在类中实现比较：

class Rect(object):
    def __init__(self,w,b):
        self.w = w
        self.b = b

    def area(self):
        return self.w * self.b

    def __str__(self):
        return '(%s,%s)' % (self.w,self.b)

    def __lt__(self, other):
        return self.area() < other.area()

    def __gt__(self, other):
        return self.area() > other.area()


rect1 = Rect(1,2)
rect2 = Rect(3,4)
print(rect1 < rect2)

打印

True

__lt__和__gt__方法可省去其中一个，也能实现大小的比较，如：

class Rect(object):
    def __init__(self,w,b):
        self.w = w
        self.b = b

    def area(self):
        return self.w * self.b

    def __str__(self):
        return '(%s,%s)' % (self.w,self.b)

    def __lt__(self, other):
        return self.area() < other.area()


rect1 = Rect(1,2)
rect2 = Rect(3,4)
print(rect1 > rect2)

打印

False

原理解释：
在类Rect中只实现了小于比较，Python内部实现了自动转化rect2 < rect1，而事实上该不等式不成立，所以为False，所以返回False。
但是要实现等于或者其他比较，还需要在类中实现相应的方法，很麻烦。
导入库对类加装饰器：

from functools import total_ordering

@total_ordering
class Rect(object):
    def __init__(self,w,b):
        self.w = w
        self.b = b

    def area(self):
        return self.w * self.b

    def __str__(self):
        return '(%s,%s)' % (self.w,self.b)

    def __lt__(self, other):
        return self.area() < other.area()

    def __eq__(self, other):
        return self.area() == other.area()


rect1 = Rect(1,2)
rect2 = Rect(3,4)
print(rect1 > rect2)
print(rect1 < rect2)
print(rect1 >= rect2)
print(rect1 <= rect2)
print(rect1 == rect2)
print(rect1 != rect2)

打印

False
True
False
True
False
True

即此时只需要在类中实现__lt__和__eq__两个方法即可实现所有类型的比较。
两个类之间比较：

from functools import total_ordering
import math

@total_ordering
class Rect(object):
    def __init__(self,w,b):
        self.w = w
        self.b = b

    def area(self):
        return self.w * self.b

    def __str__(self):
        return '(%s,%s)' % (self.w,self.b)

    def __lt__(self, other):
        return self.area() < other.area()

    def __eq__(self, other):
        return self.area() == other.area()

class Cricle(object):
    def __init__(self,r):
        self.r = r

    def area(self):
        return math.pi * pow(self.r,2)

    def __lt__(self, other):
        return self.area() < other.area()

    def __eq__(self, other):
        return self.area() == other.area()

rect = Rect(1,2)
c = Cricle(1)
print(rect > c)
print(rect < c)
print(rect == c)

打印

False
True
False

对比较进行优化：用抽象基类简化代码

from functools import total_ordering
import math
import abc

@total_ordering
class Shape(metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def area(self):
        pass

    def __lt__(self, other):
        return self.area() < other.area()

    def __eq__(self, other):
        return self.area() == other.area()


class Rect(Shape):
    def __init__(self,w,b):
        self.w = w
        self.b = b

    def area(self):
        return self.w * self.b


class Cricle(Shape):
    def __init__(self,r):
        self.r = r

    def area(self):
        return math.pi * pow(self.r,2)


rect = Rect(1,2)
c = Cricle(1)
print(rect > c)
print(rect < c)
print(rect == c)

执行结果与之前相同。

六、环状数据结构中管理内存

垃圾回收机制：

class A(object):
    #被释放时执行
    def __del__(self):
        print('del')

a = A()
print('after a = A()')
a2 = a
print('after a2 = a')
a2 = None
print('after a2 = None')
a = None
print('after a = None')

打印

after a = A()
after a2 = a
after a2 = None
del
after a = None

易知，在a被改变时A的实例被释放回收；
垃圾回收机制针对的是引用计数。
双向循环链表：

class Node:
    def __init__(self, data):
        self.data = data
        self.left = None
        self.right = None

    def add_right(self, node):
        self.right = node
        node.left = self

    def __str__(self):
        return 'Node:<%s>' % self.data

    def __del__(self):
        print('in __del__: delete %s' % self)

def create_linklist(n):
    head = current = Node(1)
    for i in range(2, n + 1):
        node = Node(i)
        current.add_right(node)
        current = node
    return head

#创建节点1000次
head = create_linklist(1000)
#释放节点
head = None

import time
for _ in range(1000):
    time.sleep(1)
    print('run...')
input('wait...')

在命令行执行Python文件，并强制结束时，会打印

run...
run...
run...
run...
run...
run...
run...
Traceback (most recent call last):
  File "demo.py", line 267, in <module>
    time.sleep(1)
KeyboardInterrupt
in __del__: delete Node:<999>
in __del__: delete Node:<1000>
in __del__: delete Node:<1>
in __del__: delete Node:<296>
in __del__: delete Node:<297>
in __del__: delete Node:<2>
...
in __del__: delete Node:<993>
in __del__: delete Node:<994>
in __del__: delete Node:<995>
in __del__: delete Node:<996>
in __del__: delete Node:<997>
in __del__: delete Node:<998>

即回收时会调用__del__。
弱引用：

import weakref

class A(object):
    def __del__(self):
        print('del')

a = A()
print('after a = A()')
a2 = weakref.ref(a)
print('after a2 = weakref.ref(a)')
a3 = a2()
print('after a3 = a2()')
del a3
print('after del a3')

打印

after a = A()
after a2 = weakref.ref(a)
after a3 = a2()
after del a3
del

弱引用会在所有执行结束后再调用__del__；
弱引用不占用引用计数，Python中的垃圾回收是根据引用计数来的。
a2即是弱引用，a3是真实的引用。

上述双向链表可修改小部分代码即可变为弱引用：

import weakref

class Node:
    def __init__(self, data):
        self.data = data
        self.left = None
        self.right = None

    def add_right(self, node):
        self.right = node
        # node.left = self
        node.left = weakref.ref(self)

    def __str__(self):
        return 'Node:<%s>' % self.data

    def __del__(self):
        print('in __del__: delete %s' % self)

def create_linklist(n):
    head = current = Node(1)
    for i in range(2, n + 1):
        node = Node(i)
        current.add_right(node)
        current = node
    return head

#创建节点1000次
head = create_linklist(1000)
#释放节点
head = None

import time
for _ in range(1000):
    time.sleep(1)
    print('run...')
input('wait...')

打印

in __del__: delete Node:<1>
in __del__: delete Node:<2>
in __del__: delete Node:<3>
in __del__: delete Node:<4>
in __del__: delete Node:<5>
in __del__: delete Node:<6>
...
in __del__: delete Node:<995>
in __del__: delete Node:<996>
in __del__: delete Node:<997>
in __del__: delete Node:<998>
in __del__: delete Node:<999>
in __del__: delete Node:<1000>
run...
run...
run...
run...
run...
run...
...

此时与之前的运行结果不一样，先调用__del__释放对象再运行后面的代码。

来源：CSDN

作者：cupyter

链接：https://blog.csdn.net/CUFEECR/article/details/104117162