Python-asyncio异步编程基础

前言

asyncio是Python 3.4版本引入的标准库，直接内置了对异步IO的支持，并且如今asyncio的单线程异步性能已经做到与Go / Node持平

目前还没有基于asyncio开发大型项目的经历（主要还是在用golang）

需要了解的几个词

• 协程（coroutine）：与线程很相似，不同之处在于多协程是同一个线程来执行的，这样就省去了线程切换的时间，而且不需要多线程的锁机制了，执行效率高很多
• 异步IO：异步IO的概念和同步IO相对。当一个异步过程调用发出后，调用者不能立刻得到结果。实际处理这个调用的部件在完成后，通过状态、通知和回调来通知调用者
• 事件循环：事件循环是一种处理多并发量的有效方式，在维基百科中它被描述为「一种等待程序分配事件或消息的编程架构」，我们可以定义事件循环来简化使用轮询方法来监控事件，通俗的说法就是「当A发生时，执行B」

asyncio 是什么

asyncio模块提供了使用协程构建并发应用的工具，它使用一种单线程单进程的的方式实现并发，应用的各个部分彼此合作, 可以显式的切换任务，一般会在程序阻塞I/O操作的时候发生上下文切换如等待读写文件,或者请求网络；同时asyncio也支持调度代码在将来的某个特定时间运行，从而支持一个协程等待另一个协程完成，以处理系统信号和识别其他一些事件

其他的并发模型大多数采取线性的方式编写，并且依赖于语言运行时系统 / 操作系统的底层线程 / 进程来适当地改变上下文，而基于asyncio的应用要求应用代码显式地处理上下文切换

asyncio基本使用

asyncio的核心编程模型就是一个消息循环，我们从asyncio模块中直接获取一个EventLoop的引用，然后把需要执行的协程扔到EventLoop中执行，就实现了异步IO

import threading
import asyncio

@asyncio.coroutine
def my_coroutine():
    print(f'I ain\'t got no money ({threading.currentThread()})')
    yield from asyncio.sleep(1)
    print(f'Do not go gentle into that good night ({threading.currentThread()})')


if __name__ == '__main__':
    loopEvent = asyncio.get_event_loop()
    tasks = [my_coroutine(), my_coroutine()]
    loopEvent.run_until_complete(asyncio.wait(tasks))
    loopEvent.close()

输出：

I ain't got no money (<_MainThread(MainThread, started 14260)>)
I ain't got no money (<_MainThread(MainThread, started 14260)>)
Do not go gentle into that good night (<_MainThread(MainThread, started 14260)>)
Do not go gentle into that good night (<_MainThread(MainThread, started 14260)>)

由打印的当前线程id可以看出，两个coroutine是由同一个线程并发执行的

如果把asyncio.sleep()换成真正的IO操作，则多个coroutine就可以由一个线程并发执行

async / await

Python 3.5开始引入了新的语法async和await，可以让coroutine的代码更简洁易读

用起来也很简单，只需要做两件事情：

• 把@asyncio.coroutine替换为async
• 把yield from替换为await

更新后的代码：

import threading
import asyncio


async def my_coroutine():
    print(f'I ain\'t got no money ({threading.currentThread()})')
    await asyncio.sleep(1)
    print(f'Do not go gentle into that good night ({threading.currentThread()})')


if __name__ == '__main__':
    loopEvent = asyncio.get_event_loop()
    tasks = [my_coroutine(), my_coroutine()]
    loopEvent.run_until_complete(asyncio.wait(tasks))
    loopEvent.close()

简洁 & 优雅

协程中调用普通函数

在协程中可以通过调用EventLoop对象的call_soon，call_later，call_at方法来调用普通函数

call_soon

字面意思，立即调用

call_soon(self, callback, *args, context=None)

Arrange for a callback to be called as soon as possible.This operates as a FIFO queue: callbacks are called in theorder in which they are registered. Each callback will becalled exactly once.Any positional arguments after the callback will be passed tothe callback when it is called.

在下一个迭代的时间循环中立刻调用回调函数，看代码：

import threading
import asyncio


def callback(args):
    print(f'callback: {args} ({threading.currentThread()})')


async def my_coroutine(loop):
    print(f'I ain\'t got no money ({threading.currentThread()})')
    await asyncio.sleep(1)
    loop.call_soon(callback, 'first time')
    loop.call_soon(callback, 'second time')
    print(f'Do not go gentle into that good night ({threading.currentThread()})')


if __name__ == '__main__':
    loopEvent = asyncio.get_event_loop()
    tasks = [my_coroutine(loopEvent), my_coroutine(loopEvent)]
    loopEvent.run_until_complete(asyncio.wait(tasks))
    loopEvent.close()

输出：

I ain't got no money (<_MainThread(MainThread, started 11056)>)
I ain't got no money (<_MainThread(MainThread, started 11056)>)
Do not go gentle into that good night (<_MainThread(MainThread, started 11056)>)
Do not go gentle into that good night (<_MainThread(MainThread, started 11056)>)
callback: first time (<_MainThread(MainThread, started 11056)>)
callback: second time (<_MainThread(MainThread, started 11056)>)
callback: first time (<_MainThread(MainThread, started 11056)>)
callback: second time (<_MainThread(MainThread, started 11056)>)

call_later

事件循环在delay一定时间后执行callback函数

call_later(self, delay, callback, *args, context=None)

Arrange for a callback to be called at a given time.Return a Handle: an opaque object with a cancel() method thatcan be used to cancel the call.The delay can be an int or float, expressed in seconds. It isalways relative to the current time.Each callback will be called exactly once. If two callbacksare scheduled for exactly the same time, it undefined whichwill be called first.Any positional arguments after the callback will be passed tothe callback when it is called.

直接看代码：

import threading
import asyncio


def callback(args):
    print(f'callback: {args} ({threading.currentThread()})')


async def my_coroutine(loop):
    print(f'I ain\'t got no money ({threading.currentThread()})')
    loop.call_later(0.4, callback, 'second time')
    loop.call_soon(callback, 'first time')
    await asyncio.sleep(0.5)
    print(f'Do not go gentle into that good night ({threading.currentThread()})')


if __name__ == '__main__':
    loopEvent = asyncio.get_event_loop()
    tasks = [my_coroutine(loopEvent), my_coroutine(loopEvent)]
    loopEvent.run_until_complete(asyncio.wait(tasks))
    loopEvent.close()

输出：

I ain't got no money (<_MainThread(MainThread, started 17904)>)
I ain't got no money (<_MainThread(MainThread, started 17904)>)
callback: first time (<_MainThread(MainThread, started 17904)>)
callback: first time (<_MainThread(MainThread, started 17904)>)
callback: second time (<_MainThread(MainThread, started 17904)>)
callback: second time (<_MainThread(MainThread, started 17904)>)
Do not go gentle into that good night (<_MainThread(MainThread, started 17904)>)
Do not go gentle into that good night (<_MainThread(MainThread, started 17904)>)

call_at

call_at(self, when, callback, *args, context=None)

Like call_later(), but uses an absolute time.

Absolute time corresponds to the event loop’s time() method.

call_at第一个参数的含义代表的是一个单调时间，它和我们平时说的系统时间有点差异，

这里的时间指的是事件循环内部时间，可以通过loop.time()获取，然后可以在此基础上进行操作。后面的参数和前面的两个方法一样。实际上call_later内部就是调用的call_at

上代码：

import threading
import asyncio


def callback(args):
    print(f'callback: {args} ({threading.currentThread()})')


async def my_coroutine(loop):
    print(f'I ain\'t got no money ({threading.currentThread()})')
    loop.call_at(loop.time() + 0.2, callback, 'second time')
    loop.call_soon(callback, 'first time')
    await asyncio.sleep(0.5)
    print(f'Do not go gentle into that good night ({threading.currentThread()})')


if __name__ == '__main__':
    loopEvent = asyncio.get_event_loop()
    tasks = [my_coroutine(loopEvent), my_coroutine(loopEvent)]
    loopEvent.run_until_complete(asyncio.wait(tasks))
    loopEvent.close()

输出：

I ain't got no money (<_MainThread(MainThread, started 18504)>)
I ain't got no money (<_MainThread(MainThread, started 18504)>)
callback: first time (<_MainThread(MainThread, started 18504)>)
callback: first time (<_MainThread(MainThread, started 18504)>)
callback: second time (<_MainThread(MainThread, started 18504)>)
callback: second time (<_MainThread(MainThread, started 18504)>)
Do not go gentle into that good night (<_MainThread(MainThread, started 18504)>)
Do not go gentle into that good night (<_MainThread(MainThread, started 18504)>)