作为一个学习练习,我正在尝试修改aiohttp的快速入门示例以使用单个ClientSession获取多个URL(文档建议通常应为每个应用程序创建一个ClientSession) .
import aiohttp
import asyncio
async def fetch(session, url):
async with session.get(url) as response:
return await response.text()
async def main(url, session):
print(f"Starting '{url}'")
html = await fetch(session, url)
print(f"'{url}' done")
urls = (
"https://python.org",
"https://twitter.com",
"https://tumblr.com",
"https://example.com",
"https://github.com",
)
loop = asyncio.get_event_loop()
session = aiohttp.ClientSession()
loop.run_until_complete(asyncio.gather(
*(loop.create_task(main(url, session)) for url in urls)
))
# session.close() <- this doesn't make a difference
但是,在协程外创建ClientSession显然不是可行的方法:
➜ python 1_async.py
1_async.py:30: UserWarning: Creating a client session outside of coroutine is a very dangerous idea
session = aiohttp.ClientSession()
Creating a client session outside of coroutine
client_session:
Starting 'https://python.org'
Starting 'https://twitter.com'
Starting 'https://tumblr.com'
Starting 'https://example.com'
Starting 'https://github.com'
'https://twitter.com' done
'https://example.com' done
'https://github.com' done
'https://python.org' done
'https://tumblr.com' done
1_async.py:34: RuntimeWarning: coroutine 'ClientSession.close' was never awaited
session.close()
Unclosed client session
client_session:
Unclosed connector
connections: ['[(, 15024.110107067)]', '[(, 15024.147785039)]', '[(, 15024.252375415)]', '[(, 15024.292646968)]', '[(, 15024.342368087)]', '[(, 15024.466971983)]', '[(, 15024.602057745)]', '[(, 15024.837045568)]']
connector:
FWIW,在我尝试上述改变之前,这是 main
:
async def main(url):
async with aiohttp.ClientSession() as session:
print(f"Starting '{url}'")
html = await fetch(session, url)
print(f"'{url}' done")
这样做的正确方法是什么?我想过将一个url列表传递给main,但是无法以非顺序的方式使它工作 .
1 回答
Creating a client session outside of coroutine is a very dangerous idea
因为当你创建它时,它被绑定到当前循环 . 如果你之后更改了运行循环,它将挂起 . 但如果你仔细使用它,你可以忽略它 . Related doc .至于我,我只是忽略了这个警告 . 但它也很容易克服它:
此外,您不需要显式创建
Task
对象,只需执行此协同程序函数:最后,别忘了
close
是一个协程 . 您应该使用loop.run_until_complete(session.close())
来关闭session
.顺便说一句,如果你想创建一个类似async的循环,你可以参考my another answer .