在Python 3中从Web下载文件

6 回答

• 450

  如果要将网页的内容转换为变量，只需 read urllib.request.urlopen的响应：

  import urllib.request
  ...
  url = 'http://example.com/'
  response = urllib.request.urlopen(url)
  data = response.read()      # a `bytes` object
  text = data.decode('utf-8') # a `str`; this step can't be used if data is binary
  

  ---

  下载和保存文件的最简单方法是使用urllib.request.urlretrieve函数：

  import urllib.request
  ...
  # Download the file from `url` and save it locally under `file_name`:
  urllib.request.urlretrieve(url, file_name)
  

  import urllib.request
  ...
  # Download the file from `url`, save it in a temporary directory and get the
  # path to it (e.g. '/tmp/tmpb48zma.txt') in the `file_name` variable:
  file_name, headers = urllib.request.urlretrieve(url)
  

  但请记住 urlretrieve 被认为是legacy并且可能会被弃用（但不确定为什么） .

  因此，最正确的方法是使用urllib.request.urlopen函数返回表示HTTP响应的类文件对象，并使用shutil.copyfileobj将其复制到实际文件中 .

  import urllib.request
  import shutil
  ...
  # Download the file from `url` and save it locally under `file_name`:
  with urllib.request.urlopen(url) as response, open(file_name, 'wb') as out_file:
      shutil.copyfileobj(response, out_file)
  

  如果这看起来太复杂，您可能希望更简单并将整个下载存储在 bytes 对象中，然后将其写入文件 . 但这适用于小文件 .

  import urllib.request
  ...
  # Download the file from `url` and save it locally under `file_name`:
  with urllib.request.urlopen(url) as response, open(file_name, 'wb') as out_file:
      data = response.read() # a `bytes` object
      out_file.write(data)
  

  ---

  可以动态提取压缩数据，但这样的操作可能需要HTTP服务器支持对文件的随机访问 .

  import urllib.request
  import gzip
  ...
  # Read the first 64 bytes of the file inside the .gz archive located at `url`
  url = 'http://example.com/something.gz'
  with urllib.request.urlopen(url) as response:
      with gzip.GzipFile(fileobj=response) as uncompressed:
          file_header = uncompressed.read(64) # a `bytes` object
          # Or do anything shown above using `uncompressed` instead of `response`.
  

  回复于 2024-04-26T04:13:44+08:00
• 3

  每当我想要与HTTP请求相关的东西时，我都会使用 requests 包，因为它的API非常容易入手：

  首先，安装 requests

  $ pip install requests
  

  那么代码：

  from requests import get  # to make GET request
  
  
  def download(url, file_name):
      # open in binary mode
      with open(file_name, "wb") as file:
          # get request
          response = get(url)
          # write to file
          file.write(response.content)
  

  回复于 2024-04-26T04:13:44+08:00
• 85

  我希望我理解正确的问题，即：当URL存储在字符串类型中时，如何从服务器下载文件？

  我使用以下代码下载文件并在本地保存：

  import requests
  
  url = 'https://www.python.org/static/img/python-logo.png'
  fileName = 'D:\Python\dwnldPythonLogo.png'
  req = requests.get(url)
  file = open(fileName, 'wb')
  for chunk in req.iter_content(100000):
      file.write(chunk)
  file.close()
  

  回复于 2024-04-26T04:13:44+08:00
• 13

  在这里我们可以在Python3中使用urllib的Legacy接口：

  以下函数和类从Python 2模块urllib（而不是urllib2）移植 . 他们可能会在将来的某个时候被弃用 .

  示例 (2 lines code) ：

  import urllib.request
  
  url = 'https://www.python.org/static/img/python-logo.png'
  urllib.request.urlretrieve(url, "logo.png")
  

  回复于 2024-04-26T04:13:44+08:00
• -3

  您可以使用 wget 这是一个流行的下载shell工具 . https://pypi.python.org/pypi/wget这将是最简单的方法，因为它不需要打开目标文件 . 这是一个例子 .

  import wget
  url = 'https://i1.wp.com/python3.codes/wp-content/uploads/2015/06/Python3-powered.png?fit=650%2C350'  
  wget.download(url, '/Users/scott/Downloads/cat4.jpg')
  

  回复于 2024-04-26T04:13:44+08:00
• 4

  from urllib import request
  
  def get(url):
      with request.urlopen(url) as r:
          return r.read()
  
  
  def download(url, file=None):
      if not file:
          file = url.split('/')[-1]
      with open(file, 'wb') as f:
          f.write(get(url))
  

  回复于 2024-04-26T04:13:44+08:00