首页 文章

使用google-api-client gem将大文件上传到Google Cloud 存储时执行过期错误

提问于
浏览
1

我正在尝试使用我的rails4应用中的google-api-client gem将大小为83.4 MB的大型csv文件上传到谷歌 Cloud 存储中的存储桶 . 我在谷歌 Cloud 存储中将文件上传到存储桶的代码如下:

def upload_file_resumable(local_file_path,mime_type,relative_bucket_path)
    media = Google::APIClient::UploadIO.new(local_file_path,mime_type)
    bucket_name = APP_CONFIG[:google_storage_bucket]
    file_name   = relative_bucket_path.split("/").join("/") + "/" + File.basename(local_file_path)
    resumable_result = client.execute(
      api_method: api.objects.insert,
      media: media,
      parameters: {
        uploadType: 'resumable',
        bucket: bucket_name,
        name: file_name,
      },
      body_object: {contentType: mime_type}
    )
    upload = resumable_result.resumable_upload
    if upload.resumable?
      client.execute(upload)
    end 
  end

这适用于小尺寸的文件,但对于大文件,我得到以下错误跟踪:

2016-08-22T12:39:14.515Z 3710 TID-ouzo56edo WARN: execution expired
2016-08-22T12:39:14.515Z 3710 TID-ouzo56edo WARN: /Users/raza/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/openssl/buffering.rb:326:in `syswrite'
/Users/raza/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/openssl/buffering.rb:326:in `do_write'
/Users/raza/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/openssl/buffering.rb:344:in `write'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient/session.rb:370:in `<<'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient/http.rb:500:in `dump'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient/http.rb:926:in `dump'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient/session.rb:615:in `block in query'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient/session.rb:613:in `query'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient/session.rb:164:in `query'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient.rb:1087:in `do_get_block'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient.rb:891:in `block in do_request'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient.rb:985:in `protect_keep_alive_disconnected'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient.rb:890:in `do_request'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/httpclient-2.4.0/lib/httpclient.rb:778:in `request'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/faraday-0.9.2/lib/faraday/adapter/httpclient.rb:36:in `call'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/faraday-0.9.2/lib/faraday/response.rb:8:in `call'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/google-api-client-0.8.6/lib/google/api_client/request.rb:163:in `send'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/google-api-client-0.8.6/lib/google/api_client/request.rb:174:in `send'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/google-api-client-0.8.6/lib/google/api_client.rb:648:in `block (2 levels) in execute!'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/retriable-1.4.1/lib/retriable/retry.rb:27:in `perform'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/retriable-1.4.1/lib/retriable.rb:15:in `retriable'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/google-api-client-0.8.6/lib/google/api_client.rb:645:in `block in execute!'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/retriable-1.4.1/lib/retriable/retry.rb:27:in `perform'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/retriable-1.4.1/lib/retriable.rb:15:in `retriable'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/google-api-client-0.8.6/lib/google/api_client.rb:636:in `execute!'
/Users/raza/.rvm/gems/ruby-2.2.2@views/gems/google-api-client-0.8.6/lib/google/api_client.rb:679:in `execute'
/Users/raza/workspace/views/app/services/google_cloud_storage_service.rb:15:in `upload_file_resumable'

跟踪表明该错误发生在此代码块中

resumable_result = client.execute(
      api_method: api.objects.insert,
      media: media,
      parameters: {
        uploadType: 'resumable',
        bucket: bucket_name,
        name: file_name,
      },
      body_object: {contentType: mime_type}
    )

我怎样才能解决这个问题 ?

谢谢 .

1 回答

  • 1

    也许上传需要很长时间,而您的请求处理程序会超时 . 尝试增加超时或截止日期设置 .

    要确认确实是一个缓慢的上传是原因,请使用大文件从命令行运行您的函数,看到它成功,并测量它需要多长时间 .

相关问题