首页 文章

Azure DocumentDB偶尔会抛出SocketException / GoneException

提问于
浏览 次
6

UPDATE (2/8/17) ：

见下面的答案 .

UPDATE (2/7/17) ：

我发现重启可以让我从Visual Studio 2015成功运行Web应用程序，并多次点击 endpoints . 但是，当我停止应用程序并重新启动它时，它很可能会失败 . 然后它将重复失败，直到我重新启动计算机 . 重启VS'15还不够好 .

一旦它开始失败，从VS代码运行应用程序或从命令行运行dotnet.exe表现出相同的行为 .

ORIGINAL POST ：

我们有一个微服务系统设置，可以从几个API和Azure Functions调用DocumentDB集合 . 它从API端使用 SocketException （ GoneException 嵌套）间歇性地失败 . 据我们所知，只要它偶尔起作用，调用它的代码大多是正确的 . Azure功能可以正常工作 .

编辑澄清：通过"intermittently,"我的意思是它可能每天短暂工作一两次，然后在没有通话的剩余时间内进入失败状态 . 这不像每100个左右的呼叫失败一样 . 这更像是1或2次成功呼叫后的不间断故障 .

通过编写一个简单的控制台应用程序来从DocumentDB中读取并将结果打印到Debug输出，我能够重新创建相同的异常 . 这运行一两次没有任何问题，然后每次都开始抛出以下异常 . 它会在有几个小时之前执行此操作，然后再拨打几个电话，然后重新投入 .

虽然下面的测试器是原始的，但主API充分利用了vNext项目结构 . 它使用单个DocumentClient进行连接（通过本机DI注入），它几乎完全异步从控制器一直到调用db的服务层 . 我们使用一个单独的库来管理对DocumentDB的访问（如果它们不存在则创建集合，添加扩展方法，简单的CRUD操作等），但是如下所示直接调用会产生相同的结果 .

我注意到的一件事是它对于DocumentDB客户端的核心版本（ "Microsoft.Azure.DocumentDB.Core": "1.0.0" ）而言比它对net46版本更成功 . 由于其他库，我们的API需要4.6 .

我可以在多台机器，多个网络，多种连接类型上重新创建它 .

Question: Why are we getting this exception and how do we fix it?

Azure信息：

  • 美国东部2

  • 1000 RU / s

  • 标准层

  • 会话一致性

  • 懒惰索引策略：

{“indexingMode”：“lazy”，“automatic”：true，“includedPaths”：[{“path”：“/ *”，“indices”：[{“kind”：“Range”，“dataType”：“Number “，”“precision”： - 1}，{“kind”：“Hash”，“dataType”：“String”，“precision”：3}]}]，“excludedPaths”：[]}

Test class

using System;
using System.Diagnostics;
using System.Threading.Tasks;
using Microsoft.Azure.Documents.Client;
using Microsoft.Azure.Documents.Linq;
using Newtonsoft.Json;

namespace TestConnection
{
    public class Program
    {
        public static void Main(string[] args)
        {
            try
            {
                using (var client = new DocumentClient(
                    new Uri("https://<our-docdb-name>.documents.azure.com:443/"),
                    "our access key",
                    new ConnectionPolicy
                    {
                        ConnectionMode = ConnectionMode.Direct,
                        ConnectionProtocol = Protocol.Tcp
                    }))
                {
                    var query = client.CreateDocumentQuery(UriFactory.CreateCollectionUri("Imports", "User"),
                        "SELECT * FROM c where c.importId = \"816d8e92-bd08-4705-9989-09a0ece5892a\"");
                    var docQuery = query.AsDocumentQuery();
                    GetResults(docQuery).Wait();
                    Debug.WriteLine("done");
                }
            }
            catch (Exception e)
            {
                Debug.WriteLine(e);
            }
        }

        private static async Task GetResults(IDocumentQuery<dynamic> docQuery)
        {
            Debug.WriteLine("getting");
            var results = await docQuery.ExecuteNextAsync();
            Debug.WriteLine(JsonConvert.SerializeObject(results));
        }
    }
}

project.json

{
    "version": "1.0.0-*",
    "buildOptions": {
        "debugType": "portable",
        "emitEntryPoint": true
    },
    "dependencies": {
        "Microsoft.Azure.DocumentDB": "1.11.3"
    },
    "frameworks": {
        "net46": {}
    }
}

Exception

'TestConnection.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\projects\TestConnection\src\TestConnection\bin\Debug\net46\win7-x64\TestConnection.exe'. Symbols loaded.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\projects\TestConnection\src\TestConnection\bin\Debug\net46\win7-x64\Microsoft.Azure.Documents.Client.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Net.Http\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Net.Http.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\projects\TestConnection\src\TestConnection\bin\Debug\net46\win7-x64\Newtonsoft.Json.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
DocDBTrace Information: 0 : DocumentClient with id 1 initialized at endpoint: https://<our-docdb-name>.documents.azure.com/ with ConnectionMode: Direct, connection Protocol: Tcp, and consistency level: null
getting
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.CSharp\v4.0_4.0.0.0__b03f5f7f11d50a3a\Microsoft.CSharp.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Numerics\v4.0_4.0.0.0__b77a5c561934e089\System.Numerics.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Runtime.Serialization\v4.0_4.0.0.0__b77a5c561934e089\System.Runtime.Serialization.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml.Linq\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.Linq.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'TestConnection.exe' (CLR v4.0.30319: TestConnection.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
DocDBTrace Information: 0 : Set WriteEndpoint https://<our-docdb-name>-eastus2.documents.azure.com/ ReadEndpoint https://<our-docdb-name>-eastus2.documents.azure.com/
DocDBTrace Information: 0 : Mapped resourceName dbs/Imports/colls/User to resourceId u81pAO1OFwA=. '00000000-0000-0000-0000-000000000000'
DocDBTrace Information: 0 : Mapped resourceName dbs/Imports/colls/User to resourceId u81pAO1OFwA=. '00000000-0000-0000-0000-000000000000'
The thread 0x3888 has exited with code 0 (0x0).
The thread 0x2c20 has exited with code 0 (0x0).
The thread 0x39fc has exited with code 0 (0x0).
The thread 0x3610 has exited with code 0 (0x0).
The thread 0x3824 has exited with code 0 (0x0).
The thread 0x33d8 has exited with code 0 (0x0).
The thread 0x38d0 has exited with code 0 (0x0).
DocDBTrace Information: 0 : GetOpenConnection failed: RID: dbs/Imports/colls/User, ResourceType Document, Op: (operationType: Query, resourceType: Document), Address: rntbd://bn6prdddc05-docdb-1.documents.azure.com:18817/apps/d54f0cf3-23d7-4050-9810-99d319d441a8/services/d77a45f3-5611-4c1d-a08e-0f3ef60a31d9/partitions/wkjhgkwj-c85a-4b08-b026-6bc8010b1bb5/replicas/131287308072454308s/, Exception: Microsoft.Azure.Documents.GoneException: Message: The requested resource is no longer available at the server.
ActivityId: d71bc76d-1411-414d-a844-9f76a46ebcfd, Request URI: rntbd://bn6prdddc05-docdb-1.documents.azure.com:18817/apps/d54f0cf3-23d7-4050-9810-99d319d441a8/services/d77a45f3-5611-4c1d-a08e-0f3ef60a31d9/partitions/wkjhgkwj-c85a-4b08-b026-6bc8010b1bb5/replicas/131287308072454308s/ ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 13.68.28.135:18817
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.TcpClient.EndConnect(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdConnection.<OpenSocket>d__1c.MoveNext()
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Documents.RntbdConnection.<OpenSocket>d__1c.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdConnection.<Open>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdConnectionDispenser.<OpenNewConnection>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.ConnectionPool.<GetOpenConnection>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.ConnectionPoolManager.<GetOpenConnection>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdTransportClient.<InvokeStoreAsync>d__0.MoveNext()
DocDBTrace Information: 0 : Exception Microsoft.Azure.Documents.GoneException: Message: The requested resource is no longer available at the server.
ActivityId: d71bc76d-1411-414d-a844-9f76a46ebcfd, Request URI: rntbd://bn6prdddc05-docdb-1.documents.azure.com:18817/apps/d54f0cf3-23d7-4050-9810-99d319d441a8/services/d77a45f3-5611-4c1d-a08e-0f3ef60a31d9/partitions/wkjhgkwj-c85a-4b08-b026-6bc8010b1bb5/replicas/131287308072454308s/ ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 13.68.28.135:18817
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.TcpClient.EndConnect(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdConnection.<OpenSocket>d__1c.MoveNext()
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Documents.RntbdConnection.<OpenSocket>d__1c.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdConnection.<Open>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdConnectionDispenser.<OpenNewConnection>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.ConnectionPool.<GetOpenConnection>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.ConnectionPoolManager.<GetOpenConnection>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdTransportClient.<InvokeStoreAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.StoreReader.<CompleteActivity>d__1f.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.StoreReader.<ReadMultipleReplicasInternalAsync>d__a.MoveNext() is thrown while doing readMany
DocDBTrace Warning: 0 : Received gone exception, will retry, Microsoft.Azure.Documents.GoneException: Message: The requested resource is no longer available at the server.
ActivityId: d71bc76d-1411-414d-a844-9f76a46ebcfd, Request URI: rntbd://bn6prdddc05-docdb-1.documents.azure.com:18817/apps/d54f0cf3-23d7-4050-9810-99d319d441a8/services/d77a45f3-5611-4c1d-a08e-0f3ef60a31d9/partitions/wkjhgkwj-c85a-4b08-b026-6bc8010b1bb5/replicas/131287308072454308s/ ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 13.68.28.135:18817
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.TcpClient.EndConnect(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.RntbdConnection.<OpenSocket>d__1c.MoveNext()
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Documents.StoreReader.<ReadMultipleReplicasInternalAsync>d__a.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.StoreReader.<ReadMultipleReplicaAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.ConsistencyReader.<ReadSessionAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.ReplicatedResourceClient.<InvokeAsync>d__b.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.ReplicatedResourceClient.<>c__DisplayClass1.<<InvokeAsync>b__0>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.<>c__DisplayClassf`1.<<ExecuteAsync>b__d>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.<ExecuteRetry>d__1b.MoveNext()
The thread 0x36e4 has exited with code 0 (0x0).
The thread 0x3adc has exited with code 0 (0x0).
The thread 0x3a68 has exited with code 0 (0x0).
The thread 0x18b0 has exited with code 0 (0x0).
...
/* The above repeats about 3 more times.*/
...
The program '[14648] TestConnection.exe' has exited with code 0 (0x0).
c# azure .net-core azure-cosmosdb

1 回答

  • 3

    原来是我们的企业 Bitdefender Endpoint Security 造成了这个问题 . 我们最初并没有怀疑它，因为它清楚地记录了 dotnet.exe 被列入白名单并允许在我们运行应用程序时在适当的端口上进行通信 . 我们卸载了它，问题就消失了 . 我们_1867821是一个临时解决方案，原始问题与代码无关 . 希望这有助于某人 .

    回复于

相关问题