首页 文章

使用自定义JsonExtractor的gz的U-SQL提取失败

提问于
浏览
1

我在提取过程中遇到异常,同时尝试使用带有gzip压缩文件的 Microsoft.Analytics.Samples.Formats 库中的自定义 JsonExtractor .

Microsoft.Cosmos.ScopeStudio.BusinessObjects.Debugger.ScopeDebugExceptionwasunhandledMessage: Anunhandledexceptionoftype'Microsoft.Cosmos.ScopeStudio.BusinessObjects.Debugger.ScopeDebugException'occurredinMicrosoft.Cosmos.ScopeStudio.BusinessObjects.Debugger.dllAdditionalinformation: {
    "diagnosticCode": 195887142,
    "severity": "Error",
    "component": "RUNTIME",
    "source": "User",
    "errorId": "E_RUNTIME_USER_UNHANDLED_EXCEPTION_FROM_USER_CODE",
    "message": "An unhandled exception from user code has been reported when invoking the method 'Extract' on the user type 'Microsoft.Analytics.Samples.Formats.Json.JsonExtractor'",
    "description": "Unhandled exception from user code: \"Unexpected character encountered while parsing number: j. Path '', line 1, position 33.\"\nThe details includes more information including any inner exceptions and the stack trace where the exception was raised.",
    "resolution": "Make sure the bug in the user code is fixed.",
    "helpLink": "",
    "details": "==== Caught exception Newtonsoft.Json.JsonReaderException
       at Newtonsoft.Json.JsonTextReader.ReadNumberIntoBuffer()
       at Newtonsoft.Json.JsonTextReader.ParseNumber(ReadType readType)
       at Newtonsoft.Json.JsonTextReader.ParseValue()
       at Microsoft.Analytics.Samples.Formats.Json.JsonExtractor.<Extract>d__2.MoveNext()
       at ScopeEngine.SqlIpExtractor<ScopeEngine::GZipInput,Extract_0_Data0>.GetNextRow(SqlIpExtractor<ScopeEngine::GZipInput23644,Extract_0_Data0>* , Extract_0_Data0* output) in c:23644projects23644data-insights23644data_insights_ingestion_service23644datainsights.internalbi.scripts23644bin23644debug23644746497ac301de51323644n-populateaccountcurrencycodes.usql.debug_6b364b066929ac9723644sqlmanaged.h:line 1873",
    "internalDiagnostics": ""
}

如果传递JSON文件而不进行gzipping,则提取成功 .

我也测试过gzipped csv文件,它们似乎工作得很好 . 在LocalRun期间遇到此问题 . 我还没有将这个特定的脚本提交到 Cloud 端 .

1 回答

  • 1

    sponturious,

    发生此问题时使用的Azure Data Lake Tools版本(最新版本为2.2.5000.0)?另外,您可以尝试“gzip -t your_gzip_data.gz”来验证gzip文件的完整性吗?

    gzip for windows可以从这里获得:http://gnuwin32.sourceforge.net/packages/gzip.htm

    谢谢,余冬

相关问题