火花提交中的NoSuchMethodError

我使用spark-submit将我的jar依赖项提交给spark . 在我的jar的main方法中，我想创建 HttpAsyncCliens 实例并执行一些请求（apache http async client library）：

val httpClient = HttpAsyncClients.custom.setMaxConnTotal(10).setMaxConnPerRoute(10).build
httpClient.start()
httpClient.execute(new HttpGet("https://google.com"), new FutureCallback[HttpResponse] {
    /* callbacks */
})

它抛出异常：

线程“pool-1-thread-1”中的异常java.lang.NoSuchMethodError：org.apache.http.util.Asserts.check（ZLjava / lang / String; Ljava / lang / Object;）V org.apache . 位于org.apache.http.impl的org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute（PoolingNHttpClientConnectionManager.java:191）中的http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute（AbstractMultiworkerIOReactor.java:315） . java.lang.Thread.run中的nio.client.CloseableHttpAsyncClientBase $ 1.run（CloseableHttpAsyncClientBase.java:64）（Thread.java:745）线程“main”中的异常java.lang.NoSuchMethodError：org.apache.http.util . Asserts.check（ZLjava / lang / String; Ljava / lang / Object;）V org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning（CloseableHttpAsyncClientBase.java:90）at org.apache.http.impl . 位于org.apache.http.imp的org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute（CloseableHttpAsyncClient.java:74）中的nio.client.InternalHttpAsyncClient.execute（InternalHttpAsyncClient.java:123） l.nio.client.CloseableHttpAsyncClient.execute（CloseableHttpAsyncClient.java:107）atg.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute（CloseableHttpAsyncClient.java:91）at spark.Application $ .main（Application.scala ：37）在sun.reflect.DelegatingMethodAccessorImpl.invoke的sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:62）的sun.reflect.NativeMethodAccessorImpl.invoke0（Native Method）的spark.Application.main（Application.scala）处 . DelegatingMethodAccessorImpl.java:43）在org.apache.spark.deploy.SparkSubmit $ .org $ apache $ spark $ deploy $ java.lang.ref . （ . scala：731）在org.apache的org.apache.spark.deploy.SparkSubmit $ .doRunMain $ 1（SparkSubmit.scala：181）org.apache.spark.deploy.SparkSubmit $ .submit（SparkSubmit.scala：206） . spark.deploy.SparkSubmit $ .main（SparkSubmit.scala：121）at org.apache.spark.deploy.SparkSubmit.main（SparkSubmit.scala）

似乎我的jar中没有http-core依赖，但我可以在http客户端创建和请求之前或之后在代码中调用此方法（ org.apache.http.util.Asserts.check(ZLjava/lang/String;Ljava/lang/Object;)V ）：

org.apache.http.util.Asserts.check(true, "test", "test2") // it produces no exception

val httpClient = HttpAsyncClients.custom.setMaxConnTotal(10).setMaxConnPerRoute(10).build
httpClient.start()
httpClient.execute(new HttpGet("https://google.com"), new FutureCallback[HttpResponse] {
    /* callbacks */
}) // it produces exception

如果我可以在代码中从同一个类路径调用此方法，为什么我得到NoSuchMethodError？

Apache httpasyncclient v4.1