apache-spark - 在 Windows 上安装 apache spark

您好,我正在尝试安装 apache spark,但出现此错误。我已经安装了兼容apache spark、phyton和scala的java版本。我会很感激你的帮助。

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
22/05/18 22:50:07 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/05/18 22:50:09 ERROR SparkContext: Error initializing SparkContext.
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
...more
Caused by: java.net.URISyntaxException: Illegal character in path at index 32: spark://DESKTOP-H3SRII2:63213/C:\classes
        at java.net.URI$Parser.fail(Unknown Source)
...more
Caused by: java.net.URISyntaxException: Illegal character in path at index 32: spark://DESKTOP-H3SRII2:63213/C:\classes
        at java.net.URI$Parser.fail(Unknown Source)
        at java.net.URI.<init>(Unknown Source)
        at org.apache.spark.repl.ExecutorClassLoader.<init>(ExecutorClassLoader.scala:57)
22/05/18 22:50:09 ERROR Utils: Uncaught exception in thread shutdown-hook-0
java.lang.ExceptionInInitializerError
        at org.apache.spark.executor.Executor.stop(Executor.scala:333)
...more
Caused by: java.lang.NullPointerException
        at org.apache.spark.shuffle.ShuffleBlockPusher$.<init>(ShuffleBlockPusher.scala:465)
        at org.apache.spark.shuffle.ShuffleBlockPusher$.<clinit>(ShuffleBlockPusher.scala)
22/05/18 22:50:09 WARN ShutdownHookManager: ShutdownHook '$anon$2' failed, java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
        at java.util.concurrent.FutureTask.report(Unknown Source)
        at java.util.concurrent.FutureTask.get(Unknown Source)
        at org.apache.hadoop.util.ShutdownHookManager.executeShutdown(ShutdownHookManager.java:124)
        at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:95)

...更多原因:org.apache.spark.executor.Executor.stop(Executor.scala:333) 中的 java.lang.ExceptionInInitializerError ...更多原因:org.apache.spark 中的 java.lang.NullPointerException。 shuffle.ShuffleBlockPusher$.(ShuffleBlockPusher.scala:465) 在 org.apache.spark.shuffle.ShuffleBlockPusher$.(ShuffleBlockPusher.scala)

回答1

我下载了较低版本的 spark 3.2.1,一切都开始工作了。我不知道这有什么解释,但显然这是解决方案。

相似文章

最新文章