Spark Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory

环境:spark 1.1

spark-shell启用的时候一直出现这个提示:

Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory

查了一下午才明白,需要在master结点上做如下修改:

vi /etc/spark/conf/spark-defaults.conf

添加一项

spark.driver.memory=10g

注:spark.driver.memory表示master的memory大小设置, 而spark.executor.memory代表worker结点内存大小。

作者: hqiang1984

量化自我,极简主义