<ruby id="bdb3f"></ruby>

    <p id="bdb3f"><cite id="bdb3f"></cite></p>

      <p id="bdb3f"><cite id="bdb3f"><th id="bdb3f"></th></cite></p><p id="bdb3f"></p>
        <p id="bdb3f"><cite id="bdb3f"></cite></p>

          <pre id="bdb3f"></pre>
          <pre id="bdb3f"><del id="bdb3f"><thead id="bdb3f"></thead></del></pre>

          <ruby id="bdb3f"><mark id="bdb3f"></mark></ruby><ruby id="bdb3f"></ruby>
          <pre id="bdb3f"><pre id="bdb3f"><mark id="bdb3f"></mark></pre></pre><output id="bdb3f"></output><p id="bdb3f"></p><p id="bdb3f"></p>

          <pre id="bdb3f"><del id="bdb3f"><progress id="bdb3f"></progress></del></pre>

                <ruby id="bdb3f"></ruby>

                ??碼云GVP開源項目 12k star Uniapp+ElementUI 功能強大 支持多語言、二開方便! 廣告
                **1. 首先檢查是否安裝了 jdk8。** **2. 下載安裝包并解壓** 直接訪問:https://archive.apache.org/dist/spark/spark-2.4.4/spark-2.4.4-bin-hadoop2.6.tgz a. 如需歷史版本下載請訪問:https://archive.apache.org/dist/spark/。 b. Spark 不是必須安裝 Hadoop,但如果已經有 Hadoop 集群或 HDFS,建議下載對應版本的 Spark。 下載完成后解壓到/opt/install 目錄。 ```shell tar -zvxf spark-2.4.4-bin-hadoop2.6.tgz -C /opt/install -- 創建軟連接 ln -s spark-2.4.4-bin-hadoop2.6 spark ``` **3. 配置環境變量** ```sql vim /etc/profile export SPARK_HOME=/opt/install/spark export PATH=$SPARK_HOME/bin:$SPARK_HOME/sbin:$PATH ``` 使環境變量生效 ``` source /etc/profile ``` **4. 配置文件** ```sql cd /opt/install/spark/conf cp spark-env.sh.template spark-env.sh vim spark-env.sh export JAVA_HOME=/opt/install/jdk export HADOOP_HOME=/opt/install/hadoop export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop #指定 master 的主機 export SPARK_MASTER_HOST=hadoop101 #指定 master 的端口 export SPARK_MASTER_PORT=7077 ``` 修改配置文件 {spark_home}/conf/slaves(單臺可以不配置,也可以按下面方式配置) ```sql -- 先把 slaves.template 重命名為 slaves mv slaves.template slaves vim slaves hadoop101 ``` **5. 啟動 Spark 集群,在 spark 目錄下,輸入:** ```sql sbin/start-all.sh ``` 查看是否有 Master 與 Worker 進程 ```sql [root@hadoop101 spark]# jps 3694 Worker 3599 Master ``` **6. 啟動 spark-shell 測試 scala 交互式環境** ```sql [root@hadoop101 spark]# spark-shell --master spark://hadoop101:7077 或者 [root@hadoop101 sbin]# spark-shell --master local[4] 21/01/04 10:05:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). Spark context Web UI available at http://hadoop101:4040 Spark context available as 'sc' (master = local[4], app id = local-1609725951951). Spark session available as 'spark'. Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 2.4.4 /_/ Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_171) Type in expressions to have them evaluated. Type :help for more information. scala> ``` 可以通過瀏覽器訪問:http://hadoop101:8080/ ![](https://img.kancloud.cn/e8/6b/e86b2faca084f98c210325e1df6c5416_1048x448.png) **7. 測試 Spark on YARN** ```sql -- 需提前啟動 yarn 進程。啟動yarn [root@hadoop101 sbin]# ./start-yarn.sh -- Spark on Yarn [root@hadoop101 sbin]# spark-shell --master yarn 21/01/04 14:23:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 21/01/04 14:23:58 WARN util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041. 21/01/04 14:23:59 WARN yarn.Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME. Spark context Web UI available at http://hadoop101:4041 Spark context available as 'sc' (master = yarn, app id = application_1609741310740_0001). Spark session available as 'spark'. Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 2.4.4 /_/ Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_171) Type in expressions to have them evaluated. Type :help for more information. scala> ```
                  <ruby id="bdb3f"></ruby>

                  <p id="bdb3f"><cite id="bdb3f"></cite></p>

                    <p id="bdb3f"><cite id="bdb3f"><th id="bdb3f"></th></cite></p><p id="bdb3f"></p>
                      <p id="bdb3f"><cite id="bdb3f"></cite></p>

                        <pre id="bdb3f"></pre>
                        <pre id="bdb3f"><del id="bdb3f"><thead id="bdb3f"></thead></del></pre>

                        <ruby id="bdb3f"><mark id="bdb3f"></mark></ruby><ruby id="bdb3f"></ruby>
                        <pre id="bdb3f"><pre id="bdb3f"><mark id="bdb3f"></mark></pre></pre><output id="bdb3f"></output><p id="bdb3f"></p><p id="bdb3f"></p>

                        <pre id="bdb3f"><del id="bdb3f"><progress id="bdb3f"></progress></del></pre>

                              <ruby id="bdb3f"></ruby>

                              哎呀哎呀视频在线观看