Server infrastructure configuration and infrastructure construction

https://www.bilibili.com/video/BV1NCQtBEEw5

Configure IP for 3 servers respectively

View Gateway Methods

sudo vim /etc/netplan/50-cloud-init.yaml

Just change the ip in the addresses and the gateway behind the via (the /24 behind it does not need to be changed. How to check the gateway See above, the dhcp: true in the original text needs to be deleted!)

The ip address and gateway address in the figure are configured according to actual conditions. If ens160 in the figure is inconsistent with the original ones, the original ens number will be retained! The following is an example configuration:

network:
  version: 2
  ethernets:
    ens33:
      addresses:
      - "10.1.100.20/24"
      nameservers:
        addresses:
        - 8.8.8.8
        - 114.114.114.114
        search: []
      routes:
      - to: "default"
        via: "10.1.100.254"

Then reload the configuration

sudo netplan apply

Add users and grant permissions to each of the three servers

add users

sudo adduser hadooper

Then enter the same password twice (remember this password!) Just follow the instructions and continue executing

granted this permission

sudo usermod -aG sudo hadooper

sudo vim /etc/sudoers

The configuration is shown in the following figure

Change the hosts and host names of the three servers respectively

Edit hosts configuration

sudo vim /etc/hosts

Add the following configuration to the file and delete 127.0.1.1 ubuntubase in the second line

The IP address needs to be changed to the corresponding server IP address

That is, the IP address of the node + the node name

10.1.100.20 hadoop01
10.1.100.21 hadoop02
10.1.100.22 hadoop03

Change the host names of each of the three servers

hadoop01:

sudo hostnamectl set-hostname hadoop01

hadoop02:

sudo hostnamectl set-hostname hadoop02

hadoop03:

sudo hostnamectl set-hostname hadoop03

After this step is completed, restart all three servers

configuration test

An example of instructions to ping two other servers on three servers is as follows In Hadoop01:

ping hadoop02

ping hadoop03

The same applies to other servers, so I won’t go into it too much here

Starting from this step, the default execution user is hadooper. If the user using is not hadooper, use su hadooper and enter the hadooper user’s password to switch to hadooper user

source replacement

Backup the original configuration

sudo cp /etc/apt/sources.list.d/ubuntu.sources /etc/apt/sources.list.d/ubuntu.sources.bak

sudo vim /etc/apt/sources.list.d/ubuntu.sources

Then enter ggdG (delete all content, strictly follow case, this is the basic operation of vim), enter the following configuration

Types: deb

URIs: https://mirrors.aliyun.com/ubuntu/

Suites: noble noble-updates noble-backports

Components: main restricted universe multiverse

Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

 

Types: deb

URIs: https://mirrors.aliyun.com/ubuntu/

Suites: noble-security

Components: main restricted universe multiverse

Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

Update upgrade software package

sudo apt update

sudo apt upgrade -y

Install and configure SSH services

Install SSH services on each of the three servers

sudo apt install openssh-server

Set up self-booting on each of the three servers

sudo systemctl enable ssh

Generate keys on each of the three servers (the place where this command needs to be entered is empty by default, that is, press Enter)

ssh-keygen -t rsa

Copy the public key to each server on each of the three servers (i.e. each server needs to execute the following three instructions)

ssh-copy-id hadoop01

ssh-copy-id hadoop02

ssh-copy-id hadoop03

Create a storage directory

create a directory

Create 3 directories on 3 servers respectively. data is used to store data files, servers is the software installation directory, and software is used to place software packages

sudo mkdir -p /export/data

sudo mkdir -p /export/servers

sudo mkdir -p /export/software

Empowering folders

Empowering folders on 3 servers

sudo chown -R hadooper /export/

sudo chmod -R 755 /export/

Replacing Java

Note: All three servers need to be replaced, so the java configuration of server 01 should be carried out first, and then distributed

Uninstall existing java

Retrieves which versions of java are installed

dpkg -l | grep -i jdk

Uninstall installed java

sudo apt purge java版本(此处版本为上一步检索出的java版本)

#copy-jdk-configs可不进行删除

install Java

Switch to software catalog

cd /export/software

Download jdk file

wget -P /export/software/ https://repo.huaweicloud.com/java/jdk/8u181-b13/jdk-8u181-linux-x64.tar.gz

decompression

tar -zxvf /export/software/jdk-8u181-linux-x64.tar.gz -C /export/servers/

cd /export/servers/

mv /export/servers/jdk1.8.0_181 /export/servers/jdk

Configure environment variables

Edit environment variable configuration file

sudo vim /etc/profile

Add the following configuration to the end of the file

export JAVA_HOME=/export/servers/jdk
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

Reload environment variables

source /etc/profile

Verify that java was successfully installed

java -version

Distribute java configuration

Distribute configuration to hadoop02

scp /etc/profile hadoop02:/tmp/profile

ssh hadoop02 "sudo cp /tmp/profile /etc/profile && sudo rm /tmp/profile"

scp -r /export/servers/jdk hadoop02:/tmp/

ssh hadoop02 "sudo cp -r /tmp/jdk/ /export/servers/ && sudo rm -rf /tmp/jdk"

Distribute configuration to hadoop03

scp /etc/profile hadoop03:/tmp/profile

ssh hadoop03 "sudo cp /tmp/profile /etc/profile && sudo rm /tmp/profile"

scp -r /export/servers/jdk hadoop03:/tmp/

ssh hadoop03 "sudo cp -r /tmp/jdk/ /export/servers/ && sudo rm -rf /tmp/jdk"

Confirm that distribution is correct

After distribution, enter the following command into server 01 to determine whether the distribution was successful

ssh hadoop02 "bash -c 'source /etc/profile && java -version'"

ssh hadoop03 "bash -c 'source /etc/profile && java -version'"

installation tool

Installation of net-tools

Install net-tools on each of the three servers

sudo apt install net-tools

hadoop deployment basic environment