Cloud-init

Figure 998. Introduction and reference

Figure 999. In a nutshell

Distribution image containing pre-installed Cloud Init
Script configurable installation options

Figure 1000. Configuration options

Individual CRUD file operations
Supplying ssh user and host keys.
Adding users
...

Installing packages
System Upgrade + reboot
Arbitrary command execution

Figure 1001. Terraform interface to Cloud Init

resource "hcloud_server" "web" {
  name         = var.server_name
       ...
  user_data = file("userData.yml")
}

Figure 1002. »hello, world ...« userData.yml file

#cloud-config
packages:
  - nginx
runcmd:
  - systemctl enable nginx
  - rm /var/www/html/*
  - >
    echo "I'm Nginx @ $(dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com) 
    created $(date -u)" >> /var/www/html/index.html

Figure 1003. Using template files

resource "local_file" "user_data" {
  content         = templatefile("tpl/userData.yml", {
  loginUser              = "devops"
                    })
  filename        = "gen/userData.yml"
}

resource "hcloud_server" "helloServer" {
  ...
  # Will become gen/userData.yml
  user_data    = local_file.user_data.content
}

...
users:
  - name: ${loginUser}
    ...

...
users:
  - name: devops
     ...

Figure 1004. Validation

# cloud-init schema --system --annotate

...
apt:		# E1
-   debconf_selections: openssh-server  openssh-server/password-authentication  boolean
        false openssh-server  openssh-server/permit-root-login        boolean false
...
# Errors: -------------
# E1: [{'debconf_selections': 'openssh-server  ....boolean false'}] is not of type 'object'

# cloud-init schema --config-file /var/lib/cloud/instance/user-data.txt
```
Valid cloud-config: /var/lib/cloud/instance/user-data.txt
```

Figure 1005. Yaml syntax quirk 1: Missing indentation

resource "tls_private_key" "host" {
  algorithm   = "ED25519"
}
...
resource "local_file" "user_data" {
  content  = templatefile("tpl/userData.yml", {
     host_ed25519_private =
       tls_private_key.host.private_key_openssh
     host_ed25519_public  = ...
                })
  filename = "gen/user_data.yml"
}

...
ssh_keys:
  ed25519_private: |
    -----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUA...
-----END OPENSSH PRIVATE KEY-----

# E1: File gen/user_data.yml is not valid YAML.
  in "<unicode string>", line 19, column 1:
    b3BlbnNzaC1rZXktdjEAAAAABG5vbmUA ...

Figure 1006. Yaml missing indentation resolution

main.tf gen/usr_data.yml

`main.tf`	`gen/usr_data.yml`
resource "local_file" "user_data" { content = templatefile("tpl/userData.yml", { host_ed25519_private = indent(4, tls_private_key.host.private_key_openssh) ...}) filename = "gen/user_data.yml" }	`ssh_keys: ed25519_private: \| ␣␣␣␣-----BEGIN OPENSSH PRIVATE KEY----- ␣␣␣␣b3BlbnNzaC1 ... ␣␣␣␣-----END OPENSSH PRIVATE KEY-----`

resource "local_file" "user_data" {
  content  = templatefile("tpl/userData.yml", {
     host_ed25519_private = 
       indent(4, tls_private_key.host.private_key_openssh)
     ...})
  filename = "gen/user_data.yml"
}

ssh_keys:
  ed25519_private: |
␣␣␣␣-----BEGIN OPENSSH PRIVATE KEY-----
␣␣␣␣b3BlbnNzaC1 ...
␣␣␣␣-----END OPENSSH PRIVATE KEY-----

Figure 1007. Watch out for your enemies!

root@hello:~# journalctl -f
May 06 04:41:20 hello cloud-init[898]: Cloud-init v. 22.4.2 finished at Mon, 06 May 2024 04:41:20 +0000. Datasource DataSourceHetzner.  Up 11.78 seconds
   ...
May 06 04:46:16 hello sshd[927]: Invalid user abc from 43.163.218.130 port 33408
May 06 04:46:17 hello sshd[927]: Received disconnect from 43.163.218.130 port 33408:11: Bye Bye [preauth]
May 06 04:46:17 hello sshd[927]: Disconnected from invalid user abc 43.163.218.130 port 33408 [preauth]
   ...
May 06 04:50:54 hello sshd[930]: fatal: Timeout before authentication for 27.128.243.225 port 59866
   ...
May 06 04:52:45 hello sshd[933]: Invalid user cos from 43.163.218.130 port 59776
   ...
May 06 04:53:04 hello sshd[935]: Invalid user admin from 194.169.175.35 port 51128
May 06 04:53:49 hello sshd[937]: User root from 43.163.218.130 not allowed because not listed in AllowUsers
May 06 04:53:49 hello sshd[937]: Disconnected from invalid user root 43.163.218.130 port 50592 [preauth]

exercise No. 7

Working on Cloud-init

We continue our exercise series Incrementally creating a base system by adding a Cloud-init configuration:

Follow Figure 1001, “Terraform interface to Cloud Init ” and Figure 1002, “»hello, world ...« userData.yml file ” to create simple web server.

Tip

You will have to extend your current firewall configuration allowing inbound traffic to port 80.

On success pointing your web browser of choice to http://<your server's IP> should result in something similar to:
```
I'm Nginx @ "95.217.154.104" created Sun May 5 06:58:37 PM UTC 2024
```

With respect to Figure 1007, “Watch out for your enemies! ” inspect the output of journalctl -f on your own server for a while. Then modify your current sshd configuration:

Read the sshd_config(5) - Linux man page
Disallow ssh password based logins
Disallow root login altogether
Create a user devops eligible for both ssh login and sudo execution.

Tip

On success the following sequence should be possible:

$ ssh -v devops@95.217.154.104
...
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey ❶
debug1: Next authentication method: publickey
...
Last login: Sun May  5 19:21:12 2024 from 217.245.243.187
devops@hello:~$ sudo su - ❷
root@hello:~# hostname
hello

❶	`password` is not among the list of allowed authentication methods.
❷	User `devops` may execute sudo commands by virtue of his membership in group `sudo`.

On contrary ssh root login must be prohibited:

$ ssh root@95.217.154.104
root@95.217.154.104: Permission denied (publickey).

Read the cloud-init documentation and/or related tutorials to:
- Currently your (most likely outdated) cloud provider supplied distribution does not get upgraded on installation time:
```
$ ./bin/ssh 
...
devops@hello:~$ sudo su -
root@hello:~# apt update
Hit:1 http://security.debian.org/debian-security bookworm-security InRelease
Hit:2 http://deb.debian.org/debian bookworm InRelease                                              
...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
6 packages can be upgraded. Run 'apt list --upgradable' to see them.

# apt list --upgradable
Listing... Done
less/stable-security,stable-security 590-2.1~deb12u2 amd64 [upgradable from: 590-2]
libc-bin/stable-security,stable-security 2.36-9+deb12u7 amd64 [upgradable from: 2.36-9+deb12u6]
libc-l10n/stable-security,stable-security 2.36-9+deb12u7 all [upgradable from: 2.36-9+deb12u6]
libc6/stable-security,stable-security 2.36-9+deb12u7 amd64 [upgradable from: 2.36-9+deb12u6]
locales-all/stable-security,stable-security 2.36-9+deb12u7 amd64 [upgradable from: 2.36-9+deb12u6]
locales/stable-security,stable-security 2.36-9+deb12u7 all [upgradable from: 2.36-9+deb12u6]
```
  Modify your Cloud-init configuration to upgrade your distribution at server creation time. If so required your system should also reboot.
- Install and configure fail2ban limiting ssh failed connection attempts.
  Tip
  Develop your desired configuration manually on an already created server. Then automate the process.
  
  Read the answer On Debian 12 there are a couple of things you have to do to make it work.
- Install the plocate file indexer package and initialize it.
On success all packages should be up to date:
```
$ ./bin/ssh 
...
devops@hello:~$ sudo su -
root@hello:~# apt update
Hit:1 http://security.debian.org/debian-security bookworm-security InRelease
Hit:2 http://deb.debian.org/debian bookworm InRelease                                              
...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
```
Failed login attempts should be banned: Keep a second login open in advance when trying to simulate login failures! You should then see a report similar to:
```
root@hello:~# fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed:	2
|  |- Total failed:	14
|  `- Journal matches:	_SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
   |- Currently banned:	2
   |- Total banned:	2
   `- Banned IP list:	170.64.133.30 213.136.94.219
```
Searching for file name components should work like e.g.:
```
root@hello:~# locate ssh_host
/etc/ssh/ssh_host_ed25519_key
/etc/ssh/ssh_host_ed25519_key.pub
```

Figure 1008. Problem: Duplicate known_hosts entry on re-creating server

Problem of repeated terraform apply:

$ ssh root@128.140.108.60
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!

Figure 1009. Solution: Generating known_hosts ...

resource "local_file" "known_hosts" {
  content         = "${hcloud_server.helloServer.ipv4_address} ...
                      ... ${tls_private_key.host.public_key_openssh}"
  filename        = "gen/known_hosts"
  file_permission = "644"
}

Figure 1010. ... and ssh wrapper

main.tf tpl/ssh.sh

`main.tf`	`tpl/ssh.sh`
`resource "local_file" "ssh_script" { content = templatefile("tpl/ssh.sh", { ip=hcloud_server.hello.ipv4_address }) filename = "bin/ssh" file_permission = "700" depends_on = [local_file.known_hosts] }`	`#!/usr/bin/env bash GEN_DIR=$(dirname "$0")/../gen ssh -o UserKnownHostsFile= \ "$GEN_DIR/known_hosts" devops@${ip} "$@"`

resource "local_file" "ssh_script" {
  content = templatefile("tpl/ssh.sh", {
    ip=hcloud_server.hello.ipv4_address
  })
  filename        = "bin/ssh"
  file_permission = "700"
  depends_on      = [local_file.known_hosts]
}

#!/usr/bin/env bash

GEN_DIR=$(dirname "$0")/../gen

ssh -o UserKnownHostsFile= \
  "$GEN_DIR/known_hosts" devops@${ip} "$@"

exercise No. 8

Solving `~/.ssh/known_hosts` quirk

Q:	Extend Working on Cloud-init generating both a `bin/ssh` wrapper and a corresponding `gen/known_hosts` file.

Figure 1011. Failsafe console login

#cloud-config

chpasswd:
  list: |
    root:bingo9wingo
  expire: False

Figure 1012. Avoiding Yaml parsing issues

#cloud-config

write_files:
  - content:
        ${base64encode(private_key_pem)}
    encoding: base64
    path:  /etc/nginx/snippets/cert/private.pem

Note

Congrats to paultyng

Prev	Up	Next
Working with Terraform	Home	Volumes