# Goal of knowledge base

1. You only feel happy when you know what you doing
2. Don’t be lazy, don’t make excuses, no one cares. Work fucking harder.

![](/files/-LVrTiF8yPysZZiKcbZ_)


# Linux


# Record SSH session for reporting

The issue is whenever logging ssh to a linux vm for investigating or doing something, after done, I want to report on **what I did** for writing into ticket. In the past, I use the `history`command and scroll back to the beginning. It's waste of time & effort, sometime cannot trace because of lacking scroll or closing terminal tab by human mistake, or wanna trace in the next few days.

So that I need a good solution for this to work effectively

```bash
touch /ssh.log
ssh <target> -v | tee -ia /ssh.log
```

For seeing output for tracking & reporting, use `tail`

```bash
tail -f -n +1 /ssh.log
```


# Compress / Decompress files

### 0. Requirement packages

```
apt install tar zip unzip gzip
```

### 1. Compress

* Compress single file into `.tar.gz`&#x20;

```
tar -zcvf telegraf-legacy.log.tar.gz telegraf.log --totals --verbose
```

* Compress a directory into `.tar.gz`&#x20;

```
tar -zczf monit-5.26.0.tar.gz monit-5.26.0/ --totals --verbose
```

After that if you want **remove original file** then use with option `--remove-files`

* Compress single file into `.gz`&#x20;

```
# From tocdo.log --> tocdo.gz (dropped original file)

gzip tocdo.log 
```

* Compress single file into `.zip`

```
# From tocdo.log --> tocdo.zip (dropped original file)

zip x tocdo.log
```

### 2. Decompress&#x20;

```
tar -xzf foo.tar.gz --totals --verbose
```

```
unzip -a x.zip
```


# Colorize logs

Using package `ccze` for formatting input `tail -f /var/log/logsystem/llnx/fathom.log | ccze`

```
root@logsystem:~# apt search ccze
Sorting... Done
Full Text Search... Done
ccze/stable,now 0.2.1-3 amd64 [installed]
  robust, modular log coloriser
```

![ccze](/files/-L_Nd8bhQcGzHhnauutV)

&#x20;


# Cron output & logging

### A basic solution:

* use `$()` for executing `date` command and return output
* format **datetime** to UTC, escape the `%` character with `\`
* add `2>&1` at the end for streaming both `stdout` and `stderr` into that log file

### Example:

```
* * * * * echo "Test crontab log" > /tmp/crontab.log.$(date --utc +\%Y\%m\%d_\%H\%M\%SZ) 2>&1
```

### Output:

```
ls -lh /tmp | grep log

-rw-rw-r-- 1 ubuntu  ubuntu    17 May  4 05:06 crontab.log.20190504_050601Z
-rw-rw-r-- 1 ubuntu  ubuntu    17 May  4 05:07 crontab.log.20190504_050701Z
```

### Reference

1. <https://unix.stackexchange.com/questions/29578/how-can-i-execute-date-inside-of-a-cron-tab-job/517052#517052>
2. <https://crontab.guru/>
3. <https://yasoob.me/posts/6-tips-before-you-write-your-next-bash-cronjob/>


# Signal

About system calls, SIGINT, SIGTERM, SIGKILL

## Definition:

* System calls: communication chanel between user space program and kernel
* Signals: a different channel, used for inter-process communication
* Signals don't carry any agrgument, they are self explanatory by their name
* Some signals identified by a number, ie `SIGKILL` (9)
* That's why we use `kill -9 <PID>` to kill a process, because the kill command will send a defined signal to a process with a given identity `<PID>`
* when we run `kill -9 <PID>` command, that process is not terminate itself, instead we're telling that OS to stop running the program, no matter what the program is doing

## Some useful signals

* `SIGINT`: is the program interrupt signal. When user presses `CTRL+C`, the terminal emulator sends this signal to the foreground process, it will terminate process, but it can be caught or ignored, it a graceful shutdown
* `SIGTERM`: is the termination signal. It used to cause process termination, this signal can be blocked, handled and ignored. It is the normal way to ask a process to terminate. The `kill` command generates SIGTERM by default
* `SIGKILL`: is an immediate termination signal. It cannot be caught or ignored by the process, because when we send a `SIGKILL` to a proces, we remove any chance for that process to do a tidy cleanup and shutdown.&#x20;

For example, when a process does not die by using `Ctrl+C` (SIGINT), we should use the command `kill -9` on that process PID

* `SIGSTOP`: is a process suspend signal, which tells the OS to stop/suspend a process. This signal cannot be caught or ignored. To resume the process, use SIGCONT signal to continue
* `Ctrl+C`: The interrupt signal, sends `SIGINT` to the job running in the foreground.

When a process is in a limbo state it is reasonable to send the process the `SIGKILL` signal, which can be invoked by running the kill command with the -9 flag. Unlike `SIGTERM` the `SIGKILL` signal cannot be captured by the process and thus it cannot be ignored. The `SIGKILL` signal is handled outside of the process completely, and is used to stop the process immediately. The problem with using `SIGKILL` is that it does not allow an application to close its open files or database connections cleanly and over time could cause other issues; therefor it is generally better to reserve the `SIGKILL` signal as a last resort.

## Reference & Read more

1. <http://bencane.com/2014/04/01/understanding-the-kill-command-and-how-to-terminate-processes-in-linux/>
2. <https://en.wikipedia.org/wiki/Unix_signal>
3. <http://unix.stackexchange.com/questions/149741/why-is-sigint-not-propagated-to-child-process-when-sent-to-its-parent-process>
4. <http://askubuntu.com/questions/890591/why-doesnt-ctrl-c-kill-the-terminal-itself/890597>
5. <https://www.shellscript.sh/>
6. <https://www.win.tue.nl/~aeb/linux/lk/lk-5.html>
7. <https://lasr.cs.ucla.edu/vahab/resources/signals.html>
8. <http://www.linuxprogrammingblog.com/all-about-linux-signals?page=show>
9. ftp\://ftp.gnu.org/old-gnu/Manuals/glibc-2.2.3/html\_chapter/libc\_24.html
10. <http://askubuntu.com/questions/184071/what-is-the-purpose-of-the-9-option-in-the-kill-command>
11. <https://www.digitalocean.com/community/tutorials/how-to-use-ps-kill-and-nice-to-manage-processes-in-linux>
12. <https://s905060.gitbooks.io/site-reliability-engineer-handbook/content/signals.html>
13. <http://bencane.com/2014/04/01/understanding-the-kill-command-and-how-to-terminate-processes-in-linux/>
14. <https://major.io/2010/03/18/sigterm-vs-sigkill/>


# Break out and escape SSH session

### Problem

SSH session is stuck and cannot be exited by entering `exit` or `CTRL+D`&#x20;

And we need to break out this session without closing terminal emulator

### How to solve?

**Press** `~` **then press** `.`

Why? Because `~.` is an escape sequence that can terminate SSH session

More details:

```
Supported escape sequences:
  ~.  - terminate session
  ~B  - send a BREAK to the remote system
  ~R  - Request rekey (SSH protocol 2 only)
  ~#  - list forwarded connections
  ~?  - this message
  ~~  - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)
```


# Mount volume permanently

### List volumes

```
root@server:/# blkid

/dev/xvda1: LABEL="cloudimg-rootfs" UUID="ef263917-4ffc-4c36-880c-ae41d52b0d8e" TYPE="ext4"
/dev/xvdf: UUID="2c21a384-9e0e-4b44-b8d1-ceb452e8cc5c" TYPE="ext4"

root@server:/home/ubuntu# lsblk

NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0  32G  0 disk
└─xvda1 202:1    0  32G  0 part /
xvdf    202:80   0  32G  0 disk 
```

### Permanently mount

Mount volume `xvdf` to `/var/lib/mysql`

```
root@server:/# mount /dev/xvdf /var/lib/mysql
```

Recheck after mount

```
root@server:/# lsblk

NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0  32G  0 disk
└─xvda1 202:1    0  32G  0 part /
xvdf    202:80   0  32G  0 disk /var/lib/mysql
```

Mount volume permanently - even after rebooting

```
root@server:/# vim /etc/fstab

UUID="2c21a384-9e0e-4b44-b8d1-ceb452e8cc5c" /data ext4 defaults  0 0

root@server:/# mount -fav
/                        : ignored
/var/lib/mysql           : already mounted
```

We must config `fstab` (permanent mount) based on **`UUID`** or  **`LABEL`** like this

```
LABEL=cloudimg-rootfs                       /                      ext4  defaults,discard  0 0
UUID="c46cf311-d31b-41ce-bce5-5d8ad0a6b109" /var/lib/elasticsearch ext4  defaults,nofail  0 2
UUID="2c21a384-9e0e-4b44-b8d1-ceb452e8cc5c" /data                  ext4  defaults  0 0
```

**DON'T config based on device name** like this fuck

```
LABEL=cloudimg-rootfs	   /	                  ext4	defaults,discard	0 0
/dev/nvme1n1p1             /var/lib/elasticsearch ext4  defaults,nofail     0 2
/dev/xvdf                  /data                  ext4  defaults            0 0
```

### Mountfuck

![Entire fucking dir in fuckin /var/lib/elasticsearch](/files/-LXP1hKCjsmH8PWFRnWv)

Root cause: <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/nvme-ebs-volumes.html>

> EBS uses single-root I/O virtualization (SR-IOV) to provide volume attachments on Nitro-based instances using the NVMe specification.These devices rely on standard NVMe drivers on the operating system.These drivers typically discover attached devices by scanning the PCI bus during instance boot, and create device nodes based on the order in which the devices respond, not on how the devices are specified in the block device mapping.In Linux, NVMe device names follow the pattern /dev/nvme\<x>n\<y>, where \<x> is the enumeration order, and, for EBS, \<y> is 1.Occasionally, devices can respond to discovery in a different order in subsequent instance starts, which causes the device name to change.

So, if we use NVMe disk for some new types of AWS EC2, please note that the **device name is nearly randomize after each reboot**. It means if we have 2 NVMe disks on one EC2 vm, so we cannot know which device name delegate to which real disk.

```
/dev/nvme1n1p1
/dev/nvme0n1p1
```

### Reference

1. <https://askubuntu.com/questions/45607/how-to-mount-partition-permanently/45618#45618>
2. <https://docs.oracle.com/cloud/latest/computecs_common/OCSUG/GUID-075D8B13-A089-4A81-BB5C-DD7B09995C47.htm#OCSUG276>


# Show processes most consuming CPU & MEM

### For CPU

```
$ ps -aux --sort=-pcpu
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     30437  3.3  2.9 127052 60564 ?        S    Sep28 377:06 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170928064335Z backend-admin-deploy.yml
root     30440  3.3  2.6 194920 55220 ?        Sl   Sep28 377:06 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170928064335Z backend-admin-deploy.yml
root      2798  3.2  2.9 121952 59680 ?        S    Sep06 1401:46 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170906030038Z backend-admin-deploy.yml
root      2801  3.2  2.6 194916 55188 ?        Sl   Sep06 1415:24 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170906030038Z backend-admin-deploy.yml
root      2970  3.2  2.9 121952 59752 ?        S    Sep06 1409:28 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170906030350Z backend-admin-deploy.yml
root      2973  3.2  2.6 194916 55020 ?        Sl   Sep06 1416:16 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170906030350Z backend-admin-deploy.yml
root      3252  3.2  2.9 121952 59756 ?        S    Sep06 1408:45 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170906030604Z backend-admin-deploy.yml
root      3255  3.2  2.6 194916 54964 ?        Sl   Sep06 1413:53 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170906030604Z backend-admin-deploy.yml
root      3459  3.2  2.9 121952 59700 ?        S    Sep06 1412:12 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170906030817Z backend-admin-deploy.yml
root      3462  3.2  2.6 194916 55092 ?        Sl   Sep06 1410:22 /usr/bin/python /usr/bin/ansible-playbook -i hosts/web-qc -e ansistrano_release_version=20170906030817Z backend-admin-deploy.yml
jenkins  17319  0.1 26.9 2592812 552296 ?      Sl   Sep07  52:55 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
root         1  0.0  0.2 121692  5820 ?        Ss   Jun22   0:35 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Jun22   0:00 [kthreadd]
```

### For MEM

```
ps -aux --sort=+%mem | tail -n 10
```

### Reference: <https://www.pslinux.online/ps-aux.html>


# Improve and optimize battery life on Linux

### Goal

Maximum battery life on linux within reasonable and acceptable daily workloads

Case study: `Thinkpad x240` with `9cells` battery on `ubuntu 16.04`

* Before: from 100% to 5% - \~ `3.5hours`
* After installing **TLP**: \~ `5hours`&#x20;

### Solution

**TLP** brings you the benefits of advanced power management for Linux without the need to understand every technical detail. TLP comes with a default configuration **already optimized for battery life**, so you may **just install** and forget it. Nevertheless, TLP is highly customizable to fulfill your specific requirements.

**Please note**: `TLP` runs on every laptop brand. Setting the battery charge thresholds is available for IBM/Lenovo ThinkPads only.

### Installation

```
# Add PPA
sudo add-apt-repository ppa:linrunner/tlp
sudo apt-get update

# Especially for thinkpad
sudo apt-get install tp-smapi-dkms acpi-call-dkms

# Install
sudo apt-get install tlp tlp-rdw
```

**Start:** `TLP` will start automatically. To avoid having to restart the system, the first time you can start it manually by using the following command:

```
sudo tlp start
```

**Check:** Use the `tlp-stat` terminal command to check if TLP is working properly

```
▶ sudo tlp-stat -s
[sudo] password for devops: 
--- TLP 1.0 --------------------------------------------

+++ System Info
System         = LENOVO ThinkPad X240 20AMS1WY00
BIOS           = GIET75WW (2.25 )
Release        = Ubuntu 16.04.2 LTS
Kernel         = 4.10.0-27-generic #30~16.04.2-Ubuntu SMP Thu Jun 29 16:07:46 UTC 2017 x86_64
/proc/cmdline  = BOOT_IMAGE=/boot/vmlinuz-4.10.0-27-generic.efi.signed root=UUID=34cadb32-d632-4d98-a4a8-a1c53753dc35 ro quiet splash vt.handoff=7
Init system    = systemd v229
Boot mode      = UEFI

+++ TLP Status
State          = enabled
Last run       = 10:41:53 PM,   3685 sec(s) ago
Mode           = battery
Power source   = battery
```

### Reference

<http://linrunner.de/en/tlp/docs/tlp-linux-advanced-power-management.html>


# File ownership & groups in linux

### File concept

Every file in Linux is managed by a specific user and a specific group.

#### **1. Display ownership and group information:**

```
$ ls -l file.txt
-rw-rw-r-- 1 root www-data 0 Feb 25 15:51 file.txt
```

This file is owned by the root user and belongs to the www-data group.

#### **2. Change the ownership of a file by using chown**

**Important:** ONLY root user or members of the `sudo group` may transfer ownership of a file

```
$ sudo chown robert file.txt
$ ls -l file.txt
-rw-rw-r-- 1 robert www-data 0 Feb 25 15:51 file.txt
```

**3. Changing the Group Ownership of a file by using chgrp**

All users on the system belong to at least one group. You can find out which groups you belong to using the following command: `groups username`

Change the group ownership of a specific file using the chgrp command

```
$ chgrp webdev file.txt
$ ls -l file.txt
-rw-rw-r-- 1 robert webdev 0 Feb 25 15:51 file.txt
```

The file file.txt now belongs to the `webdev` group.

### Most important:

Change both the **owner** and **group** of a file using just the `chown` command

```
$ sudo chown tito:editors file.txt
$ ls -l file.txt
-rw-rw-r-- 1 tito editors 0 Feb 25 15:51 file.txt
```


# Automatic security update/patch on Ubuntu

### Problem & Goal:

Automatic update packages, every day, when have security patch

### Solution:

1. ubuntu ko tự upgrade cho mình
2. muốn tự upgrade thì cài gói `unattended-upgrades` vào, cấu hình upgrade cái gì (kernel, sec,… )
3. mỗi ngày system gọi thằng `/etc/cron.daily/apt-compat` lên làm
4. trong thằng `/etc/cron.daily/apt-compat`, nó có gọi thằng `/usr/lib/apt/apt.systemd.daily` lên execute
5. trong đây nó có chỗ set biến là 0 (tức là disable), đọc cái file `50unattended-upgrades` lên, lấy giá trị của nó, override lại cái biến khai báo = 0 ban đầu, nếu override rồi mà vẫn = 0 thì ko chạy, còn ra =1 thì update lên

Debug by using kernel/system log: `alternatives.log` + `unattended-upgrades` + `dpkg.log`

### Reference:

<https://help.ubuntu.com/community/AutomaticSecurityUpdates>


# Clean buffers and cached on linux

Need root permission

```
# clean buffer and cached
root@appv2-1:~# free -m && sync && echo 3 > /proc/sys/vm/drop_caches && free -m

# step by step
root@appv2-1:~# free -m
             total       used       free     shared    buffers     cached
Mem:          3951       3260        691          0         46        121
-/+ buffers/cache:       3092        859
Swap:         4095          0       4095

root@appv2-1:~# sync
root@appv2-1:~# echo 3 > /proc/sys/vm/drop_caches

root@appv2-1:~# free -m
             total       used       free     shared    buffers     cached
Mem:          3951       3065        886          0          2         22
-/+ buffers/cache:       3040        911
Swap:         4095          0       4095
```

Before:&#x20;

* `buffers = 46 mb`
* `cached = 121 mb`

After

* `buffers = 2 mb`
* `cached = 22 mb`


# Bash completion on Linux/Mac

### 1. Debian 9

By default, `debian 9 stretch` does not have completion as `ubuntu` distro, we will need it in some specific cases and can speed up daily operation tasks

* Autocomplete / hint when pressing tab
* Apply for all user/profile

```
apt install bash-completion

# In file /etc/profile
if [ -f /etc/bash_completion ]; then
 . /etc/bash_completion
fi
```

### 2. Mac (with wireguard vpn)&#x20;

```
bash --version
GNU bash, version 5.0.7(1)-release (x86_64-apple-darwin18.5.0)

brew install bash-completion
brew reinstall bash-completion

# In file ~/.bash_profile

[[ ${BASH_VERSINFO[0]} -ge 4 ]] || return 0
if [ -f $(brew --prefix)/etc/bash_completion ]; then
    . $(brew --prefix)/etc/bash_completion
fi
```

### Reference

1. <https://lists.zx2c4.com/pipermail/wireguard/2018-October/003418.html>
2. <https://github.com/Homebrew/homebrew-core/issues/32535>


# Core services


# Nginx reload

### Nginx reload - reloading the configuration file

Changes made in the configuration file will not be applied until the command to reload configuration is sent to nginx or it is restarted. To reload configuration, execute:

```
nginx -s reload
```

* Once the master process receives the signal to reload configuration, it checks the syntax validity of the new configuration file and tries to apply the configuration provided in it.&#x20;
* If this is a success, the master process starts new worker processes and sends messages to old worker processes, requesting them to shut down. Otherwise, the master process rolls back the changes and continues to work with the old configuration.&#x20;
* Old worker processes, receiving a command to shut down, stop accepting new connections and continue to service current requests until all such requests are serviced. After that, the old worker processes exit.

### Reference:

1. <http://nginx.org/en/docs/beginners_guide.html>
2. <https://www.nginx.com/resources/wiki/start/topics/tutorials/commandline/>


# OpenVPN Split tunneling

### Fundamental

For example, suppose a user utilizes a remote access VPN software client connecting to a corporate network using a hotel wireless network. The user with split tunneling enabled is able to connect to file servers, database servers, mail servers and other servers on the corporate network through the VPN connection. When the user connects to Internet resources (Web sites, FTP sites, etc.), the connection request goes directly out the gateway provided by the hotel network.

#### Advantage

* Alleviate bottlenecks and conserve bandwidth as Internet traffic does not have to pass through the VPN server.
* A user works at a supplier or partner site and needs access to network resources on both networks throughout the day. Split tunneling prevents the user from having to continually connect and disconnect.

### Configuration

**Goal**:

* Direct connect for most requests, don't use VPN
* only requests from local client to `171.253.181.55` is in the tunnel, secure, encrypted

**Current**

* OpenVPN server IP: 45.79.85.159 from us
* client configuration file: `client.ovpn`

**Config**

```
$ vim client.ovpn
# Add 2 lines into beginning of this client config file
# route-nopull 
# route  171.253.181.55
```

{% code title="client.ovpn" %}

```ruby
route-nopull 
route  171.253.181.55
client
dev tun
proto udp
sndbuf 0
rcvbuf 0
remote 45.79.85.159 1194
...
```

{% endcode %}

### Reference

1. <https://en.wikipedia.org/wiki/Split_tunneling>
2. <https://www.ibvpn.com/billing/knowledgebase/330/Split-Tunneling-for-OpenVPN-GUI.html>


# Nmap commands

NOTE: Add `-F` if you want to **scan faster** because it's fast mode that will **scan fewer ports** than the default scan

### Enable scripts, service detection, OS fingerprinting and traceroute

```bash
sudo nmap -A -Pn 45.79.85.159

Starting Nmap 7.60 ( https://nmap.org ) at 2017-11-08 20:52 +07
Nmap scan report for li1184-159.members.linode.com (45.79.85.159)
Host is up (0.19s latency).
Not shown: 996 closed ports
PORT     STATE SERVICE     VERSION
22/tcp   open  ssh         OpenSSH 7.2p2 Ubuntu 4ubuntu2.2 (Ubuntu Linux; protocol 2.0)
| ssh-hostkey:
|   2048 c1:bd:c3:9e:75:74:27:76:f7:a3:21:25:c5:bf:41:ea (RSA)
|   256 64:e6:37:97:dc:f7:f0:69:e0:51:f2:73:2d:11:17:fe (ECDSA)
|_  256 d1:0d:f4:74:d9:41:d9:85:32:d2:74:e1:8d:ef:14:8d (EdDSA)
25/tcp   open  smtp        Postfix smtpd
|_smtp-commands: usnode.members.linode.com, PIPELINING, SIZE 10240000, VRFY, ETRN, STARTTLS, ENHANCEDSTATUSCODES, 8BITMIME, DSN,
| ssl-cert: Subject: commonName=usnode
| Not valid before: 2017-10-17T14:41:31
|_Not valid after:  2027-10-15T14:41:31
|_ssl-date: TLS randomness does not represent time
80/tcp   open  http        nginx 1.10.3 (Ubuntu)
|_http-server-header: nginx/1.10.3 (Ubuntu)
|_http-title: Welcome to nginx!
9000/tcp open  cslistener?
| fingerprint-strings:
|   FourOhFourRequest:
|     HTTP/1.0 400 Bad Request
|     Accept-Ranges: bytes
|     Content-Type: application/xml
|     Server: Minio/DEVELOPMENT.2017-10-29T10-14-45Z (linux; amd64)
|     Vary: Origin
|     X-Amz-Request-Id: 14F520AC5557517F
|     Date: Wed, 08 Nov 2017 13:52:54 GMT
|     <?xml version="1.0" encoding="UTF-8"?>
|     <Error><Code>InvalidBucketName</Code><Message>The specified bucket is not valid.</Message><Key></Key><BucketName></BucketName><Resource>/nice ports,/Trinity.txt.bak</Resource><RequestId>3L137</RequestId><HostId>3L137</HostId></Error>
|   GetRequest:
|     HTTP/1.0 403 Forbidden
|     Accept-Ranges: bytes
|     Content-Type: application/xml
|     Server: Minio/DEVELOPMENT.2017-10-29T10-14-45Z (linux; amd64)
|     Vary: Origin
|     X-Amz-Request-Id: 14F520A98FF1826F
|     Date: Wed, 08 Nov 2017 13:52:42 GMT
|     <?xml version="1.0" encoding="UTF-8"?>
|     <Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key></Key><BucketName></BucketName><Resource>/</Resource><RequestId>3L137</RequestId><HostId>3L137</HostId></Error>
|   HTTPOptions:
|     HTTP/1.0 200 OK
|     Vary: Origin
|     Vary: Access-Control-Request-Method
|     Vary: Access-Control-Request-Headers
|     Date: Wed, 08 Nov 2017 13:52:43 GMT
|     Content-Length: 0
|     Content-Type: text/plain; charset=utf-8
|   RTSPRequest, SIPOptions:
|     HTTP/1.1 400 Bad Request
|     Content-Type: text/plain; charset=utf-8
|     Connection: close
|_    Request
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service :
SF-Port9000-TCP:V=7.60%I=7%D=11/8%Time=5A030C2A%P=x86_64-apple-darwin16.7.
SF:0%r(GetRequest,1C1,"HTTP/1\.0\x20403\x20Forbidden\r\nAccept-Ranges:\x20
SF:bytes\r\nContent-Type:\x20application/xml\r\nServer:\x20Minio/DEVELOPME
SF:NT\.2017-10-29T10-14-45Z\x20\(linux;\x20amd64\)\r\nVary:\x20Origin\r\nX
SF:-Amz-Request-Id:\x2014F520A98FF1826F\r\nDate:\x20Wed,\x2008\x20Nov\x202
SF:017\x2013:52:42\x20GMT\r\n\r\n<\?xml\x20version=\"1\.0\"\x20encoding=\"
SF:UTF-8\"\?>\n<Error><Code>AccessDenied</Code><Message>Access\x20Denied\.
SF:</Message><Key></Key><BucketName></BucketName><Resource>/</Resource><Re
SF:questId>3L137</RequestId><HostId>3L137</HostId></Error>")%r(HTTPOptions
SF:,CD,"HTTP/1\.0\x20200\x20OK\r\nVary:\x20Origin\r\nVary:\x20Access-Contr
SF:ol-Request-Method\r\nVary:\x20Access-Control-Request-Headers\r\nDate:\x
SF:20Wed,\x2008\x20Nov\x202017\x2013:52:43\x20GMT\r\nContent-Length:\x200\
SF:r\nContent-Type:\x20text/plain;\x20charset=utf-8\r\n\r\n")%r(RTSPReques
SF:t,67,"HTTP/1\.1\x20400\x20Bad\x20Request\r\nContent-Type:\x20text/plain
SF:;\x20charset=utf-8\r\nConnection:\x20close\r\n\r\n400\x20Bad\x20Request
SF:")%r(FourOhFourRequest,1F7,"HTTP/1\.0\x20400\x20Bad\x20Request\r\nAccep
SF:t-Ranges:\x20bytes\r\nContent-Type:\x20application/xml\r\nServer:\x20Mi
SF:nio/DEVELOPMENT\.2017-10-29T10-14-45Z\x20\(linux;\x20amd64\)\r\nVary:\x
SF:20Origin\r\nX-Amz-Request-Id:\x2014F520AC5557517F\r\nDate:\x20Wed,\x200
SF:8\x20Nov\x202017\x2013:52:54\x20GMT\r\n\r\n<\?xml\x20version=\"1\.0\"\x
SF:20encoding=\"UTF-8\"\?>\n<Error><Code>InvalidBucketName</Code><Message>
SF:The\x20specified\x20bucket\x20is\x20not\x20valid\.</Message><Key></Key>
SF:<BucketName></BucketName><Resource>/nice\x20ports,/Trinity\.txt\.bak</R
SF:esource><RequestId>3L137</RequestId><HostId>3L137</HostId></Error>")%r(
SF:SIPOptions,67,"HTTP/1\.1\x20400\x20Bad\x20Request\r\nContent-Type:\x20t
SF:ext/plain;\x20charset=utf-8\r\nConnection:\x20close\r\n\r\n400\x20Bad\x
SF:20Request");
Device type: general purpose|WAP|storage-misc|broadband router
Running (JUST GUESSING): Linux 3.X|4.X|2.6.X|2.4.X (95%), Asus embedded (92%), HP embedded (91%)
OS CPE: cpe:/o:linux:linux_kernel:3 cpe:/o:linux:linux_kernel:4 cpe:/o:linux:linux_kernel cpe:/h:asus:rt-ac66u cpe:/h:hp:p2000_g3 cpe:/o:linux:linux_kernel:3.4 cpe:/o:linux:linux_kernel:2.6.22 cpe:/o:linux:linux_kernel:2.4
Aggressive OS guesses: Linux 3.10 - 4.8 (95%), Linux 3.13 (95%), Linux 3.13 or 4.2 (95%), Linux 4.4 (95%), Linux 3.16 (94%), Linux 3.16 - 4.6 (94%), Linux 3.12 (93%), Linux 3.2 - 4.8 (93%), Linux 3.8 - 3.11 (93%), Asus RT-AC66U WAP (92%)
No exact OS matches for host (test conditions non-ideal).
Network Distance: 12 hops
Service Info: Host:  usnode.members.linode.com; OS: Linux; CPE: cpe:/o:linux:linux_kernel

TRACEROUTE (using port 53/tcp)
HOP RTT       ADDRESS
1   4.28 ms   172.16.0.1
2   6.12 ms   static.vnpt.vn (14.169.128.1)
3   ...
4   6.07 ms   static.vnpt.vn (113.171.14.37)
5   5.49 ms   static.vnpt.vn (113.171.7.209)
6   ...
7   392.21 ms unknown.telstraglobal.net (202.127.78.129)
8   ...
9   78.34 ms  100ge8-2.core1.tyo1.he.net (184.105.64.130)
10  201.74 ms 100ge8-1.core1.sea1.he.net (184.105.213.117)
11  180.47 ms 173.230.159.3
12  182.92 ms li1184-159.members.linode.com (45.79.85.159)

OS and Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 77.96 seconds
```

### Perform TCP and UDP scanning

```bash
sudo nmap -sSU 45.79.85.159

Starting Nmap 7.60 ( https://nmap.org ) at 2017-11-08 20:57 +07
sendto in send_ip_packet_sd: sendto(4, packet, 28, 0, 45.79.85.159, 16) => Network is down
Offending packet: UDP 172.16.6.163:33418 > 45.79.85.159:49176 ttl=44 id=39224 iplen=7168
Stats: 0:02:28 elapsed; 0 hosts completed (1 up), 1 undergoing UDP Scan
UDP Scan Timing: About 16.00% done; ETC: 21:12 (0:12:26 remaining)
Stats: 0:02:29 elapsed; 0 hosts completed (1 up), 1 undergoing UDP Scan
.
.
UDP Scan Timing: About 63.58% done; ETC: 21:14 (0:06:03 remaining)
Nmap scan report for li1184-159.members.linode.com (45.79.85.159)
Host is up (0.19s latency).
Not shown: 1995 closed ports
PORT     STATE         SERVICE
22/tcp   open          ssh
25/tcp   open          smtp
80/tcp   open          http
9000/tcp open          cslistener
53/udp   open|filtered domain

Nmap done: 1 IP address (1 host up) scanned in 1045.86 seconds.
```

![](/files/-LXOxRdAq-tTS4iMoM-B)


# Hardware


# CPU Architecture fundamental

### Terminology

* `Processor`: the **physical chip** that plugs into a socket on the system or processor board and contains one or more CPUs **implemented as cores or hardware threads**.
* `Core`: an independent **CPU instance** on a multicore processor. The use of cores is a way to scale processors, called chip-level multiprocessing (CMP).
* `Hardware thread`: a CPU architecture that supports executing **multiple threads in parallel on a single core** (including Intel’s Hyper-Threading Technology), where each thread is an independent CPU instance. One name for this scaling approach is **multithreading**.
* `CPU instruction`: a single CPU operation, from its **instruction set**. There are instructions for arithmetic operations, memory I/O, and control logic.
* `Logical CPU`: also called a virtual processor (vCPU), an operating system CPU instance (a schedulable CPU entity). This may be implemented by the processor as a hardware thread (in which case it may also be called a virtual core), a core, or a single-core processor.
* `Scheduler`: the kernel subsystem that **assigns threads** to run on CPUs.
* `Run queue`: a queue of runnable threads that are **waiting to be serviced** by CPUs. For Solaris, it is often called a dispatcher queue.

### Example

Intel® Core™ [i7-5557U](https://ark.intel.com/products/84993/Intel-Core-i7-5557U-Processor-4M-Cache-up-to-3_40-GHz) Processor on Macbook pro: 2 cores, 4 threads

```
sysctl -n hw.ncpu # = 4
sysctl -n hw.physicalcpu # = 2
sysctl -n hw.logicalcpu # = 4
```

### Reference

Chapter 6: CPUs in <http://www.brendangregg.com/sysperfbook.html>


# MySQL


# InnoDB - innodb\_file\_per\_table parameter

### Working with InnoDB Tablespaces to Improve Crash Recovery Times

Every table in MySQL consists of a table definition, data, and indexes. The MySQL storage engine InnoDB stores table data and indexes in a ***tablespace***. InnoDB creates a global shared **tablespace** that contains a data dictionary and other relevant metadata, and it can contain table data and indexes. InnoDB can also create separate tablespaces for each table and partition. These separate tablespaces are stored in files with a .ibd extension and the header of each tablespace contains a number that uniquely identifies it.

Amazon RDS provides a parameter in a MySQL parameter group called `innodb_file_per_table`. This parameters controls whether InnoDB adds new table data and indexes to the shared tablespace (by setting the parameter value to 0) or to individual tablespaces (by setting the parameter value to 1). Amazon RDS sets the default value for `innodb_file_per_table` parameter to 1, which allows you to drop individual InnoDB tables and reclaim storage used by those tables for the DB instance. In most use cases, setting the `innodb_file_per_table` parameter to 1 is the recommended setting.

You should set the `innodb_file_per_table` parameter to 0 when you have a large number of tables, such as over 1000 tables when you use standard (magnetic) or general purpose SSD storage or over 10,000 tables when you use Provisioned IOPS storage. When you set this parameter to 0, individual tablespaces are not created and this can improve the time it takes for database crash recovery.

MySQL processes each metadata file, which includes tablespaces, during the crash recovery cycle. The time it takes MySQL to process the metadata information in the shared tablespace is negligible compared to the time it takes to process thousands of tablespace files when there are multiple tablespaces. Because the tablespace number is stored within the header of each file, the aggregate time to read all the tablespace files can take up to several hours. For example, a million InnoDB tablespaces on standard storage can take from five to eight hours to process during a crash recovery cycle. In some cases, InnoDB can determine that it needs additional cleanup after a crash recovery cycle so it will begin another crash recovery cycle, which will extend the recovery time. Keep in mind that a crash recovery cycle also entails rolling-back transactions, fixing broken pages, and other operations in addition to the processing of tablespace information.

Since the `innodb_file_per_table` parameter resides in a parameter group, you can change the parameter value by editing the parameter group used by your DB instance without having to reboot the DB instance. After the setting is changed, for example, from 1 (create individual tables) to 0 (use shared tablespace), new InnoDB tables will be added to the shared tablespace while existing tables continue to have individual tablespaces. To move an InnoDB table to the shared tablespace, you must use the `ALTER TABLE`command.

### Real use case - example

One RDS MySQL 5.7 instance (`gp2`) have over **93 databases** (Multi-tenant SaaS database arch). Each database has the same schema & structure, having exactly **96 tables per database**.

So totally we have `93 x 96 = 8928 tables` need to take care.

Then the parameter should be `innodb_file_per_table = 0`

### References

1. <https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.MySQL.CommonDBATasks.html>
2. <https://dev.mysql.com/doc/refman/8.0/en/innodb-multiple-tablespaces.html>

<br>


# MySQL - enable slow query log

### What is slow query log?

* Slow queries can affect database and server performance.&#x20;
* The slow query log consists of SQL statements that took more than `long_query_time` seconds to execute.&#x20;
* This greatly simplifies the task of finding inefficient or time-consuming queries.
* **By default, the slow query log is disabled**

### How to enable?

1. Login to MySQL, database `abc_prod`

   ```sql
    shell> mysql -u root -p abc_prod
   ```
2. Enable slow query log

   ```sql
    mysql> SET GLOBAL slow_query_log = 'ON';
   ```
3. Check path to log file

   ```sql
    mysql> SHOW VARIABLES LIKE 'slow_query_log_file';
    +---------------------+-----------------------------+
    | Variable_name       | Value                       |
    +---------------------+-----------------------------+
    | slow_query_log_file | /var/lib/mysql/db3-slow.log |
    +---------------------+-----------------------------+
    1 row in set (0.00 sec)
   ```
4. Change long query time to 5 seconds - **default is 10 seconds**

   ```sql
    mysql> SET GLOBAL long_query_time = 5;
   ```
5. Logout MySQL session then login again
6. Take a small test to ensure slow query log is enable

   ```sql
    mysql> SELECT SLEEP(10);
   ```
7. Check sleep on slow query log

   ```
    root@db3:/home/ubuntu:~$ cat /var/lib/mysql/db3-slow.log
   ```

   ```sql
    /usr/sbin/mysqld, Version: 5.6.35-log (MySQL Community Server (GPL)). started with:
    Tcp port: 3306  Unix socket: /var/run/mysqld/mysqld.sock
    Time                 Id Command    Argument
    # Time: 170321  7:15:52
    # User@Host: root[root] @ localhost []  Id:  1160
    # Query_time: 7.000249  Lock_time: 0.000000 Rows_sent: 1  Rows_examined: 0
    SET timestamp=1490080552;
    SELECT SLEEP(7);
    # Time: 170321  7:17:31
    # User@Host: root[root] @ localhost []  Id:  1161
    # Query_time: 10.000215  Lock_time: 0.000000 Rows_sent: 1  Rows_examined: 0
    use abc_prod;
    SET timestamp=1490080651;
    SELECT SLEEP(10);
   ```

**Most important:**

Using `mysqldumpslow` tool for summarize slow query log files

```
root@db3:/home/ubuntu:~$ mysqldumpslow /var/lib/mysql/db3-slow.log
```

```sql
Reading mysql slow query log from /var/lib/mysql/db3-slow.log
Count: 2  Time=8.50s (17s)  Lock=0.00s (0s)  Rows=1.0 (2), root[root]@localhost
  SELECT SLEEP(N)
```

### How to turn it off?

```sql
mysql> SET GLOBAL slow_query_log = 'OFF';
```


# MySQL - export large tables

### 1. Goal

Exporting large tables of MySQL database with showing progress and compression output

### 2. Implementation

```
tmux new -s exporting
mysqldump -udevops -pxxxxxx -hhost.xxxx \
           dbname table1 table2 table3 | pv | gzip -9 > 3tables.sql.gz
```


# MongoDB


# Docker


# ADD or COPY in Dockerfile

### Concept & fundamental

`ADD` and `COPY` are functionally similar

* `ADD` has some features like local-only tar extraction and **remote URL support**

  Consequently, the best use for `ADD` is local tar file auto-extraction into the image, as in&#x20;

  ```
   ADD rootfs.tar.xz /.
  ```
* `COPY` **only** supports the basic copying of local files into the container

  &#x20;With multiple steps in Dockerfile, that use **different files** from your context  --> `COPY` them individually, rather than all at once. This will ensure that each step’s build cache is only invalidated (forcing the step to be re-run) if the specifically required files change.

  ```
   COPY requirements.txt /tmp/
   RUN pip install --requirement /tmp/requirements.txt
   COPY . /tmp/
  ```

  &#x20;Results in fewer cache invalidations for the `RUN` step, than if you put the `COPY . /tmp/` before it.

Because image size matters, using `ADD` to fetch packages from remote URLs is **strongly discouraged**; you **should use** `curl` **or** `wget` instead. That way you can delete the files you no longer need after they’ve been extracted and you won’t have to add another layer in your image.

For example, you should avoid doing things like:

```
ADD http://example.com/big.tar.xz /usr/src/things/
RUN tar -xJf /usr/src/things/big.tar.xz -C /usr/src/things
RUN make -C /usr/src/things all
```

And instead, **do** something like ⭐️ :

```
RUN mkdir -p /usr/src/things \
    && curl -SL http://example.com/big.tar.xz \
    | tar -xJC /usr/src/things \
    && make -C /usr/src/things all
```

For other items (files, directories) that do not require `ADD`’s tar auto-extraction capability

You should always use `COPY`.

### Reference:

1. <https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#add-or-copy>
2. <https://stackoverflow.com/questions/24958140/what-is-the-difference-between-the-copy-and-add-commands-in-a-dockerfile>


# Clean data of docker completely

### Goal

Getting low on space, `/var/lib/docker/aufs` takes a lot of space. Need to remove all of them

```
root@CI:/var/lib/docker/aufs# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            992M     0  992M   0% /dev
tmpfs           200M  3.2M  197M   2% /run
/dev/xvda1       20G   19G  708M  97% /
tmpfs          1000M     0 1000M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs          1000M     0 1000M   0% /sys/fs/cgroup
tmpfs           200M     0  200M   0% /run/user/112
tmpfs           200M     0  200M   0% /run/user/1000

root@CI:/var/lib/docker# du -shc *
16G    aufs
4.0K    containers
332K    image
60K    network
20K    plugins
4.0K    swarm
4.0K    tmp
4.0K    trust
60K    volumes
16G    total
```

### Solution

```
docker rm $(docker ps -a -q)
docker rmi --force $(docker images -q)
docker system prune --force
systemctl stop docker
rm -rf /var/lib/docker/aufs
apt-get autoclean
apt-get autoremove
systemctl start docker
```


# Podman


# Ansible


# Output format

### Human-readable

Best way to format output for easy debugging is running with one level of verbose `-v` and config `ANSIBLE_STDOUT_CALLBACK=debug` for callback output login, also print pure `stdout`&#x20;

Or add `stdout_callback = debug` to file `ansible.cfg`

```yaml
TASK [Config backend directory] ************************************************
changed: [default] => {
    "changed": true, 
    "gid": 33, 
    "group": "www-data", 
    "mode": "0775", 
    "owner": "www-data", 
    "path": "/var/www", 
    "size": 4096, 
    "state": "directory", 
    "uid": 33
}

TASK [shell] *******************************************************************
changed: [default] => {
    "changed": true, 
    "cmd": "cp -r /home/ubuntu/* /****/", 
    "delta": "0:00:00.003310", 
    "end": "2019-02-20 04:27:35.078530", 
    "rc": 0, 
    "start": "2019-02-20 04:27:35.075220"
}

TASK [copy] ********************************************************************
changed: [default] => {
    "changed": true, 
    "checksum": "827c6dd2d1f982794c8bf413708c9b8207252c5b", 
    "dest": "/etc/supervisor/conf.d/msg-worker-supervisord.conf", 
    "gid": 0, 
    "group": "****", 
    "md5sum": "6d007a703729673056bc9b0ec4a1367b", 
    "mode": "0775", 
    "owner": "****", 
    "size": 261, 
    "src": "/****/msg-worker-supervisord.conf", 
    "state": "file", 
    "uid": 0
}

TASK [Run migration on template subdomain] *************************************
changed: [default] => {
    "changed": true, 
    "cmd": "runuser --user=www-data -- /var/www/backend-symfony/current/bin/console doctrine:migrations:migrate --instance=instance_template.ahihi.io --env=prod", 
    "delta": "0:00:00.458270", 
    "end": "2019-02-20 04:27:37.223567", 
    "rc": 0, 
    "start": "2019-02-20 04:27:36.765297"
}

STDOUT:

****@instance_template: 
                                                              
                    Application Migrations                    
                                                              

Migrating up to 20190218074800 from 20190215121527

  ++ migrating 20190218074800

     -> DELETE FROM contents_status_translations WHERE status_id IN (1, 2, 3, 4)
     -> INSERT INTO contents_status_translations (status_id, `language`, field, `value`) VALUES (1, 'de', 'name', 'Idee'), (2, 'de', 'name', 'In Produktion'), (3, 'de', 'name', 'Veröffentlicht'), (4, 'de', 'name', 'Abgelehnt')

  ++ migrated (0.06s)

  ------------------------

  ++ finished in 0.06s
  ++ 1 migrations executed
  ++ 2 sql queries
```

### Reference

1. Docs: <https://docs.ansible.com/ansible/devel/plugins/callback.html>
2. Comparison: <https://rndmh3ro.github.io/>


# Jenkins


# Jenkins - force exit pipeline when failure

### 1. Issue

In some specific cases, if we use `try` `catch` exception, Jenkins knows that issues, but still continue running other stage in pipeline. We need a force handling to make this pipeline failure immediately

### 2. Solution

```groovy
currentBuild.result = 'FAILURE'
error("Failure right here, force exit")
```

```groovy
stage("DB migration") {
  steps { script {
    slackSend (color: COLOR_MAP['SUCCESS'],
               channel: '#live-deployment',
               token: 'Fawxxxxxxxxxxxx',
               message: "*`STARTING`*: *core db migrating*, build #${env.BUILD_NUMBER}, <${env.BUILD_URL}|Go to this job>")
    try {
      withCredentials([usernamePassword(credentialsId: 'xxxxx-credential', usernameVariable: 'DB_USERNAME', passwordVariable: 'DB_PASSWORD')]) {
        withAWS(region:"eu-cexxxxxxx", credentials:"aws") {
          sh """
            packer build -force dbmigration/vm-db-migration.json
          """
        }
      }
    } catch (error) {
        slackSend (color: COLOR_MAP['FAILURE'],
                   channel: '#live-deployment',
                   token: 'Fawxxxxxxxxxxxx',
                   message: "*`FAILURE`*: *core db migrating*, build #${env.BUILD_NUMBER} \n${error}\n<${env.BUILD_URL}|Go to this job>")
        currentBuild.result = 'FAILURE'
        error("${error}")
    }
  }}
}
```


# PHP


# Composer

### 0. Introduction

`composer` is a PHP binary script and it's using [Unix Shebang](https://en.wikipedia.org/wiki/Shebang_\(Unix\)) to define which interpreter for executing.

```
root@home:~# head /usr/local/bin/composer -n 5

#!/usr/bin/env php
<?php
/*
 * This file is part of Composer.
 *


/usr/bin/php7.1 /usr/local/bin/composer update
```

### 1. Execute composer with specific PHP version

So if you're running `composer` without any specific option, it will use your PHP version at `/usr/bin/env php` (which is your default PHP)

We can have many versions of PHP inside your Linux and you can flexible use which PHP version for executing as you want like this way:

```
root@home:~# /usr/bin/php7.1 /usr/local/bin/composer update
```

Or just

```
root@home:~# php7.1 /usr/local/bin/composer
root@home:~# php7.2 /usr/local/bin/composer
root@home:~# php7.3 composer
...
```

### 2. Check & find all installed PHP packages

```
root@home:~# ls /usr/bin/ | grep php
php
php5.6
php7.1
php7.2
php7.3
php-config
php-config7.1
phpdbg
phpdbg5.6
phpize
phpize7.1
```


# php-redis & php-igbinary

### 1. Issue

```
+ php7.3 /usr/local/bin/composer install --ansi
 PHP Warning:  PHP Startup: Unable to load dynamic library 'redis.so' 
 (tried: /usr/lib/php/20180731/redis.so (/usr/lib/php/20180731/redis.so: undefined symbol: igbinary_serialize), 
 /usr/lib/php/20180731/redis.so.so (/usr/lib/php/20180731/redis.so.so: undefined symbol: igbinary_serialize)) 
 in Unknown on line 0
```

### 2. Solution

Outdated PHP package `php-igbinary` (`2.x`) may cause that issue `/usr/lib/php/20180731/redis.so: undefined symbol: igbinary_serialize`

```
root@home:# dpkg -l | grep php

php-igbinary                         2.0.6~rc1-1+ubuntu16.04.1+deb.sury.org+1
php-redis                            4.3.0-1+ubuntu16.04.1+deb.sury.org+1
```

Solution is re-install both `php-redis` & `php-igbinary`

```
root@home:# apt install php-redis php-igbinary
...
```

After that, they're working smoothly. Here is new version of `php-igbinary`

```
php-igbinary                         3.0.1+2.0.8-1+ubuntu16.04.1+deb.sury.org+1
php-redis                            4.3.0-1+ubuntu16.04.1+deb.sury.org+1
```


# Technical based


# Writing well

### Undervalued Software Engineering Skills: Writing Well

I have noticed a few skills that people often underestimate the importance of developing. Skills that add a significant boost to the impact of any developer. One of these is writing.

**It is with a larger organisation that writing becomes important** for messages to reach a wider group of people. For software engineers, writing becomes the tool to reach, converse with and influence engineers and teams outside their immediate peers. Writing becomes essential to make thoughts, tradeoffs and decisions durable. Writing things down makes these thoughts available for a wide range of people to read. Things that should be made durable can include proposals and decisions, coding guidelines, best practices, learnings, runbooks, debugging guides, postmortems. Even code reviews.

**For people to read what you write, it needs to be written well**. If you grab people's attention early on, they will keep reading and they will receive the message you intended to get across. More of them will respond to it and do it without few misunderstandings on what you meant. By writing well, you can scale your ability to communicate efficiently to multiple teams, to an organisation or across the company. And the ability to communicate and influence beyond your immediate team is the essential skill for engineers growing in seniority - from senior engineer to what organizations might call lead, principle, staff or distinguished engineer.

### References

1. <https://blog.pragmaticengineer.com/on-writing-well/>

![](/files/-LgRzKws-O1Ac4fKWJjW)


# Reinvent The Wheel

### Don't Reinvent The Wheel, Unless You Plan on Learning More About Wheels

Indeed. If anything, "Don't Reinvent The Wheel" should be used as a call to arms for deeply educating yourself about all the existing solutions – not as a bludgeoning tool to undermine those who legitimately want to build something better or improve on what's already out there. In my experience, sadly, it's much more the latter than the former. **So, no, you shouldn't reinvent the wheel. Unless you plan on learning more about wheels**, that is.

Reference

1. <https://blog.codinghorror.com/dont-reinvent-the-wheel-unless-you-plan-on-learning-more-about-wheels/>
2. <https://en.wikipedia.org/wiki/Reinventing_the_wheel>


# Approach a new system

### Khi tiếp cận một hệ thống mới, nên follow theo những hướng như sau:

1. Tìm hiểu tổng quan của system, architecture
2. **Đào sâu** từng module của system, **phải** hiểu tất cả câu trả lời của những câu hỏi dưới đây:
   * Module/service này là gì?
   * Tại sao lại cần nó?
   * How it works?

### Khi hiện thực feature/service/module mới cho system:

1. What is the most important thing that we need to do right now
2. Trả lời câu hỏi **Tại sao lại cần nó?**
3. List ra những solution hiện có (from google, github, community, ...)
4. List advantage and disadvantage of all solutions
5. Choose right tool for right job&#x20;
6. Cố gắng tìm tất cả document có sẵn, những thứ related với vấn đề cần build
7. Hiểu những inventory **hiện có** trên system để hỗ trợ cho the new thing

### Khi module/service cũ có lỗi:

1. Tìm mọi log của module/service đó, có thể là log của service, có thể là log từ webapp, log từ các module connect tới nó
2. Đọc hiểu kĩ càng, **hiểu rõ lỗi** trước khi bắt tay vào fix
3. **Hạn chế** rebuild all the thing from scratch, nên dựa vào những step, những document có sẵn, xem người trước họ implement thế nào, tại sao họ lại làm vậy?


# Backup philosophy

### Mindset

* 3-2-1 Backup strategy
* **Restore strategy is more important than backup strategy**
* Testing backup plans would not be a bad idea. If we don't test backups, we don't have them. We must recheck backup/restore plans monthly, quarterly or yearly

### The 3-2-1 Rule

* 3 means: having at least 3 total copies of data

  ```
  - local machine
  - external hard drive/removeable disk device
  - cloud storage
  ```
* 2 means: keep the backed-up data on 2 different storage types

  ```
  - local machine
  - cloud
  ```
* 1 means: having at least 1 copy offsite

  ```
  Even if you have two copies on two separate storage types but both are stored onsite, 
  a local disaster could wipe out both of them. 
  Keep a third copy in an offsite location, like the cloud.
  ```


# Mindset for building HA and scalable system

## Goal: system or infrastructure must have

* Fault tolerance
* No single point of failure
* More than one or two security layers
* Auto-failover without requiring human intervention
* Heartbeat monitoring on all running components
* Infrastructure as code

![kubernetes](/files/-LXPBz8qGcZkET-jysN5)

### 1. Fault tolerance

It is the property that enables a system to **continue operating** properly in the event of the **failure** of (or one or more faults within) some of its **components**. If its operating quality decreases at all, the decrease is proportional to the severity of the failure, as compared to a naively designed system in which even a small failure can cause total breakdown. Fault tolerance is particularly sought after in high-availability or life-critical systems.

* Distributed read/write to MySQL replication cluster
* CDN system like Cloudfront/Cloudflare
* &#x20;Micro-services, separated databases for some big components

### 2. Single point of failure SPOF

A single point of failure (SPOF) is a **part of a system** that, **if it fails**, will **stop the entire system** from working. **SPOFs** are undesirable in any system with a goal of high availability or reliability, be it a business practice, software application, or other industrial system.

* MySQL multi-master - galera cluster
* AWS RDS multi-AZ feature
* Elasticsearch master nodes
* Redis sentinel

### 3. Defense in depth

Defense in depth (also known as Castle Approach) is an information assurance (IA) concept in which **multiple layers of security controls** (defense) are placed throughout an information technology (IT) system. Its intent is to provide redundancy in the event a security control fails or a vulnerability is exploited that can cover aspects of personnel, procedural, technical and physical for the duration of the system's life cycle.

* Cloudflare Anti DDOS layer
* IPtable / AWS secgroup
* VPN
* Snort / Ossec

### 4. Failover

A method of protecting computer systems from failure, in which **standby equipment automatically takes over when the main system fails**. In computing, failover is switching to a redundant or standby computer server, system, hardware component or network upon the failure or abnormal termination of the previously active application, server, system, hardware component, or network. Failover and switchover are essentially the same operation, except that failover is automatic and usually operates without warning, while **switchover requires human intervention**.

* HAproxy / AWS ALB & ELB
* Auto promote on MySQL replication

### 5. Heartbeat

In computer science, a heartbeat is a **periodic signal** generated by hardware or software to **indicate normal operation or to synchronize** other parts of a computer system. Usually a heartbeat is **sent between machines** at a regular interval in the order of seconds. If the endpoint does **not receive a heartbeat** for a time —usually a few heartbeat intervals—, the machine that should have sent the heartbeat is **assumed to have failed**.

* Uptime tools (Monit, Newrelic synthetics, AWS LB healh-check)
* Percona `pt-heartbeat`

### 6. Infrastructure as code

All configuration is defined in executable configuration definition files, such as shell scripts, Ansible playbooks, Chef recipes, or Puppet manifests ...

* Infra & network layer: Terraform, Cloudformation
* Application layer: Ansible playbook, Puppet, Chef, Salt stack


# GitLab database incident

### For your information:

* [Gitlab Database incident](https://docs.google.com/document/d/1GCK53YDcBWQveod9kfzW-VCxIABGiryG7_z_6jHdVik/pub)
* [Discussion on HackerNews](https://news.ycombinator.com/item?id=13537052)

![](/files/-LXOtz3u53HPerQgP5Zj)

### Lessons learned:

* Engineers should get more sleeps
* Restore strategy is more important than backup strategy
* Testing backup plans would not be a bad idea. If we don't test backups, we don't have them. We must rechecking backup/restore plans monthly, quarterly or yearly
* Always careful, anything with `sudo` command, we need to **double/triple check**
* Change terminal PS1 format/colors to make it clear whether you’re using production or staging
  * **RED** for production
  * **Blue/green** for staging
* Show the full hostname in the bash prompt for all users by default (e.g: `db1.staging.gitlab.com` instead of just `db1`)


# Non-technical based


# How to read news efficiency?

### Recognize how toxic news can be

* **News is irrelevant**: the consumption of news is irrelevant to you. But people find it very difficult to recognize what's relevant. It's much easier to recognize what's new. The relevant versus the new is the fundamental battle of the current age. Media organisations want you to believe that news offers you some sort of a competitive advantage. Many fall for that. We get anxious when we're cut off from the flow of news. In reality, news consumption is a competitive disadvantage. The less news you consume, the bigger the advantage you have.
* **News inhibits thinking:** Thinking requires concentration. Concentration requires uninterrupted time. News pieces are specifically engineered to interrupt you. They are like viruses that steal attention for their own purposes. News makes us shallow thinkers. But it's worse than that. News severely affects memory. There are two types of memory. Long-range memory's capacity is nearly infinite, but working memory is limited to a certain amount of slippery data. The path from short-term to long-term memory is a choke-point in the brain, but anything you want to understand must pass through it. If this passageway is disrupted, nothing gets through. Because news disrupts concentration, it weakens comprehension. Online news has an even worse impact. [In a 2001 study two scholars in Canada](http://www.wired.com/magazine/2010/05/ff_nicholas_carr/) showed that comprehension declines as the number of hyperlinks in a document increases. Why? Because whenever a link appears, your brain has to at least make the choice not to click, which in itself is distracting. News is an intentional interruption system.
* **News wastes time.** If you read the newspaper for 15 minutes each morning, then check the news for 15 minutes during lunch and 15 minutes before you go to bed, then add five minutes here and there when you're at work, then count distraction and refocusing time, you will lose at least half a day every week. Information is no longer a scarce commodity. But attention is. You are not that irresponsible with your money, reputation or health. Why give away your mind?

### References

1. <https://www.theguardian.com/media/2013/apr/12/news-is-bad-rolf-dobelli>


# How long should you nap?

According to experts, **10 to 20 minutes is quite enough** to refresh your mind and increase your energy and alertness. The sleep isn't as deep as longer naps and you're able to get right back at your day immediately after waking up. If you nap for **30 minutes you may deal with a 30-minute grogginess** period because you wake up just as your body started entering a deeper stage of sleep. The same can be said if you sleep for an hour, but on the other hand, these **60-minute** naps provide an excellent memory boost. The longest naps— lasting about **90 minutes** —are recommended for those people who just don't get enough sleep at night. Since it's a complete sleep cycle, it can improve emotional memory and creativity.

Reference:&#x20;

1. [https://www.healthspiritbody.com/nap-benefits](https://www.healthspiritbody.com/nap-benefits/)
2. <https://www.economist.com/business/2019/05/18/sleepless-in-silicon-valley>

![](/files/-LltQIxe1VLlAjx07V6F)

May thay, [quân đội Mỹ](https://tinhte.vn/tags/quan-doi-my/) có một mánh nho nhỏ chỉ mất 2 phút để đi vào giấc ngủ sâu. Mánh này đã được đề cập từ năm 1981 trong cuốn sách mang tên “Relax and Win: Championship Performance”, nhưng mới đây tác giả Sharon Ackman đã có một bài viết giới thiệu lại [kỹ thuật](https://tinhte.vn/tags/ky-thuat/) rất đơn giản này. Đối với hải quân, hay quân đội nói chung, khả năng tỉnh táo và trực chiến là thứ quan trọng nhất. Để giúp người lính có thể ngủ mọi lúc mọi nơi, trong mọi trường hợp, kỹ thuật đi vào giấc ngủ này đã được hải quân Mỹ phát triển, và sau 6 tháng thử nghiệm, 96% số người thử nghiệm đều cho biết họ có thể chìm vào giấc ngủ bất kể điều kiện xung quanh.\
\
Làm thế nào mà tài vậy? Những phi công tập sự của hải quân Mỹ được huấn luyện ngủ khi ngồi trên ghế dựng thẳng lưng. Sau khi đã ngồi yên vị, hãy nhắm mắt và tập trung vào khuôn mặt đầu tiên. Anh em cố gắng tập trung vào việc giãn hết cơ mặt sao cho thoải mái nhất, cả ở miệng, má, lưỡi và hàm, thậm chí cả những cơ xung quanh mắt nữa.

Đó là bước quan trọng nhất. Sau đó đến phần còn lại của cơ thể. Sau khi cơ mặt đã giãn, hãy thả lỏng vai và cổ. Nếu làm đúng cách, anh em sẽ thấy toàn bộ sức nạng bắt đầu “trôi” thẳng xuống phần dưới cơ thể. Sau đó tiếp tục tập trung vào phần bắp tay, rồi cẳng tay, kế tiếp là bàn tay và ngón tay. Hãy làm điều này với tay thuận của anh em trước, rồi kế đến là tay còn lại. Khi nửa trên cơ thể đã được thả lỏng hoàn toàn, anh em sẽ cảm thấy rất dễ chìm vào giấc ngủ.

Tương tự như vậy với hai chân. Anh em bắt đầu với đùi và từ từ thả lỏng xuống bắp chân, bàn chân. Mỗi bước, anh em sẽ cảm thấy cơ thể trôi dần xuống mặt đất vì trọng lực. Sau khi toàn bộ cơ thể đã được thả lỏng, hãy để tâm trí “lạc trôi” trong 10 giây, không nghĩ về bất kỳ điều gì cả. Nếu bị stress hay quá căng thẳng, hãy tưởng tượng anh em đang nằm vô cùng thoải mái trên cái nệm thân quen ở nhà. Đó chính là kỹ thuật giúp ngủ ngon của lính Mỹ. Nó không chỉ giúp anh em khó ngủ về đêm mà còn giúp anh em có thể chợp mắt ngắn vào giờ nghỉ trưa ở văn phòng, hay khi ở sân bay, bến xe…

Chúc anh em áp dụng thành công và sống vui khỏe


# Assume good faith

https\://en.wikipedia.org/wiki/Wikipedia:Assume\_good\_faith

**Assuming good faith** (**AGF**) is a fundamental principle on [Wikipedia](https://en.wikipedia.org/wiki/Wikipedia). It is the assumption that editors' edits and comments are made in [good faith](https://en.wikipedia.org/wiki/Good_faith). Most people try to help the project, not hurt it. If this were untrue, a project like Wikipedia would be doomed from the beginning. This guideline does not require that editors continue to assume good faith in the presence of obvious evidence to the contrary ([e.g.](https://en.wikipedia.org/wiki/List_of_Latin_phrases_\(E\)#exempli_gratia) [vandalism](https://en.wikipedia.org/wiki/Wikipedia:Vandalism)). Assuming good faith does not prohibit discussion and criticism. Rather, editors should not attribute the actions being criticized to [malice](https://en.wikipedia.org/wiki/Malice_\(law\)) unless there is specific evidence of such.

When disagreement occurs, try to the best of your ability to explain and resolve the problem, not cause more conflict, and so give others the opportunity to reply in kind. Consider whether a dispute stems from different perspectives, and look for ways to reach [consensus](https://en.wikipedia.org/wiki/Wikipedia:Consensus).

When doubt is cast on good faith, continue to assume good faith yourself when possible. Be [civil](https://en.wikipedia.org/wiki/Wikipedia:Civility) and follow [dispute resolution procedures](https://en.wikipedia.org/wiki/Wikipedia:Dispute_resolution), rather than [attacking](https://en.wikipedia.org/wiki/Wikipedia:No_personal_attacks) editors or [edit-warring](https://en.wikipedia.org/wiki/Wikipedia:Edit_warring) with them. If you wish to express doubts about the conduct of fellow [Wikipedians](https://en.wikipedia.org/wiki/Wikipedians), please substantiate those doubts with specific [diffs](https://en.wikipedia.org/wiki/Help:Diff) and other relevant evidence, so that people can understand the basis for your concerns. Although bad conduct may seem to be due to bad faith, it is usually best to address the conduct without mentioning motives, which might exacerbate resentments all around.

Be careful about citing this principle too aggressively. Just as one can incorrectly judge that another is acting in bad faith, so too can one mistakenly conclude that bad faith is being assumed; exhortations to "Assume Good Faith" can themselves [reflect negative assumptions about others](https://en.wikipedia.org/wiki/Wikipedia:Assume_the_assumption_of_good_faith).

Everyone makes mistakes, both behavioral (such as [personal attacks](https://en.wikipedia.org/wiki/Wikipedia:No_personal_attacks)) and content-based (such as adding [original research](https://en.wikipedia.org/wiki/Wikipedia:No_original_research)). Most of the time, we can correct such mistakes with simple reminders. However, there will be disagreements on Wikipedia for which no policy or guideline has an easy answer. When disagreements happen, ill intent may not be involved. Keep a [cool head](https://en.wikipedia.org/wiki/Wikipedia:Staying_cool_when_the_editing_gets_hot), and consider [dispute resolution](https://en.wikipedia.org/wiki/Wikipedia:Dispute_resolution) if disagreements seem intractable; many of them are not.

Violation of policies—such as engaging in [sock-puppetry](https://en.wikipedia.org/wiki/Wikipedia:Sock_puppetry), violating [consensus](https://en.wikipedia.org/wiki/Wikipedia:Consensus), and so on—may be perpetrated in either good or bad faith. There are processes for dealing with all of these, and [sanctions](https://en.wikipedia.org/wiki/Wikipedia:Blocking_policy) for repeated violation of policy will apply regardless of whether bad faith was involved or not.


# Books


# Sysadmin/SRE

![discussion on vietnamrb slack (May 06, 2019)](/files/-LeCUdIvTvQwdRGXWar9)

### How Linux Works, 2nd Edition

Ngày xưa mới ra trường, chả biết khỉ gió gì, dc anh sếp bảo lấy cuốn này đọc cho hết đi, nhờ đó mà đỡ ngu chút, biết dc chút chút.

### UNIX and Linux System Administration Handbook

Chừng hiểu dc chút chút linux rồi, thì đọc tiếp cuốn này, cũng là cuốn đọc nhiều nhất, lâu lâu bí cái gì là search trong cuốn này ra đọc trước, hiểu khái niệm & concept, xong từ đó mới research thêm. Homepage: <https://www.admin.com>

### Systems Performance: Enterprise and the Cloud

Rảnh rỗi, dư hơi, thì đọc cuốn này của ông kẹ, nhiều chương về cpu, mem, … rất đáng & giá trị

### High Performance MySQL: Optimization, Backups, and Replication

Ngày trước có thời gian làm nhiều với MySQL, nên đọc cuốn này, hay, nhưng hơi cũ, đọc xong phải lấy docs mới nhất của MySQL ra compare & chiêm nghiệm lại


# Mindsets

### *Time Management for System Administrators*&#x20;

Book by Thomas A. Limoncelli

Cuốn này mới đọc gần đây (năm rồi), đem ra nhiều chủ đề rất quan trọng trong việc quản lý thời gian, xử lý task, operation, nhiều ví dụ rất thực tế cho mấy bạn sysadmin/sre đọc (vì tác giả cũng là sysadmin)

### Coders At Work

Meta chút xíu thì có Coders At Work, đọc xong cảm nhận nhiều hơn chút về cách tư duy của các legend hồi xưa.

### The design of everyday things

Cuốn này chắc không cần nói nhiều nhỉ :smile: cái học được nhiều nhất là cách nhìn ở góc độ con người (nghiêng nhiều về tâm lý). Cuốn này đặt ra khá nhiều nền tảng trong công việc của mình sau này.


# Software fundamentals

![discussion on vietnamrb slack (May 06, 2019)](/files/-LeCV8zsMjhffYsXENQm)

### Modern Technical Writing: An Introduction to Software Documentation

Cuốn này tìm thấy trong thư viện của team copy của Anduin. Chỉ có 45 trang nên tính ra khá hiệu quả. Cuốn này là nền tảng để mình viết khá nhiều về technical.

### Human-Computer Interaction

Cuốn này hồi SV được thầy khuyên đọc. Giờ lâu rồi cũng k nhớ rõ mấy topic trong đó, nhưng đại khái cũng thuộc dạng nền tảng với mình.

Công bằng mà nói thì cuốn này đọc lúc SV khá hợp vì nó là dạng college book. 2 cuốn sau pratical/industry focus hơn. Cuốn này như kiểu inspire overview.

### The design of everyday things

Cuốn này chắc không cần nói nhiều nhỉ :smile: cái học được nhiều nhất là cách nhìn ở góc độ con người (nghiêng nhiều về tâm lý). Cuốn này đặt ra khá nhiều nền tảng trong công việc của mình sau này.

### Refactoring UI

Cuốn này đọc gần đây, hơi commercial, nhưng là 1 trong những cuốn về UI engineering viết cho engineer dạng developer hiệu quả (in the mean of rõ ràng, to the point) nhất mình từng đọc. Cuốn này biết do nằm trong thư viện team design.

### The Mathematica Handbook 5, Practical Common Lisp.&#x20;

Hai quyển này giúp hiểu khá nhiều về căn bản lập trình high-level (vs systems programming). Quyển đầu thì nên đọc trong cái built-in doc browser của Mathematica rồi nghịch luôn (nhưng những bản sau này có vẻ tệ hơn). Đọc xong hai quyển này thấy nhiều thứ ở những chỗ khác khá là "primitive" :disappointed:

Linux System Programming, Linux Kernel Development, Programming Rust, Systems Performance, với cái đống documentation cũ của Apple bị nó quăng vào \~sọt rác\~ archive.

### Linux Programming Interface

Mềnh thì là cuốn Linux Programming Interface, mỗi lần đọc lại học dc mớ thứ


# English

![https://news.ycombinator.com/item?id=19825632](/files/-LeFTvkpTCinJCC6ayl2)