Western Mass Hosting's Security Articles

High Performance Wordpress and Server - Part I - Server Setup

I have successfully managed to get around a 1 second load time on my WordPress site, While getting 250 concurrent users over a 1 minute test period. (Source: https://webpagetest.org/result/210128_Di26_15d11aa18013d1df9ebc867e1aa9c2f3/)

This was done with a combination of items, stemming from the server install up to WordPress theme development. Here is how I did it, so maybe you can too.

Server Setup

Here we will start from the ground up. Items you will need: VirtualBox, Ubuntu 16.04 64b Server ISO, Time

My virtual machine is setup with 4G of RAM, using 2 CPU’s, with 80G SSD, and a Bridged Networking adapter

Boot to the ISO, and start the installation process. Everything can be setup how you wish, however, I custom partitioned, as well as, only installed the “standard system utilities”, and OpenSSH during the install process.

During paritioning (with the size above), make sure to select Manual, and setup the 4 Partitions I layout below

Since we are creating a tmp, cache, and swap partition make sure to reserve at least the same amount as you have in RAM, so with my 4G of RAM, I need to reserve at least 12G of disk, however, I am going to reserve 16G because I want my swap partition twice the amount of RAM

• 1st Partition:
  • Mount Point: /
  • 69.9G (officially my drive was 85.9G)
  • Primary – Beginning
  • Mount Options: discard
  • Reserved blocks: 1%
  • Typical Usage: news
  • Bootable: on
• 2nd Partition:
  • Mount Point: /tmp
  • 4G
  • Primary – Beginning
  • Mount Options: discard, noatime, nodiratime
  • Reserved blocks: 1%
  • Typical Usage: news
  • Bootable: on
• 3rd Partition:
  • Mount Point: /cache (probably will have to create this manually)
  • 4G
  • Primary – Beginning
  • Mount Options: discard, noatime, nodiratime
  • Reserved blocks: 1%
  • Typical Usage: news
  • Bootable: on
• 4th Partition:
  • Use as: swap
  • 8G

Now finish up your install process, and let the machine reboot. Once it boots, login to the machine and drop into a sudo session using sudo -s and let the “fun” begin 🙂

We are going to configure our server to use bash only, setup the default system control, install our software, and configure it… so be prepared to have your time sucked up 😉

Use bash only: dpkg-reconfigure dash

Now, we’ll remove apparmor since we’ll be using ufw as our firewall

service apparmor stop
update-rc.d -f apparmor remove
apt-get remove apparmor apparmor-utils

Speaking of firewall, we can set that up now too

ufw allow http
ufw allow https
ufw allow ssh
ufw enable

This will allow only web and ssh connections to the server. Feel free to allow anything else you deem necessary

Now we’ll modify our system controller to allow a ton of connections, allow a ton of files to be open, and mod our networking and swap configuration

First run rm -f /etc/sysctl.conf then nano /etc/sysctl.conf and paste in the following:

# for /etc/sysctl.conf
# Protection from SYN flood attack.
net.ipv4.tcp_syncookies = 1
# See evil packets in your logs.
net.ipv4.conf.all.log_martians = 1
# Discourage Linux from swapping idle server processes to disk (default = 60)
vm.swappiness = 45
# Increase number of incoming connections that can queue up before dropping
net.core.somaxconn = 50000
# Handle SYN floods and large numbers of valid HTTPS connections
net.ipv4.tcp_max_syn_backlog = 30000
# Increase the length of the network device input queue
net.core.netdev_max_backlog = 5000
# Increase system file descriptor limit so we will (probably) never run out under lots of concurrent requests. (Per-process limit is set in /etc/security/limits.conf)
fs.file-max = 100000
# Widen the port range used for outgoing connections
net.ipv4.ip_local_port_range = 10000 65000
# If your servers talk UDP, also up these limits
net.ipv4.udp_rmem_min = 8192
net.ipv4.udp_wmem_min = 8192
# Disable source routing and redirects
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
# Disable packet forwarding.
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
# Disable TCP slow start on idle connections
net.ipv4.tcp_slow_start_after_idle = 0
# Increase Linux autotuning TCP buffer limits Set max to 16MB for 1GE and 32M (33554432) or 54M (56623104) for 10GE Don't set tcp_mem itself! Let the kernel scale it based on RAM.
net.core.rmem_max = 56623104
net.core.wmem_max = 56623104
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.optmem_max = 40960
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
# Disconnect dead TCP connections after 1 minute
net.ipv4.tcp_keepalive_time = 60
# Wait a maximum of 5 * 2 = 10 seconds in the TIME_WAIT state after a FIN, to handle any remaining packets in the network. net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 5 Allow a high number of timewait sockets
net.ipv4.tcp_max_tw_buckets = 2000000
# Timeout broken connections faster (amount of time to wait for FIN)
net.ipv4.tcp_fin_timeout = 10
# Let the networking stack reuse TIME_WAIT connections when it thinks it's safe to do so
net.ipv4.tcp_tw_reuse = 1
# Determines the wait time between isAlive interval probes (reduce from 75 sec to 15)
net.ipv4.tcp_keepalive_intvl = 15
# Determines the number of probes before timing out (reduce from 9 sec to 5 sec)
net.ipv4.tcp_keepalive_probes = 5

After you save the file, run the following to load in the configuration: sysctl -p

Now add in the following line to your limits by: nano /etc/security/limits.conf

# add to bottom
* - nofile 16384
* hard nofile 500000
* soft nofile 500000
root hard nofile 500000
root soft nofile 500000

Now we’ll update the server: apt-get update && apt-get -y upgrade && apt-get -y dist-upgrade && apt-get autoclean && apt-get -y autoremove

Once again, you will need to reboot.

Once you have rebooted, lets forge ahead and install our web/database software:

nGinx & memcached (and a couple extra helpers)

apt-get -y install nginx-full memcached zip lzop

PHP 7

apt-get -y install php7.0-fpm php7.0-curl php7.0-gd php7.0-intl php7.0-mysql php7.0-json php7.0-sqlite3 php7.0-opcache php-memcached php-pear php7.0-mbstring php7.0-cli

MySQL

apt-get -y install mysql-server

We are done installing! 😀 Now the real fun begins… configuration. Since I host a multitude of sites on my server, I setup a directory structure like the following for both site configurations and files. But, you can do what you want… just remember to change your paths in the config files I post here, otherwise it will not work for you.

• /hosting
  • /hosting/DOMAINS
    • /hosting/DOMAINS/the.domain.com
      • /hosting/DOMAINS/the.domain.com/www
      • /hosting/DOMAINS/the.domain.com/fpm-pools
  • /hosting/nginx-config
  • /hosting/site-config

I remove the default nginx config otping for my own, so do a: rm -f /etc/nginx/nginx.conf && echo “include /hosting/nginx-config/nginx.conf;” > /etc/nginx/nginx.conf

Now download the following and unzip it to your /hosting/nginx-config/ directory: DOWNLOAD HERE

You are now ready to setup your first site. Run this and change the domain to the domain of your need: mkdir -p /hosting/DOMAINS/example.com/www && echo “<h1>Hello World</h1>” > /hosting/DOMAINS/example.com/www/index.php

We also need to configure the running of your site through fpm and nginx, copy and paste the following to /hosting/site-config/example.com make sure to change the paths to fit your needs, as well as the domain.

upstream your-fpm-lb {
	# PHP-FPM - make sure the ports you decide to use are open
 # You should at least have 2 pools available to you. But really no more than 4 is necessary
	server 127.0.0.1:11411;
	server 127.0.0.1:11412;
	server 127.0.0.1:11413;
	server 127.0.0.1:11414;
}

# Redirects
server {
	# what we want to redirect
	server_name www.example.com;
	# where we want to redirect to
	return 301 http://example.com$request_uri; 
}

server {
	# the document root of your site
	root /hosting/DOMAINS/example.com/www;
	# the default page for your site
	index index.php;
	# the main fqdn of the site
	server_name example.com;
	# your access log location. I leave this commented for performance
	#access_log /logs/example.com-access.log;
	# your error log location, i only enable critical errors to log - performance
	error_log /logs/kevinpirnie.com-error.log crit;
	# let's setup the php fpm processor
	location ~ [^/]\.php(/|$) {
		# let's turn on the keep alive
		fastcgi_keep_conn on;
		# include our default fastcgi configuration: see file for details
		include /hosting/nginx-config/site-fastcgi-common.conf;
		# set to your fpm upstream above
		fastcgi_pass your-fpm-lb;
	}
	# Configure memcached to be usable. See file for details 
	include /hosting/nginx-config/memcache-enabled.conf;
	# Configure caching. See file for details
	include /hosting/nginx-config/yes-cache.conf;
	# Configure no caching. See file for details
	#include /hosting/nginx-config/no-cache.conf;
	# Configure default site settings, Required. See file for details
	include /hosting/nginx-config/all-sites.conf;
	# Configure gzipping of static resources. See file for details
	include /hosting/nginx-config/gzip.conf;
	# Configure some extra security for WordPress sites. See file for details.
	include /hosting/nginx-config/wp-security.conf;
}

One last bit of configuration. We need to setup our fpm pools. Since we configured 4 upstream connections in our site config, we need to configure 4 pools

Create a new user for these pools to run under

adduser -y example-user

Copy and paste the following into 4 files located in /hosting/DOMAINS/example.com/fpm-pools

; Start a new pool
; make sure to update the number
[example1]
; what user should this pool run as
user = example-user
; keep this www-data so nginx can serve the site
group = www-data
; change this to reflect one of the ports in the upstream block of your site config
listen = 127.0.0.1:11411
; We don't need to have too high of a task priority
process.priority = 0

; fpm process management
pm = dynamic
pm.max_children = 200
pm.start_servers = 20
pm.min_spare_servers = 20
pm.max_spare_servers = 60
pm.max_requests = 500

We can now start our engines 🙂 Run this command to make sure you haven’t messed up anything 😉 nginx -t then run the following to restart all of your services to start hosting your site.

/etc/init.d/memcached restart
/etc/init.d/php7.0-fpm restart
/etc/init.d/nginx restart

Now that we have the “basics” out of the way, let’s head into configuring MySQL to handle the loads we are going to place on it. Copy/Paste the following into your mysqld.conf (usually located at /etc/mysql/mysql.conf.d)

[showhide type=”mysql” more_text=”Show mysqld.conf” less_text=”Hide mysqld.conf”]

#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
# 
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

[mysqld_safe]
socket		= /var/run/mysqld/mysqld.sock
nice		= 0

[mysqld]
#
# * Basic Settings
#
user		= mysql
pid-file	= /var/run/mysqld/mysqld.pid
socket		= /var/run/mysqld/mysqld.sock
port		= 3306
basedir		= /usr
datadir		= /var/lib/mysql
tmpdir		= /tmp
lc-messages-dir	= /usr/share/mysql
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address		= 127.0.0.1
#
# * Fine Tuning
#
key_buffer_size = 128M 
max_allowed_packet = 16M 
thread_stack = 128K 
thread_cache_size = 8 
table_open_cache = 8192 
max_heap_table_size = 256M 
innodb_file_per_table
innodb_buffer_pool_size = 4G
myisam-recover-options = BACKUP 
innodb_log_file_size = 512M
skip_name_resolve

#table_cache = 64
#thread_concurrency = 10
#
# * Query Cache Configuration
#
query_cache_limit	= 4M
query_cache_size = 1024M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
#log_slow_queries	= /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id		= 1
#log_bin			= /var/log/mysql/mysql-bin.log
expire_logs_days	= 10
max_binlog_size = 100M
#binlog_do_db		= include_database_name
#binlog_ignore_db	= include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem

[/showhide]

For the most part we are now done and ready to start serving up scalable, efficient, and fast loading WordPress websites. Just the config above alone is not enough to ensure high availability though. There is alot more work to be done, mostly with theme development.

Just like any other web application, proper development goes a long way. Design as well, if your app’s design isn’t optimized, you will still create a heavy load on the server, that is un-necessary. Always optimize your images, always set the dimensions when you call them as well. Make sure you concatenate and minify your css and javascripts where you can, and load them appropriately in your html code. (css in the head, keep as much javascript at the bottom of your documents as well).

That’s it for now, stay tuned for the rest in this series. And as always… Happy Coding 🙂